Vision-based Manipulation from Single Human Video with Open-World Object Graphs