What’s the difference between a Roomba and The Jetsons robot maid, Rosie? It’s not a joke, it’s a challenge University of Maryland doctoral student Seungjae “Jay” Lee is tackling in hopes of training real-world domestic robots to master housekeeping tasks.
His work focuses on incorporating the vast reservoir of web videos to train robots by having them watch human activity.
“People often focus on designing better model architectures, but for artificial intelligence that integrates AI algorithms into physical systems, the real bottleneck is the dataset itself,” Lee said in a story on the Maryland Today website.
Training robots to perform physical tasks is much more complicated than training artificial intelligence to write a paper. Not only does the robot need to manoeuvre and manipulate household objects in three dimensions across possibly messy environments, but data on things like grip strength and technique are critically limited, Lee said.
Robots require tactile feedback, sensor readings and action trajectories collected from real-world settings. Gathering this data is slow, expensive and technically demanding.
A Georgia Tech doctoral student is using first-person videos created in his lab to train robots. Simar Kareer recorded himself repeatedly folding a shirt and doing other tasks while wearing virtual reality headsets. Then, he constructed a humanoid robot with pincers for hands and attached the headset to the top to mimic a first-person viewpoint. If enough people use virtual reality headsets while performing their routine tasks, a virtual household chore dataset will emerge, he said.
Lee thinks the dataset may already exist in publicly available videos.
He is leading the development of TraceGen, a system that mines hundreds of thousands of publicly available videos from large datasets and platforms such as YouTube to extract useful hand and object motion.
Another project he is involved with, “Imagine, Verify, Execute,” allows robots to explore the world and learn from their own explorations. – Baltimore Sun/Tribune News Service
