For decades now, technology has been marketed as the ultimate labor saving device that will free us from the mundane drudgery of daily life and enable us to do things we really enjoy.  Most of those promises have fallen flat as we simply fill our time with other, equally mundane tasks.

It’s something that researchers continue to pursue however, with MIT’s CSAIL lab recently developing an activity simulator that they hope will eventually be able to train robots to complete household chores, such as setting the table.

The system has been trained on around 3,000 programs of various activities, which have been divided into subtasks to make it easier for it to understand.  For instance, a task such as making coffee could include subtasks like ‘grabbing the cup’.

The VirtualHome system, which has been demonstrated in a 3D virtual world akin to that found in Sims, can execute around 1,000 distinct actions in eight different domestic environments.

“Describing actions as computer programs has the advantage of providing clear and unambiguous descriptions of all the steps needed to complete a task,” the authors say. “These programs can instruct a robot or a virtual character, and can also be used as a representation for complex tasks with simpler actions.”

Household chores

Whilst household chores are relatively straightforward for humans, they are immensely complex for robots as they contain a large number of sub-tasks that we largely take for granted.

To help the robot accomplish these tasks, the team compiled descriptions of various household tasks and translated them into code that had all of the various steps involved in each task.  These were then fed into the VirtualHome 3D simulator to turn them into videos, whereby a virtual agent would complete the tasks as prescribed.

The team hope that not only with VirtualHome provide a means of training robots to do useful things, but also a large database of household chores described using natural language.  They believe this data could be useful for companies like Amazon for use with Alexa.

“This line of work could facilitate true robotic personal assistants in the future,” the researchers say. “Instead of each task programmed by the manufacturer, the robot can learn tasks just by listening to or watching the specific person it accompanies. This allows the robot to do tasks in a personalized way, or even some day invoke an emotional connection as a result of this personalized learning process.”

The next step is to train the robots on actual video footage rather than the simulation videos that they’ve used thus far.  The ideal scenario would be to develop the robots so that they can learn from YouTube style videos.

“You can imagine a setting where robots are assisting with chores at home and can eventually anticipate personalized wants and needs, or impending action,” they explain. “This could be especially helpful as an assistive technology for the elderly, or those who may have limited mobility.”

Suffice to say, the technology is still some way from the market, but it’s an interesting example of the developments in this field.  It will be fascinating to see how they evolve from here.  You can see the VirtualHome simulation in the video below.