As a cyclist, I have a wide range of different size alum keys and associated tools, so if someone were to ask if I can pass them one of them, I would seek clarification as to exactly which one they wanted.

A team from Brown University have now developed an algorithm designed to help robots do the same thing.

“Fetching objects is an important task that we want collaborative robots to be able to do,” the team say. “But it’s easy for the robot to make errors, either by misunderstanding what we want, or by being in situations where commands are ambiguous. So what we wanted to do here was come up with a way for the robot to ask a question when it’s not sure.”

Speech and gestures

The team wanted to borrow from human behaviors, by developing ways for the robots to ask for an object (and understand requests from humans) via a combination of voice and gesture.  Even that runs into complications when there are numerous types of wrenches or alum keys.

“What we want in these situations is for the robot to be able to signal that it’s confused and ask a question rather than just fetching the wrong object,” the researchers say.

The algorithm described in the recently published paper attempts to do just that.  It allows the robot to seek clarification over the exact tool the user wants, and only when the certainty levels are high will the robot hand over the desired object.  Up until that point, the robot will make an educated guess and then seek clarification.  Crucially though, these interactions are only undertaken when required, so the robot displays a degree of intelligence in its questioning.

“When the robot is certain, we don’t want it to ask a question because it just takes up time,” the researchers say. “But when it is ambiguous, we want it to ask questions because mistakes can be more costly in terms of time.”

Simple yet useful

The system is currently only asking relatively simple questions, but despite this, it is still able to infer useful information.  It utilizes a technique known as implicatures to make smart inferences about the correct object to grasp.

The system was put through its paces with a number of volunteers asked to engage with the Baxter robot used for the experiment.  They asked the robot for a number of objects under various conditions, with the robot designed to ask a varying number of questions to test different approaches.  The experiment showed that the intelligent use of questions as designed by the algorithm was most effective in terms of both speed and accuracy.

The next step is to combine the current algorithm with enhanced speech recognition capabilities to hopefully further improve the accuracy and speed of the system.  The ultimate goal is to help ensure robots are more collaborative, whether in the workplace or even the home.

Check out the video below to see the system in action.