Researchers have made tremendous progress in recent times in the ability of machines to understand the nuances of our language.  For instance, we’ve seen machines developed that are capable of understanding things such as empathy, sarcasm, humor and a range of other emotions.

This understanding has even been compiled together to have machines automate the speech writing process (although the speaking part remains a challenge).

The next clue

A recent international team believe that a simple crossword puzzle can help us to develop robots with even greater language processing capabilities.  Their recent study hypothesizes that if we can develop machines capable of understanding crossword clues, then it will go a long way to helping them understanding the complexities of human language.

The system has been placed online, and uses neural networks to try and decipher crossword clues better than both existing systems but also human crackers.

The system had its wits pitted against some of the best commercial software on the market, and proved better at answering single word clues, and also those containing short combinations.

Training the brain

The system was trained using Wikipedia and a collection of half a dozen dictionaries so that it was capable of understanding both words and sentences.  What’s more, this understanding enabled it to connect up words with sentences by developing an appreciation of how and when words are used.

“Over the past few years, there’s been a mini-revolution in machine learning,” the researchers say. “We’re seeing a lot more usage of deep learning, which is especially useful for language perception and speech recognition.”

Dictionaries provide a great resource for such deep learning as they come with so much data.  Couple this with huge repositories such as Wikipedia and you have an invaluable resource for improving automated language processing abilities.

Going deep

“Dictionaries contain just about enough examples to make deep learning viable, but we noticed that the models get better and better the more examples you give them,” the team say. “Our experiments show that definitions contain a valuable signal for helping models to interpret and represent the meaning of phrases and sentences.”

The project is crucial because it highlights the growing capability of machines not only to understand individual words, but also the way words connect together.

Machine learning remains the most commonly used approach to tackling this task, so the team hope that their approach will inspire others in their own work.

“Despite recent progress in AI, problems involving language understanding are particularly difficult, and our work suggests many possible applications of deep neural networks to language technology,” they say. “One of the biggest challenges in training computers to understand language is recreating the many rich and diverse information sources available to humans when they learn to speak and read.”

Suffice to say, despite the promising beginning, the team accept the limitations of their current work, particularly in understanding the intentions of users.  Understanding this context is often crucial to understanding the intention of the question, especially when non-verbal signals are added to the mix.

It’s a problem that driverless car developers are also grappling with as they attempt to develop machines capable of detecting the intention of other road users from as little as their eye movements.

It’s something that most humans take for granted, but is fiendishly difficult to build into an automated system.  Projects such as this do highlight the progress being made however, and it will be fascinating to see where the team go from here.