A few years ago I touched upon the growing trend of automation in journalism, with startups such as Narrative Science leading the way in autonomously producing things such as stock updates and sports results.

This isn’t the only part of the journalists job that has been automated however.  A recent paper outlines how Reuters uses AI to identify breaking news stories.

Tracing the news

The system Reuters use is called Tracer, and it autonomously trawls the web to identify breaking stories.  The system relies upon a combination of data mining and machine learning to identify what it believes to be the most important and relevant events, before then categorizing them into topics and ranking them by importance.  It then autonomously creates a headline and a summary for them, before distributing them around the news wire.

The system first examines around 12 million Tweets per day, which represents around 2% of the total volume.  These are a combination of tweets from carefully curated accounts and a random sample of the general public.

Tracer then identifies when the topical event has occurred, which it does by analyzing the Twitter stream to spot when a single event is being spoken about by multiple people at once.

Suffice to say, this uncovers a lot of rubbish, so it next tries to classify the events that they identify and rank them in order of importance.  The system is also capable of identifying the location of the event.

Determining authenticity

The final stage before the summary is written is to determine the veracity of the story.  Tracer first looks for the earliest mention of the topic on Twitter, and any sites that are linked to in those mentions.  It cross-references this against a database of known fake news websites, together with satirical sites such as The Onion.

If they’re happy with the authenticity of the story, then Tracer will produce a headline and summary from its findings.  The paper says that the system performed well during initial trials, both in its ability to provide timely analysis, but also a reliable commentary on the breaking news of the day.

Will the system eventually replace the 2,500 journalists who currently generate the 3,000 news alerts Reuters produce each day?  It’s too early to say, but the experiment does nonetheless highlight the potential for technology to do significant pieces of work.

Robo-news

With the rise in ‘robo-journalism’, it’s perhaps not surprising that thought has been given into how people perceive stories written by machine.

The researchers presented several hundred participants with an article on one of a range of topics.  Half of the participants were told the article was automatically generated, with the other half told it was written by a human journalist.

Interestingly, our preference for automation seemed to differ depending on the topic of the article.  Readers preferred the robot when the article was about finance, but the human when the topic was healthcare.

“It seems that we might not be as comfortable with robots delivering news related to health,” the authors say. “We suspect that this was because of an ‘eeriness’ or a creepy feeling the participants felt, and our results backed this up.”

Time will tell just how far Tracer like applications go, and indeed how the public will respond to them.  It is undoubtedly another aspect of an increasingly interesting trend however.