As the volume of misinformation continues to rise, the ability to fact check what we consume is more important than ever. A couple of papers from MIT’s Computer Science and Artificial Intelligence Laboratory highlight the latest developments in the field.
The first paper showcased the limitations of many of the fake news detectors on the market today, both in terms of detecting falsely generated text, but also in falsely condemning human authored works as fake.
The researchers propose using the abundance of human-written text and automatically corrupting it to alter its meaning using a GPT-2 language model to ensure that the edited text still makes a degree of sense.
“There’s a growing concern about machine-generated fake text, and for a good reason,” they explain. “I had an inkling that something was lacking in the current approaches to identifying fake information by detecting auto-generated text — is auto-generated text always fake? Is human-generated text always real?”
Focusing on the right things
The team tested their model using a simulated attack whereby an article about NASA was augmented with information about how the data collected by the scientists was useful to our understanding of the Earth’s magnetic field. Despite the text making sense, the team were nonetheless able to detect that it was fake, unlike the leading commercial tool on the market.
“We need to have the mindset that the most intrinsic ‘fake news’ characteristic is factual falseness, not whether or not the text was generated by machines,” the team say. “Text generators don’t have a specific agenda – it’s up to the user to decide how to use this technology.”
They argue that whether a text is auto generated is largely irrelevant, especially as both auto-generating tools are becoming more competent and more widely used in legitimate news organizations. Instead, we should focus our efforts on whether the text is ultimately correct or not.
Biased data
The second paper helps towards this endeavor by working to better understand the biases that influence our verification of facts. The team used the Fact Extraction and VERification (FEVER) database to develop a system capable of detecting false statements.
The FEVER database, which is the largest of its kind, has long been used by machine learning researchers as a reliable source of true and false statements, with each backed up by evidence from Wikipedia. The team were able to uncover a high degree of bias in this dataset however, which could significantly impact the output of any AI-based systems trained on it.
“Many of the statements created by human annotators contain give-away phrases,” says Schuster. “For example, phrases like ‘did not’ and ‘yet to’ appear mostly in false statements.”
For instance, it’s quite probable that systems trained on FEVER data could view negated sentences as more likely to be fake, even if they were actually true. This is because models often focus on the language of the claim rather than the evidence supporting it.
Time insensitive
This lack of real evidence base can also prevent systems from factoring in time-related issues in its deliberations. For instance, until 2019 it was true to say that Olivia Coleman had never won an Oscar, but of course that is now false and can be easily verified.
The researchers developed a system that aims to correct the biases that riddle the FEVER database, before then testing current models on their new data. They found that this significantly affected the accuracy of the models, which fell from 86% to just 58%.
“Unfortunately, the models seem to overly rely on the bias that they were exposed to, instead of validating the statements in the context of given evidence,” the researchers say.
The team developed a new model that utilizes the new dataset that was capable of outperforming existing systems across pretty much all metrics. The team hope that their work will help to improve fact-checking services and render them more robust against potential attacks.
