The news has been awash in the past month or so with the tragic stories emanating from the Mediterranean, whereby thousands of refugees are being transported from Africa into Europe, most of whom suffer the voyage in terrible conditions, with a good number failing to survive the trek.

Alas, when it comes to human trafficking, that is merely the most visible tip of the iceberg, with estimates of up to 800,000 people per year trafficked across international borders.

Using big data tools to help

The scale of the problem can often be overwhelming for law enforcement agencies, especially online when it can be akin to trying to find a needle in a very large haystack.

A team from USC have turned the problem into a big data challenge, and created a tool that sifts through escort ads before dumping the information into a database that is more easily searchable.

The tool, which the team have called DIG (or Domain-specific Insight Graphs if you prefer) offers law enforcement officers a means to search for missing children recruited into the escort industry.

“The internet contains seemingly limitless information, but we’re constrained by our ability to search that information and come up with meaningful results. DIG solves that problem,” the developers say.

The tool has been designed to require no training in order to use, and it currently has some 2 billion records stored, with around 5,000 web pages being added every hour.

“As the database continues to grow, DIG will be able to uncover new connections and patterns in the data, making it even more useful,” the team say.

The code for DIG has been made open source, so hopefully it will receive strong take up from law enforcement agencies, and maybe updated also by third parties.

The team themselves are planning quarterly updates of the tool, with the ability to automatically flag potential victims one feature in the pipeline, whilst there are also plans to identify trafficking rings purely through their ads.

It’s a nice example of how big data tools can be used for good, and it will be an interesting project to follow in the coming months.