When it comes to innovation, keeping on top of the deluge of trends that are emerging around the world is an ongoing challenge.  When you have a day job it’s incredibly difficult to find the time, hence why an industry has grown up of professionals who aim to do this for leaders.

Of course, the success of this industry largely rests on the ability of people whose job it is to follow the trends to do just that.  My own efforts to do just that typically involve following the output of all major universities in the western world, together with some innovative big and small companies, with a sprinkling of state led innovation bodies thrown into the mix.

Whilst I believe that gives me a decent chance, the reality is that with 4,000 academic papers published every day (or 1.5 million a year if you prefer) I am merely scratching the surface.

Automating horizon scanning

We’ve seen services like IBM’s Watson emerge in recent years to try and digest the vast amount of new knowledge in fields such as medicine to help clinicians stay on top of the deluge, but there are also projects looking to trawl the literature for glimpses of the future.

Enter a new service developed by SRI International and brought to market by Meta to automate much of this horizon scanning work.

The service trawls the academic literature from over 17 million academics and uses things like trajectory mapping to underpin its AI engine and provide users with what it believes are the hot trends in any given field.

“As a researcher who has seen the power of technology to uncover new insights about cancer and other diseases, the potential for Meta to have an impact on society is large. Meta’s AI accelerates science globally by shortening the path to knowledge at major bottlenecks where human efforts alone can no longer scale,” co-founder Sam Molyneaux told me recently.

Signs of progress

For instance, the service was used in 2013 to provide a number of hot areas in biomedicine, with a score of 3.0 or higher meaning that the term was likely to double in activity in the next three years (ie for now).

1. A deeper understanding of the function of the human genome, especially around long noncoding RNAs
2. Deubiquitinating enzymes (DUBs) as potential therapeutic targets
3. The role of Von Hippel-Lindau proteins in the prevention of metastatic tumors
4. Decline in shRNAs and DNA microarrays
5. CRISPR will continue to flourish in 2016

The platform mines 25 million research papers from 17 million different academics from over 340,000 institutions.

This rich dataset allows their algorithm to determine the trajectory of themes, and therefore hopefully provide strong insight into whether a topic is thriving or declining.

“As we worked with diverse groups throughout the scientific and technical ecosystem over the last 5 years, we learned many of the ways Meta can help them address their most critical questions and challenges. From enabling scientific awareness and knowledge exploration, to anticipating where research is going globally, Meta is truly AI in the service of science,” Molyneaux continues.

Over the horizon

It’s certainly a fascinating platform.  At the moment the results are very much focused around the medical field, and whilst it undoubtedly does a fine job of filtering papers, there still remains a challenge in translating findings for a mass audience.

At the moment, users are forced to read the entire paper, with little in the way of support in telling the user how papers are linked or why a particular study is so important.  Given developments around automated writing however, this is certainly an area for further development.

Likewise, the researcher tools could use a bit of context to help the user make sense of things.  For instance, a search for ‘Crispr’ returns Jennifer Doudna high up, but doesn’t say why she’s so important to the field.  It also doesn’t tell the user of the patent dispute over Crispr between Doudna and MIT’s Robert Desimone, which may influence who you wish to approach to work on Crispr.

Given that people may well wish to go straight to the expert rather than digest the knowledge themselves, this is something to work on I feel.

The Meta team have certainly hit on a very real problem as individuals and organizations struggle to digest the huge amount of information coming at them every day.

I’m sure the platform will be developed further in the coming months, and it will be a valuable tool in keeping on top of the deluge of new research being published each day.

If this is your field, it’s certainly one to check out.