Earlier this year Reuters released the latest version of their league table to rank the most innovative universities and research institutions in the world.  The rankings, which were produced in partnership with Clarivate Analytics, are based on proprietary data and analysis of things such as number of patent filings and research paper citations.

Such assessments are crucial because they often play a big role in determining grant funding, tempting hotshot researchers and various other things to showcase the prestige of the institution.  It’s pretty important therefore, to find a fair and accurate method for doing so.  It’s a challenge tackled in a recent paper.

Allocating credit

The approach revolves around something the team refer to as the credit allocation method (CAM), which they use to rank research institutes according to a wide range of factors.

“Different from other metrics based on citations, our work considers the citation network structure and provides one way to rank the credit for research institutes for different research fields from the viewpoint of academic reputation,” the team say.

The core concept is that each directed network consists of a single, randomly chosen paper that can be linked to all of the other papers that have in turn cited it.  These are then linked to the other papers that they cite, but only providing they have at least one author from the same institute as the original paper.  Got it?

The team believe that by ranking each research institute, so that those referenced in the first paper receive more than those further down the line, it’s possible to allocate the credit each institute should receive for the original paper.

It sounds complicated, but after doing this across around 500,000 papers, the team believe that they can fairly rank the 19,000 research institutes that helped compile those papers.

The final rankings produced some interesting differences compared to existing league tables, such as the Reuters one highlighted above.  For instance, the top four in the table were the University of California, Bell Labs, the Max Planck Institute, and MIT.  Of those, only the Max Planck Institute made the top 20 in the Reuters list.

It will be interesting to see whether they plan to roll the method out more widely, but for now, they hope to compile lists for specific topics.

“In future work, we also plan to investigate the citation networks of some specific research fields, such as management science, complexity, statistical physics, and computer science, in order to rank the research institute credit of these fields,” they say. “We also plan to develop a website to publish the ranking results for researchers all over the world.”