The effectiveness of administrative agencies understandably varies considerably. A recent study from Bocconi argues that success largely hinges on the bureaucratic reputation they enjoy. The researchers propose a new measure for determining such a reputation that works across agencies and even across countries using Natural Language Processing.

The approach revolves around the word embedding technique, which computes the likelihood that two or more groups of words might occur in the same linguistical context. In this instance, the researchers were particularly keen to focus on words symbolizing reputation and efficiency.

Measuring reputation

The researchers measured over 450 agencies across the US and UK over a 40-year period via parliamentary speeches made by members of Parliament and Congress respectively. The results produced by this process were then parsed through additional validation tests.

They showed that credibility ebbed and flowed during particular moments in time. For instance, the British Home Office suffered after the Windrush Scandal, while the Environmental Protection Agency has suffered an ongoing decline over the last 30 years.

The analysis found that agencies were most likely to be praised by the governing party than by the opposition, although this gap did narrow when the agencies were independent.

Political affiliation

The analysis also found that the index of reputation was reliant in large part on the political affiliation of the speaker. The nature of discourse on any given agency was often conditional on the party affiliation of the politician. This was especially so in the US, where partisan divides were commonplace.

It appeared common for those in power to speak better of agencies, with this especially so if the agency was under government control. This was further evidenced by the fact that more independent agencies weren’t so much at the mercy of the governing party in terms of discourse about them.

The researchers believe that their index provides an important tool to measure institutional reputation. They hope to further assess whether reputation affects how agencies behave and communicate with the public, and indeed influences the level of support they enjoy.