AI has been playing a growing role in the legal industry in recent years, but this has primarily been in the background doing e-discovery type work that involves trawling through large numbers of past cases to find relevant precedents and other such research.
A team of researchers have recently developed a system that promises to offer more ‘front of house’ services however. The researchers, from UCL, the University of Pennsylvania and the University of Sheffield, developed a system that could predict the decisions made by the European Court of Human Rights (ECtHR) with an accuracy of 79%.
The system is believed to be the first of its kind in the world, and arrived at it’s verdict simply by absorbing the text in the case files.
Decision support
“We don’t see AI replacing judges or lawyers, but we think they’d find it useful for rapidly identifying patterns in cases that lead to certain outcomes. It could also be a valuable tool for highlighting which cases are most likely to be violations of the European Convention on Human Rights,” the researchers say.
As the system delved inside the decision making processes of judges, it interestingly emerged that judgements are often based on non-legal facts as much as they are purely legal arguments. Of course, this isn’t suggesting the decisions are poor so much as they are based on reality rather than strict theory.
Of course, this insight into the reasoning processes of judges will undoubtedly help to both refine this particular model but also the thinking of other project teams undertaking similar work, especially if they gain access to more data.
The data used to train the algorithm was via the public database of the ECtHR, with nearly 600 cases extracted around Articles 3, 6 and 8 of the Convention, with a 50/50 split between violation and non-violation cases.
“Ideally, we’d test and refine our algorithm using the applications made to the court rather than the published judgements, but without access to that data we rely on the court-published summaries of these submissions,” the authors say.
Guilty or not guilty
When the data was analyzed, it emerged that the most reliable factor in gauging whether or not a violation would be determined or not was the kind of language used in the case text, with factors such as circumstances and topic area not far behind. When taken together, these three factors allowed the algorithm to accurately predict the outcome of the case with an accuracy of 79%.
“Previous studies have predicted outcomes based on the nature of the crime, or the policy position of each judge, so this is the first time judgements have been predicted using analysis of text prepared by the court. We expect this sort of tool would improve efficiencies of high level, in demand courts, but to become a reality, we need to test it against more articles and the case data submitted to the court,” the authors reveal.
Suffice to say, it’s unlikely that we’ll be having automated judges any time soon, but with automation already making inroads into the legal industry, it would be no surprise to see this kind of application being used internally.
