Estonia is developing a project to issue rulings through AI judges. How is the pilot working so far?
The AI judge is part of a broader process of automation, impacting different functions across the Government. In particular the AI judge pilot is in the early phases, and the Ministry of Justice leads the project. However, I can explain you which are the next stages the Government has in mind when it comes to AI.
I will start from the beginning. Right now, we have an AI task force in place; a working group. Actually, in a couple of weeks we’ll present our AI strategy; which has a real practical approach. When I say practical approach, it is because Estonia is one of the leading countries regarding applied AI when it comes to public sector.
In Estonia, we have 16 AI use cases live in the public sector. I think similar numbers are hard to find elsewhere. Our Government has an ambitious goal of having 50 AI use cases live by 2020.
We actually see that AI enables us to make processes more efficient and effective which we were unable to do before. Therefore, the Government is extremely interested in increasing the support on those cases. AI is one part of a wider approach for the Government to have automatic services – ‘life event services’.
For instance, in future when a child is born, the child will automatically be registered in the kindergarten waiting list and receive Government grants and benefits, without implicitly applying for them. Therefore, it will not be necessary to do any paperwork or go through any manual online systems. We would eliminate the human aspect from processing these decisions.
I guess we’re looking for a similar approach in the jurisdictional scene as well. The first one; which is commonly called robot lawyer, is in our scope as well. Although, I wouldn’t actually call it robot lawyer: it’s just process automation using AI.
How will the AI judge project work?
So right now, I can talk about what we, the Government, has in place.
I think the most important thing is that we talk about automating order for payment procedures.
Order for payment procedure is a procedure that enables the creditor to obtain the execution document faster with from small to no evidence. These procedures matter varies from country to country but its essence remains the same. These proceedings are also referred as small claims procedures in the US or “procesos ejecutivos” in Spain.
We are talking, for example, about parking tickets or child benefit cases, that claim up to 6,400 euros. Under these terms, we have around 32,000 cases every year. That’s why we need to automate the process and make it easier for: small and medium size companies, one parent families which should get child benefits or parking tickets related claims. Making the process automatic is crucial; the human element has to disappear from these procedures. However, there is a lot of people (more than 30 working in this aspect) whose job is to help judges make faster decisions. I think the most important aspect is that the proceedings are fact based; facts given in the claim are controlled. Therefore, we can exclude the uncertainties and complexities that arise from generating AI rulings that require the assessment of evidence.
How does the Government plan to deal with errors in resolutions issued by AI judges?
Humans always have the possibility to challenge the final decision. Everybody has the right to challenge the claim. In the end, it doesn’t really change the process. If you don’t agree with the final decision there’s no problem; it’s not the final decision. Although the payment procedure for instance is already half-automated, we want to automate it even further.
And how do you automate it even further?
It’s important to highlight that our Government is extremely digitalized. We don’t have to start from scratch and we can use our existing systems.
Therefore, if you fill in the claim form we check it against some measures and go through the facts you have given us to control whether they fit the set out requirements.
Can AI tools be applied in proceedings other than small claims?
Right now, it has been discussed, that we start with payment procedures and eventually extend the application of AI rulings to proceedings with higher monetary thresholds. We’ll start with smaller cases and depending on the confidence people give us; we’ll think on applying it to cases with higher threshold.
If you ask whether this system can be applied to cases elsewhere we haven’t considered it. However, we are using kind of a similar logic in different areas; for example, in unemployment agents’ decisions.
We started using our previous experience, for example, how long people have stayed on their jobs and, with that data, we can actually provide recommendations with a 13% higher effectiveness rate rather than a recommendation a Government employee (without the data analysis) would give. For us, effectiveness means having a 72% of people employed six months after the AI recommendation.
This helps to improve our business environment and the economy as a whole: as less people are unemployed and SMEs will actually get money faster from those companies that owe them.
Has the Government experienced any backlash when implementing AI driven systems, for instance regarding the future of jobs?
Actually, the interesting part is that in Estonia, on a wide scale, we don’t see people fear losing their jobs. At the same time, we can see that AI can actually help you find new jobs and at the same time enhance people’s current skills. For instance, in future we could give people a recommendation when it is likely that they could lose their job so they could reeducate themselves.
AI will definitely replace some jobs; we can already see it in the Government. For example, for farmers we mainly use satellite images to detect whether farmland has been mowed or not. Within the first year we were able to save 60,000 euros by reducing the number of on-spot checkups.
Focusing on the legal tools Estonia is developing. How do you train AI judges?As you know and as I mentioned before, we haven’t started off yet.
We have done a pre-analysis of the AI judge project; the judicial system is right now already half automated.
In the legal field, unlike any other economic or scientific area, the rules upon which a software is trained are subject to change, which may render useless the data sets used for training purposes. What do you do then?
I think it’s important that the whole system itself is not based only on machine learning. We still have behind these many rules based on decision making. I keep telling to every interested party that you need to have a process; we need constant progress.
AI is not a one implementation project. In the Government, as well, we always say it’s a continuous work. You need to be sure that the business rules are still valid and that there are no errors. For instance, carry out a sanity test; you have to double check whether AI outcomes or recommendations make sense.
There are so many different aspects to consider with every project. How do you evaluate every project? Which is the outcome? How can you make sure the project you thought has succeeded?
We need to make sure our data quality reflects our actual decisions. Quality data is fundamental. Otherwise we’ll not be sure the outcome matches what we expected.
Biases are one of the most important variables that we’re studying right now. For example, we’re looking at our Canadian partners which are generating algorithmic impact assessments. Every time an organization wants to implement machine learning they should go through benchmarking or questionnaires to make sure if there’s any bias in it.
Our Government is trying to support any progress referred to data quality because we think it’s the basis.
What has been your greatest challenge when trying to merge these AI tools with legal process?
Actually, I’m not sure whether there has been any big challenges. Outside Estonia these questions have raised the alarm a lot more than in our people. Actually, our citizens are much welcoming to automating processes.
Estonian people are not afraid of losing their jobs and are also quite comfortable with ICT. They are willing to experiment and being part of this huge change. I really can’t say if there have been problems. I think, for example, of Estonian language (roughly one million people speak our language) so we don’t have that much resources. Big companies like Microsoft, Alphabet, Google, don’t invest in Estonian language. Our government knows that and answers back with investments.
We need to invest more than an English-speaking country or a Spanish-speaking country as well. As a small nation we need to invest much more than the others. I mean especially from the Government side.
Is digital identity the foundation of an AI powered judicial system? Or otherwise, can countries that have not adopted digital identity mechanisms still implement successfully AI judges?
My short answer is yes. Countries without previous digitalization can make big jumps. I will tell you why I think so.
We can differentiate between projects that need previous data or those that need to have any information systems in place and collect data electronically. But at the same time when we talk about, for example, satellite images analysis then this is kind of far more straightforward. Our technologies can be actually exported to Spain, Portugal, Italy, Uruguay… You don’t need to actually have any information systems in place.
We use the satellite images which map out your information so it would be possible to get those processes signs. In other cases, unfortunately, you need to digitalize the process before.
Otherwise, sometime it’s just too complicate (even for us) because for many projects data quality is not enough. In other cases, people don’t actually understand what the different elements mean and how they are collected; making results useless.
From your perspective, what is the impact of GDPR in AI and digital innovation?
From the government side, GDPR sometimes could present some difficulties. But at the same time if citizens have given their data to the government for any reason then it shouldn’t be lawful to use their data for purposes other than the ones the citizen has consented to. We actually advocate the transparency part a lot. GDPR has allowed us to give this asset even more.
I think this transparency should move to the private sector as well. As citizens, we should be able to see when our data is used by the private sector. People will give their data if they have a high trusting level. If you’re able to be transparent and explain how you use the data you have collected, people will be more predisposed to share their data. We are right now working on a tool called “authorization service” where a citizen can authorize another stakeholder to access their data. This platform will help both the Government and citizens to have better control and management of data, more transparent and organized.
This tool is used, amongst others, for scientific purposes; you can receive better services offers. The key thing here is transparency. However, if you don’t trust the system you can take away your authorization. For example, if you don’t want your doctor to access your health data you can opt out. But I can say that only a few people have actually done it.
What are your thoughts on access to justice and AI?
The objective is that justice feels important for people. I really believe that if we’re able to be successful in the justice side then AI can actually become something really big. People will definitely trust AI.
Suscribe to our newsletter here.