Lecture Sidelights: Part 2 | Even Computers Can Select Potatoes, But What About Sentencing and Judgments?

This lecture was delivered by Cui Jia-wei, focusing on the application of artificial intelligence in the judicial field. Discussions prior to the opening noted that five years ago, it was generally believed that lawyers' work would be difficult for AI to replace. However, the rapid development and rise of AI in recent years have sparked reflections on the application of artificial intelligence in the legal field and how we should define regulatory norms.

1. Starting from a Generated Image: Good Input Yields Good Output

The speaker began with an AI-generated image of himself to illustrate that the operation of artificial intelligence is a process where "input determines output." The clearer and more complete the input data, the closer the generated result will be to being correct; conversely, if the input itself is ambiguous or biased, the output will inevitably have problems. This point is even more crucial for judicial AI, because any erroneous judgment can directly affect the rights and interests of the parties involved.

The speaker explained that artificial intelligence does not truly "understand" cases. It merely learns the connective relationships between words in data through mathematical vectors (algorithms) and then simulates them.

Several key limitations include:

• Knowledge Boundaries: AI's knowledge comes entirely from data fed in by humans, and background information is not automatically incorporated.
  
• Inherited Biases: Because data comes from humans, AI also inherits human biases, discrimination, and cognitive errors.
  
• Comparison of Strengths and Weaknesses: In processing massive, repetitive, and homogeneous data, AI far surpasses humans; but in areas requiring complex situational judgment and life experience, AI falls far short of humans.
  

2. The Benefits and Issues of Artificial Intelligence Intervening in Judicial Judgments

Entering the main topic, the speaker discussed the daily routine of legal professionals: heavy caseloads, routine overtime, and the fact that every judgment cannot be taken lightly.

Efficiency affects performance evaluations and promotions, and judgments also concern every related family. Under such pressure, AI-assisted judgment systems seem to have become a solution. The Judicial Yuan in Taiwan once trained an AI judgment system, expecting it to share judges' workloads. The speaker gave an example: if AI could make preliminary judgments on drug cases and fraud cases (these two categories make up the bulk of criminal cases in Taiwan), judges might have their burdens eased and even have more time to spend with their families on weekends.

The speaker also immediately threw out questions to the audience:

• When AI intervenes in judgments, will it sacrifice the rights and interests of the underlying parties?
  
• Do we only pursue efficiency while ignoring the essence of justice
  

He specially asked the audience: "Being able to spend weekends with family sounds wonderful, but is there really no problem with such a scenario?" The venue also fell into deep thought for a moment.

3. The US COMPAS Case: The Boundaries of AI Judgment

Next, the speaker introduced the COMPAS system in the United States, an algorithmic system used to predict criminal recidivism risk. A famous case is that of Eric Loomis:

• Loomis was brought up for trial due to a case, and the COMPAS system determined that he had a "high likelihood of recidivism."
  
• Loomis believed that such a judgment infringed upon his rights and interests, so he appealed, arguing that the algorithm was opaque and even violated constitutional due process rights.
  
• Ultimately, the court ruled that the use of COMPAS is not unconstitutional, but its judgment cannot be used solely as the basis for conviction and must be supported by other evidence.

This highlights that AI can only be an "assistant" in the judiciary and cannot serve as the sole standard.

The speaker presented US budget data, highlighting the huge gap in AI investment between the public and private sectors:

• US Department of Defense 2023 AI budget: $350 million
  
• US Department of Defense 2024 AI budget: $4.6 billion
  
• US enterprises' investment amount in AI for 2024: over $150 billio
  

Under such a gap, how the government develops or procures AI systems becomes an important issue:

• How should the government negotiate cooperation with enterprises so that enterprises are willing to invest resources while maintaining transparency and supervision?
  
• Enterprises often refuse to disclose information on the grounds of "business secrets" and "intellectual property rights." This was also highlighted in the Loomis case.
  
• If the government relies too heavily on enterprises, enterprises may use this to expand their influence or even monopolize judicial resources.
  

The speaker also added that the sources of recidivism risk systems in various US states differ; some cooperate with private enterprises, while others rely on foundations or non-profit organizations, presenting diverse models.

4. Algorithm Transparency and Open-Source Issues

Hearing this, questions were also raised from the audience: What are the judgment criteria of AI? What is the basis of the algorithm? Can we know its code?

The speaker responded that this is precisely one of the cores of global judicial AI controversies

• The algorithms of many AI systems belong to enterprises' "trade secrets" and will not be made public.
  
• Lack of transparency means judges and parties cannot truly understand AI's basis.
  
• This will lead to "black-box judgments," undermining the fairness of the judiciary
  

The speaker added that Europe already has relevant regulatory discussions requiring enterprises to provide open-source code or at least allow the government to inspect algorithms under specific circumstances to ensure transparency, which is also a trend worth Taiwan's attention.

The speaker reminded that when introducing new technologies, institutional construction often lags behind:

AI's autonomy and agency capabilities are far higher than other tools, making regulations and guidelines even more necessary, yet the government often only sees "highlights and political achievements" and ignores potential risks.

For example, when the Judicial Yuan promoted AI-written judgments in 2023, it implemented the system first and supplemented regulations later, showing that internal efficiency was prioritized while the impact on the rights and interests of external citizens might have been ignored. How should a balance be established?

When the speaker introduced the AI judgment assistance system currently being promoted by the Judicial Yuan, a student raised a question:

"Does this AI system belong to a public version (developed and maintained by the government) or a personalized version? If it is personalized, might it be more closely aligned with individual needs?"

Citing Taiwan's current institutional design, the speaker responded that judicial assistance systems of this kind will likely follow a public version, uniformly developed and maintained by government units to ensure the transparency and consistency of judicial judgments; if it were a private system, there might be concerns about data opacity or bias toward commercial interests. In this regard, the speaker also supplemented Singapore's situation: they introduced ChatGPT in the public sector, but with strict restrictions on use, and the general public cannot freely use ChatGPT. The version adopted in the judicial field is a "personalized" application, which stands in stark contrast to Taiwan's public version model.

5. The "Fayuan" and "Chi-Fa" Cases: Legal Data Organization and Copyright Dispute

The speaker also cited a recent case in Taiwan:

• Fayuan Co.: Long operated a legal database, organizing and updating original statutory articles and rewriting them into more easily understandable versions, providing services on a subscription basis.
  
• Chi-Fa Co.: Accused of using crawler programs to directly scrape content organized by Fayuan and resell it for profit.
  

The dispute in the case lies in: Statutory articles themselves are public goods and lack copyright, but after manual organization and compilation, can they become protected works?

On the surface, this appears to be just a general copyright dispute with little direct connection to AI. However, it brings out a very important issue behind judicial AI: the legitimacy and transparency of data sources.

If AI training data comes from illegal scraping or involves copyright infringement of others' organizational achievements, the legitimacy of the entire system will be questioned.

6. Institutional Design and Social Trust

Regarding institutional design, the speaker pointed out that judicial AI needs to answer three questions:

1. Before utilization: How to collect and clean data?
   
2. During utilization: Who can inspect the algorithms? How to prevent the amplification of bias?
   
3. After utilization: When errors occur, how is responsibility attributed
   

The speaker emphasized that the core of the judiciary lies in "social trust." If the introduction of AI lacks transparency and public discussion, the public will instead trust the judiciary even less. Much like Taiwan's previous Digital Intermediary Services Act, which sparked fierce controversy due to a lack of adequate communication, judicial AI cannot repeat the same mistakes.

7. Conclusion

Finally, the speaker reminded that artificial intelligence in judicial applications is both an opportunity and a risk:

• It can enhance efficiency and help share heavy caseloads.
  
• However, it may also infringe upon human rights due to opacity.
  
• More importantly, judicial legitimacy ultimately still relies on "people"—judges, lawyers, parties, and the trust of society as a whole.
  

My Observations

This lecture gave me a profound realization that judicial AI is a complex issue intertwining law, institutions, and ethics. Artificial intelligence brings us much convenience, but if the basis behind algorithms is opaque, it may cause some people to unjustly suffer from erroneous judgments. In today's information age, personal information security is extremely important. Singapore's artificial intelligence system also considered information security issues from the very beginning, deploying the system within the government's internal GCC (Government Commercial Cloud) so that enterprises cannot acquire this data, thereby safeguarding data privacy. The speaker also emphasized that the development of judicial AI cannot focus solely on technological progress; it also needs to be coordinated with transparent systems, legal norms, and social discussion and consensus, much like the invention of the automobile in the 19th century.