TL;DR:
- Lawyers must understand the limitations of AI, particularly large language models (LLMs) like ChatGPT, Bing Chat, and Google Bard, to avoid misusing them.
- LLMs have the potential to revolutionize the legal profession, including in international criminal law (ICL).
- ICL poses unique challenges for LLMs, such as diverse jurisprudence, data security risks, and complex documents.
- LLMs offer benefits in ICL, including summarizing jurisprudence, legal research, drafting assistance, disclosure and evidentiary collection research, and legal analytics.
- Challenges include character limits, hallucinations, revision requirements, confidentiality concerns, and limited data for legal analytics in ICL.
Main AI News:
The rise of Artificial Intelligence (AI) and its applications in various fields has been a topic of both excitement and concern. In the legal profession, AI, particularly large language models (LLMs) such as Open AI’s ChatGPT, Bing Chat, and Google Bard, has the potential to revolutionize the way lawyers work. However, the use of these tools without a deep understanding of their limitations can lead to serious consequences, as demonstrated by a recent incident where an attorney submitted a brief with fictitious cases generated by ChatGPT, resulting in the displeasure of a New York federal judge.
Despite the risks, it would be imprudent to dismiss the potential of AI in the legal profession outright. In fact, there has been a growing number of legal technology start-ups and scholarly contributions focused on exploring the transformative possibilities of these tools. One area where the potential of LLMs remains largely untapped is international criminal law (ICL). The effective integration of LLMs into ICL practice depends on their ability to navigate the unique features of this field.
ICL is characterized by complex documents that play a crucial role in shaping litigation. The investigation of international crimes often leads to a massive influx of documentary materials, which can be overwhelming to handle manually. LLMs, with their ability to comprehend large volumes of data, hold promise in this regard. However, challenges arise from poor-quality scans and documents in languages that are not commonly used in the models’ training, which can hinder their effectiveness.
Another distinctive aspect of ICL is the diversity of its jurisprudence. Each ICL institution represents a jurisdiction with its own unique repositories for storing legal precedents. While central compilations like the ICC Legal Tools Database exist, publicly available AI tools have not been specifically trained on these compilations. In contrast, American lawyers can more easily integrate AI into their domestic practice through platforms like Westlaw Edge, Lexis +, and Casetext.
Data security is also a significant concern in the context of ICL. The security implications of LLMs are not yet fully defined, posing potential threats. Any data breaches in the prosecution of war crimes or crimes against humanity could have severe consequences, including the identification and targeting of victims, witnesses, and others at risk.
Despite these challenges, LLMs already offer significant benefits to ICL lawyers. Their potential to transform legal practice should not be underestimated. However, it requires a paradigm shift from traditional search engine queries and Boolean search terms. Lawyers need to engage in meaningful dialogue with these models and learn to craft prompts that enhance the quality of responses. A recent article by Daniel Schwartz and Jonathan H. Choi provides valuable insights on how to effectively prompt LLMs in legal contexts.
As an ICL-trained lawyer, I must acknowledge that my understanding of LLMs and their potential is still evolving. I am more of an enthusiast than a skilled user of these tools. Therefore, more technically savvy individuals may refine and expand upon my description. For those interested in the technical aspects of LLM development, I recommend watching a video presentation by Andrej Karpathy of OpenAI from May 2023.
Applications of LLMs in ICL
- Jurisprudence Summaries and Filing Analysis: One of the immediate applications of LLMs in ICL is their ability to summarize jurisprudence or analyze specific parts of legal filings. The quality of the summaries depends on the way the prompting questions are formulated. By copying and pasting relevant passages from judgments or filings and asking LLMs to provide concise summaries, lawyers can obtain higher quality insights. Furthermore, utilizing direct quotes from the provided text allows for more meaningful follow-up questions and responses.
An obstacle to further growth in this area is the current character limits imposed by LLM subscriptions. ChatGPT4, for example, has a limit of 4000 characters, while Bing Chat has a limit of 2000 characters. ICL judgments can be extensive, exceeding hundreds of pages and containing over 700,000 characters. To address this limitation, GPT add-ons are being developed to circumvent these constraints. Additionally, tools like ChatPdf enable users to interact with whole PDF documents through ChatGPT.
- Legal Research: LLMs have the potential to answer complex legal research questions that go beyond simple summaries of known filings. While they excel in providing accurate information on widely discussed topics, more obscure or specific inquiries may require additional dialogue to obtain meaningful answers. Asking LLMs to delve deeper into a particular topic of interest can yield more comprehensive responses.
A significant obstacle in this area is the prevalence of hallucinations, where LLMs generate inaccurate citations and even fabricate cases with unwarranted confidence. To avoid disastrous consequences, it is crucial for lawyers to exercise due diligence and treat LLMs as complementary tools rather than substitutes for rigorous legal practice. Furthermore, the training data available to ChatGPT is limited to the internet’s data up until the end of 2021, which restricts access to the most recent case law. Bing Chat, on the other hand, does not face the same limitation.
- Drafting and Editing: When provided with relevant facts and applicable law, LLMs can assist in generating draft paragraphs for ICL documents. They can also aid in refining already drafted text, enhancing clarity and readability. Requesting LLMs to write in a specific style, such as mimicking the writing of renowned scholars or judges, may yield better results compared to general instructions.
The obstacle for further growth in this area lies in the extent of revisions required. Current LLMs cannot generate passages with references suitable for direct inclusion in a draft without further alteration. Changes in word choice may introduce unintended errors when precision is paramount, and LLMs may not always provide clear explanations for their sentence construction, indicating potential inaccuracies or hallucinations.
While respecting confidentiality and institutional procedures, it may be more beneficial for ICL lawyers to utilize the present drafting capabilities of LLMs as a source of inspiration or as a tool for refining already completed documents. Regardless of their use, the passages generated by LLMs should be subjected to careful scrutiny before being incorporated into broader work.
- Disclosure and Evidentiary Collection Research: LLMs trained on large evidentiary collections can provide valuable insights into the content of such collections. They can be prompted with information from pending disclosure requests or specific facts relevant to ICL trials. LLMs have the potential to summarize witness testimony, isolate key details, and facilitate the organization of evidence during trial proceedings.
Confidentiality is a significant obstacle in this area. Current technology carries inherent risks of errors, making it impractical to use publicly available LLMs for sensitive information. Developing secure and trustworthy solutions for handling confidential ICL data collections is crucial for widespread adoption. Several tech startups, like Casetext’s Co-Counsel tool, are working on such solutions. Custom-made tools may be necessary for ICL institutions to ensure the confidence needed for effective utilization. The ICC OTP’s evidence submission initiative, which incorporates AI and machine learning, is a positive step in this direction.
- Legal Analytics: LLMs possess the ability to answer highly specific questions and analyze massive amounts of data. This capacity opens up intriguing possibilities for generating “scouting reports” on how ICL litigators could respond to various litigation strategies. For example, LLMs could provide insights into a particular judge’s rulings on disclosure violations, past cross-examination techniques employed by Defense attorneys, or the sentencing ranges that Trial Chambers might accept for specific offenses. The effectiveness of legal analytics with LLMs depends on access to the right data sets and precise prompts.
The obstacle to further growth in this area is the limited number of ICL trials, which hinders the ability to generate meaningful predictions. The frequent turnover of judges, with non-renewable terms in many ICL institutions, also limits the availability of extensive datasets. Legal analytics may be more valuable for repetitive elements of trials, where larger datasets can be quickly accumulated, rather than for rare events that inherently have smaller datasets.
Conclusion:
The integration of Artificial Intelligence and large language models into international criminal law has the potential to transform the legal profession. While challenges exist, such as data security risks and limitations in handling complex ICL documents, the benefits are significant. LLMs can enhance legal research, improve drafting efficiency, facilitate evidence analysis, and provide insights through legal analytics. However, proper understanding of AI limitations, rigorous legal practice, and the development of tailored solutions are necessary for successful adoption. The market for AI-powered legal technology in ICL holds great promise for innovation and efficiency in the future.
