AI Startup Delivers Powerful Custom Legal Research Tool by Partnering with OpenAI

🔷 Subscribe to get breakdowns of the most important developments in AI in your inbox every morning.

Article: Harvey partners with OpenAI to build a custom-trained model for legal professionals.

Who:

• Harvey, an AI platform for legal, tax, and finance professionals, was founded by Winston Weinberg (attorney) and Gabe Pereyra (AI researcher)

• The startup recently partnered with OpenAI to create a custom-trained case law model

Why:

• Legal work is becoming increasingly complex, with vast amounts of contracts and documents to review

• Existing foundation models and retrieval-augmented generation systems had limitations for the unique, open-ended nature of case law research

• Harvey aimed to create an AI system that could thoroughly answer legal questions and cite relevant sources

How:

• Harvey and OpenAI collaborated to inject new knowledge and reasoning capabilities into base models

• They added the equivalent of 10 billion tokens worth of data, starting with Delaware case law and expanding to all U.S. case law

• The custom-trained model was designed to provide in-depth context and nuance for legal questions

What did they find:

• In tests with 10 large law firms, attorneys preferred the output from Harvey's custom case law model 97% of the time compared to GPT-4

• The custom model provided longer, more complete answers that addressed the nuances of the questions and covered more relevant case law

• The model significantly reduced hallucination, ensuring that every sentence was supported by a cited case

What are the limitations and what's next:

• Harvey plans to explore other applications of the case law model, such as drafting briefs and motions or comparing case law across jurisdictions

• The startup is also focusing on developing agents that combine multiple model calls into a single output to simplify the user experience

Why it matters:

• The custom-trained case law model demonstrates the potential for AI to help legal professionals manage the growing volume and complexity of their work

• By handling routine tasks, AI can allow lawyers to focus more on client interactions and decision-making

• Harvey's partnership with OpenAI showcases the value of collaborating to push the boundaries of what's possible with large language models in specific domains

Questions and Comments:

• Why wouldn’t OpenAI’s next model simply absorb the capabilities of this one and offer it as part of the standard ChatGPT pricing?

• The most expensive thing in AI is getting highly qualified humans to tell you what the right thing to do is.

• Harvey gets $250/hour attorneys to do this for free, versus paying contractors in Kenya to do it at $1/hour

• Terrific labor arbitrage, especially given the goal will be to eventually subsume the tasks that these attorneys perform

• This was also quickly followed by an expansion of fine-tuning tools on the OpenAI website

• A little disappointing, as it moves the needle toward specialization instead of toward the grand goal of general intelligence.

• From the co-founder of huggingface:

• But also, it shows OpenAI maturing as an enterprise software business… giving the clients what they want, when they want it

• Even if GPT-5 promises to subsume all this additional effort within 12 months

Become a subscriber for daily breakdowns of what’s happening in the AI world:

I