• Emergent Behavior
  • Posts
  • AI Startup Delivers Powerful Custom Legal Research Tool by Partnering with OpenAI

AI Startup Delivers Powerful Custom Legal Research Tool by Partnering with OpenAI

Harvey moves forward

🔷 Subscribe to get breakdowns of the most important developments in AI in your inbox every morning.

Who:

  • Harvey, an AI platform for legal, tax, and finance professionals, was founded by Winston Weinberg (attorney) and Gabe Pereyra (AI researcher)

  • The startup recently partnered with OpenAI to create a custom-trained case law model

Why:

  • Legal work is becoming increasingly complex, with vast amounts of contracts and documents to review

  • Existing foundation models and retrieval-augmented generation systems had limitations for the unique, open-ended nature of case law research

  • Harvey aimed to create an AI system that could thoroughly answer legal questions and cite relevant sources

How:

  • Harvey and OpenAI collaborated to inject new knowledge and reasoning capabilities into base models

  • They added the equivalent of 10 billion tokens worth of data, starting with Delaware case law and expanding to all U.S. case law

  • The custom-trained model was designed to provide in-depth context and nuance for legal questions

What did they find:

  • In tests with 10 large law firms, attorneys preferred the output from Harvey's custom case law model 97% of the time compared to GPT-4

  • The custom model provided longer, more complete answers that addressed the nuances of the questions and covered more relevant case law

  • The model significantly reduced hallucination, ensuring that every sentence was supported by a cited case

What are the limitations and what's next:

  • Harvey plans to explore other applications of the case law model, such as drafting briefs and motions or comparing case law across jurisdictions

  • The startup is also focusing on developing agents that combine multiple model calls into a single output to simplify the user experience

Why it matters:

  • The custom-trained case law model demonstrates the potential for AI to help legal professionals manage the growing volume and complexity of their work

  • By handling routine tasks, AI can allow lawyers to focus more on client interactions and decision-making

  • Harvey's partnership with OpenAI showcases the value of collaborating to push the boundaries of what's possible with large language models in specific domains

Questions and Comments:

  • Why wouldn’t OpenAI’s next model simply absorb the capabilities of this one and offer it as part of the standard ChatGPT pricing?

  • The most expensive thing in AI is getting highly qualified humans to tell you what the right thing to do is.

  • Harvey gets $250/hour attorneys to do this for free, versus paying contractors in Kenya to do it at $1/hour

  • Terrific labor arbitrage, especially given the goal will be to eventually subsume the tasks that these attorneys perform

  • This was also quickly followed by an expansion of fine-tuning tools on the OpenAI website

  • A little disappointing, as it moves the needle toward specialization instead of toward the grand goal of general intelligence.

  • From the co-founder of huggingface:

  • But also, it shows OpenAI maturing as an enterprise software business… giving the clients what they want, when they want it

  • Even if GPT-5 promises to subsume all this additional effort within 12 months

Become a subscriber for daily breakdowns of what’s happening in the AI world:

I

Join the conversation

or to participate.