The GenAI Law Firm Survey
A major UK law firm’s survey of generative AI tools has shown that gen AI cut their lawyers’ work drafting briefs in half.
Ashursts used in-depth trials of several AI tools and saw substantial time-saving benefits in legal work.
The benefits were particularly significant in creating first drafts quickly and efficiently.
For over a year, law firms have been grappling with the challenges posed by generative AI but the lack of standard evaluation tools meant that they have been relying on feedback from their attorneys and working with outside consultants and vendors to address the AI situation in terms of the effect on productivity.
A number of major law firms are using legaltech AI tools to refine and develop their processes and build both productivity and profitability.
Ashurst took a formal quantitative approach to its trials and went a step further by making the results public.
From November to March, Ashurst conducted an experiment to assess how much AI assisted its lawyers, focusing on factors like time saved, accuracy, and quality of output.
“Ultimately, this all feeds into our investment processes,” Ashursts partner and chief digital officer Tara Waters was quoted by Bloomberg Law saying.
“We want to ensure that if we are pushing hard to invest quickly, and potentially invest significant amounts, we have all the data possible to support our recommendations.
Key findings from the report include:
- Time Savings: GenAI has saved lawyers around 80 percent of the time on drafting UK corporate filings, 59% on industry research reports, and 45 percent on first draft legal briefings.
- Global Trials: From November 2023 to March 2024, Ashurst conducted three global GenAI trials involving 411 partners, lawyers, and staff across 23 offices in 14 countries.
- Quality and Accuracy: GenAI-generated content was found to be challenging to distinguish from human output when legally correct. The accuracy scores for GenAI ranged from 1 to 4 out of 5, while human content scored between 3 and 4. Experts correctly identified all human-generated content but misidentified 50 percent of GenAI output.
- Productivity Boost: Lawyers reported saving about 10 minutes per 30-minute call on average, finding GenAI useful for summarising meetings and acting as a second pair of eyes. 88 percent of staff felt more prepared for the future with GenAI.
- Human-AI Collaboration: The report stresses that GenAI is not about replacing humans but supplementing human interaction. Effective use of GenAI requires a sustained strategy, training, continuous dialogue, and digital literacy development.
The report, written by Waters and director Sarah Chambers, and senior manager Sophia Slade, emphasizes the importance of bringing people along in the AI journey to dispel misconceptions about AI replacing humans.
The report highlights the value humans bring to certain types of work that cannot be replaced by AI. Waters underscores the need for a balanced approach where GenAI works with, not against, human professionals.
