technicalmarketing.ai Free AI audit

Question

What is the best AI for lawyers?

No single product is best, and we have not tested enough to rank them. Choose by task and by terms: a legal research platform for authorities, a general assistant for drafting without client facts, and a vendor whose confidentiality terms you have read for anything that sees a file. Then check every output yourself.

Why accuracy decides the choice

Lawyers rank accuracy first. In the ABA's 2024 Legal Technology Survey, 74.7% of respondents named it the most pressing concern about AI tools, ahead of reliability at 56.3% and data privacy and security at 47.2%1. The testing supports the worry. Stanford researchers asked specific, verifiable questions about random federal court cases and found legal hallucinations 58% of the time with ChatGPT 42. Two commercial legal research products, which their sellers had described as free of hallucinations, each hallucinated between 17% and 33% of the time, less often than general-purpose GPT-43.

A later Stanford evaluation of 202 queries, which its authors call small, found the best research tool answered 65% of queries accurately and another answered 42% accurately4. Even the leader did not answer more than a third of queries accurately (100% - 65%). Those studies tested specific products at specific dates, and models change, so a published score is a reason to test, not a purchase decision.

Match the tool to the task

A task that needs a real authority belongs in a research platform where you can open the cited source. A task that needs only language, such as a first draft of a letter built on anonymized facts, can go to a general assistant. A task that puts a client file in front of the tool needs a vendor whose terms you have read and whose handling of your inputs you can describe to the client.

When a court sees fabricated authority the consequences are real. In Mata v. Avianca the court imposed a $5,000 penalty on two lawyers and their firm for non-existent cases generated by ChatGPT5. The same order said there is nothing inherently improper about using a reliable AI tool for assistance, but that attorneys keep a gatekeeping role over accuracy5.

What to ask a vendor

Ask where inputs are stored, who at the vendor can read them, whether they train models, how long they are kept and what an administrator can turn off on the plan you would actually buy. Ask whether the tool shows its sources so you can open each one. Ask for the answers in writing and keep them with your approval record. The ABA's Opinion 512 says lawyers who rely on outside providers to run AI tools carry added supervisory duties6.

Run your own bake-off

Take ten questions from closed matters, strip the client details and put the same questions to each candidate. Check every citation against the source. Count the errors and the time to correct them. A tool that saves ten minutes of drafting and costs twenty minutes of checking has not saved you anything.

We are not lawyers. Read the rule yourself or ask your counsel how it applies to you.

People also ask

Do I need to tell clients I use generative AI?

Sometimes. You must disclose if a client asks or requires it, you need informed consent before client information goes into a self-learning tool, and a general engagement-letter clause is not enough. Behind-the-scenes use with no client information may need none. Your state's opinion controls.

Full answer

What kinds of AI tools do lawyers use?

Four kinds: general assistants for drafting, legal research platforms with citation checks, document tools for review and summaries, and front-office tools such as AI receptionists for intake. Each has different data terms, so evaluate them one at a time.

Full answer

How should I train my lawyers and staff on AI?

Run a short session on your written policy: approved tools, what never goes in, how to verify every citation and who approves client-facing use. Include nonlawyer staff and repeat it whenever you add a tool.

Full answer

Can AI handle intake for a law firm?

It can answer calls and chats, say it is an AI assistant, collect a name, a number and the matter type, and hand off to your team. It should not advise callers or accept matters.

Full answer

Is Claude or ChatGPT better for lawyers?

Neither is certified for legal work. Compare their terms on training, access, retention and administrator controls for the plan you would buy, then test both on a non-confidential exercise and judge accuracy yourself.

Full answer

Sources

  1. Mark Calaguas. 2024 Artificial Intelligence TechReport. American Bar Association, Law Practice Division (ABA TechReport), 2024. americanbar.org/groups/law_practice/resources/tech-report/2024/2024-ar
  2. Matthew Dahl, Varun Magesh, Mirac Suzgun, Daniel E. Ho. Large Legal Fictions: Profiling Legal Hallucinations in Large Language Models. arXiv (v2, last revised 21 Jun 2024), 2024. arxiv.org/abs/2401.01301
  3. Varun Magesh, Faiz Surani, Matthew Dahl, Mirac Suzgun, Christopher D. Manning, Daniel E. Ho. Hallucination-Free? Assessing the Reliability of Leading AI Legal Research Tools. arXiv (submitted 30 May 2024), 2024. arxiv.org/abs/2405.20362
  4. Varun Magesh, Faiz Surani, Matthew Dahl, Mirac Suzgun, Christopher D. Manning, Daniel E. Ho. Hallucination-Free? Assessing the Reliability of Leading AI Legal Research Tools. arXiv (published in the Journal of Empirical Legal Studies, 2025), 2024. arxiv.org/pdf/2405.20362
  5. Judge P. Kevin Castel, U.S. District Court, S.D.N.Y.. Mata v. Avianca, Inc., No. 22-cv-1461 (PKC), Opinion and Order on Sanctions (Doc. 54). U.S. District Court for the Southern District of New York (RECAP/CourtListener copy of the filed order), 2023-06-22. storage.courtlistener.com/recap/gov.uscourts.nysd.575368/gov.uscourts.
  6. ABA Standing Committee on Ethics and Professional Responsibility. ABA Formal Opinion 512: Generative Artificial Intelligence Tools. American Bar Association, 2024-07-29. americanbar.org/content/dam/aba/administrative/professional_responsibi

Find out what AI says about you.

A free 20-minute call. We ask the assistants your customers' questions and show you where you come up today.

No pitch unless you ask for one.