If AI-assisted work cannot show accountability, delivery has not removed legal risk

On Nov. 4, Law360 Legal Industry reported that a public database tracking cases involving artificial-intelligence hallucinations had surpassed 2,200 entries. The article said Massachusetts District Court Associate Justice Brian Palmucci identified a deeper risk: lawyers gradually surrendering judgment to AI and losing analytical skills. That follows a separate Law360 Legal Industry report that nearly four years after ChatGPT launched and legal work began experimenting at scale with generative artificial intelligence, hallucination errors continue to appear in court filings. The immediate news is not that AI has made legal work impossible. It is that erroneous work continues to produce a visible accountability problem while adoption expands.

The consequence matters because speed and scale do not perform the functions that make legal work defensible. Legaltech News (Law.com) says AI provides speed, scale, consistency and computational power, while attorneys provide legal knowledge, context, skepticism, judgment and accountability. Legal Futures reports that a third of law firms already use AI tools without a formal policy governing how the tools are used, what data enters them or who is accountable when something goes wrong. AI does not eliminate the consequences of bad legal work; it can increase the distance between producing an output and knowing who stands behind it. For an AI-native legal service, verification and responsibility are not an afterthought to delivery. They are part of delivery.

What happened proves that AI-assisted legal work remains answerable for its failures. It does not prove that every AI tool is unsafe, that every hallucinated citation produces a sanction, or that a particular firm has solved accountability. Reuters Legal reported that a federal judge warned that increasing AI use could create client risks and hinder junior-lawyer training, even as the judge declined to sanction attorneys over AI-generated errors in a court filing. That is one judicial warning and one decision not to sanction, not a general rule. The stronger reading would require firms to show that their systems catch errors, preserve accountable human review and protect confidential information in actual matters. Legaltech News (Law.com) reports that AI agents disrupt the clearer workflow of traditional software because privilege and confidentiality obligations run through the attorney’s conduct. A service that cannot explain where client information goes or who approves the result has not removed legal risk; it has moved the risk into its design.

The next confirming fact is operational, not promotional: a law firm or legal service publishing a formal policy that identifies permitted tools, the data that may enter them, the human accountable for an output and the checks required before delivery. Legal Futures has separately reported that most law firms and in-house legal departments have yet to get on top of AI governance despite widespread use. A court order addressing an AI-generated error would also show whether accountability attaches to the lawyer, the firm or both. We will count AI-native delivery as progress only when a provider can show that its workflow makes verification, responsibility and confidentiality observable in the finished work.