Who is responsible when AI makes a consequential mistake?

Start with responsibility

If a system helps screen applicants, answer customers, or prioritize cases, someone must own its purpose and performance. “The AI decided” does not tell an affected person whom to contact, what evidence was used, or how to challenge an error. Define those routes before the system becomes essential to the process.

NIST's AI Risk Management Framework is a voluntary resource for managing risks across the design, use, and evaluation of AI systems. It is a useful starting point for organizational questions, not a universal legal certificate. National Institute of Standards and Technology: Artificial Intelligence Risk Management Framework

A practical set of questions

What problem is the system meant to solve? Which people might be harmed by a mistake? What data does it receive? How is performance checked for the people who actually use it? Can an operator stop the process? Who receives incident reports? What happens when the model or provider changes?

For a small business, a one-page policy may be a useful beginning: approved uses, prohibited data, review requirements, and an escalation contact. A more consequential system needs stronger evidence, access controls, monitoring, and a clear record of decisions.

Meaningful human oversight

Putting a person somewhere in a process is not enough. The reviewer needs time, relevant information, authority to disagree, and a workable way to correct the result. If hundreds of decisions arrive too quickly to inspect, a nominal approval button may provide little protection.

Review this proposed workflow for accountability. Identify the responsible person at each stage, the information they need to review it, and the actions that require explicit approval. Explain how a customer can request correction and how the process can be stopped.

Consider a customer-service draft that quotes a policy. The reviewer should be able to see the source policy, not merely a polished answer. That makes review more concrete and reduces the chance of approving an invented rule.

Public policy and personal participation

Laws and regulatory duties vary by place, sector, and use. For a real compliance question, consult current official guidance and qualified advice. This guide offers a framework for understanding the issues rather than a catalog of legal obligations that may become outdated.

Citizens can ask institutions whether AI is involved in consequential decisions, how performance is measured, and what correction process exists. The goal is not to demand perfect technology. It is to make limitations visible and ensure that the people bearing the consequences are not left without recourse.