AI Audit
Last Updated: July 29, 2026 | By Mihail Sebastian | AI Dictionary
An independent, formal examination of an AI system against defined criteria such as laws, standards, or internal policies, with documented findings.
What is an AI Audit?
An AI audit is a formal, independent examination of an AI system against defined criteria: laws, technical standards, or an organization’s own policies. The auditor reviews the system’s design, data, decision process, and outcomes, then documents whether it meets the criteria.
Independence is the point. The people who built or operate the system do not grade it; an internal audit function or an external firm does.
How an AI Audit Works
An audit starts with scope and criteria: which system, which decisions, and what standard it will be judged against, whether that is the EU AI Act, an anti-discrimination statute, or the company’s responsible AI policy.
The auditor then collects evidence (documentation, training data provenance, model test results, logs of live decisions) and tests the system’s outputs directly, for example by comparing selection or error rates across demographic groups.
The output is a report: findings, the evidence behind them, and a conclusion on whether the system conforms. Regulators, customers, and courts treat that report as evidence precisely because the auditor had no stake in the answer.
AI Audit vs AI Assessment
The practical difference: an audit is an independent, formal examination against defined criteria, while an AI assessment evaluates a system’s capabilities and risks, usually internally and continuously. An audit requires a standard to audit against and an examiner with no stake in the result; an assessment requires neither.
The two feed each other. Assessments surface and fix problems; audits verify the result against the standard and put it on the record.
| AI Audit | AI Assessment | |
|---|---|---|
| Question it answers | Does the system meet the required standard? | How good and how risky is this system? |
| Who performs it | An independent examiner with no stake in the result | The organization itself, usually the team |
| When | At fixed points, against a defined standard | Ongoing, before and after deployment |
| Output | A formal report that serves as evidence | Findings that feed fixes and decisions |
Example of an AI Audit
New York City’s Local Law 144, in force since July 2023, made AI audits a legal requirement for one specific use case. An employer that uses an automated tool to screen candidates for jobs in the city must have the tool undergo a bias audit by an independent auditor before use, repeat the audit annually, and publish a summary of the results.
The audit itself follows a defined recipe: compute selection rates for candidates by sex and by race or ethnicity, and report the ratios between groups. The employer must also tell candidates the tool is in use.
Every element of the audit pattern is present: defined criteria, an independent examiner, and documented, public findings.
FAQ
Is an AI audit required by law?
In specific cases, yes. New York City’s Local Law 144 requires an annual bias audit for automated hiring tools used in the city, and the EU AI Act requires high-risk systems to pass a conformity assessment, though providers can in many cases run that assessment themselves. No law requires every AI system to be audited.
How often should an AI system be audited?
As often as the applicable standard demands: Local Law 144 sets an annual cycle, and contractual or internal policies set their own. Absent a fixed deadline, the sensible trigger is change – a retrained model, a new use case, or drift in live outcomes reopens the question the last audit answered.
Related AI terms: AI Assessment · Impact Assessment · Risk Management · Compliance · AI Governance
Did you like the AI Audit gist?
Learn about 250+ need-to-know artificial intelligence terms in the AI Dictionary.
Mihail Sebastian — Writes about AI governance, regulation, and the technology behind them. Placeholder bio — replace with a real credential line. About