AI Bias Audit: What the Law Requires and How to Run One

In December 2025, the New York State Comptroller audited how New York City enforces its AI hiring law. The city agency had reviewed 32 published bias audits and found one compliance issue. The Comptroller's team found at least 17, according to DLA Piper's summary of the audit, and the agency agreed to most of the recommendations. The AI bias audit is a real requirement, and enforcement is moving from passive to active.

An AI bias audit is a structured test of whether an AI system produces systematically different outcomes for different groups of people, such as by sex, race, ethnicity, age or disability, followed by documented results, a decision on what to fix, and a repeat test when the system changes.

By the end, you will know which laws and standards actually call for a bias audit as of October 2026, how to run a proportionate one with real numbers, and what evidence to keep so a customer or auditor can see the result.

Here's what I'll cover:

• The three things people mean by an AI bias audit
• Which laws and standards require one, and what they actually say
• Which of your AI systems are in scope
• How to conduct an AI bias audit in seven steps, with a worked example
• Tools, the evidence pack, who can run the audit and how often
• The mistakes that make an audit fail

What an AI Bias Audit Is, and the Three Things People Mean by the Term

An AI bias audit tests an AI system's outcomes across groups of people and documents what the test found and what you did about it.

The term exists because models learn patterns from historical data, including proxies such as zip code, first name or employment gaps that stand in for protected attributes. The builder's intent does not matter. The system's behavior is what gets tested.

AI Bias Audit, AI Fairness Audit and Algorithmic Bias Audit: Same Idea, Different Legal Weight

The words are used loosely, and the legal weight behind them differs a lot.

"Algorithmic bias audit" and "AI fairness audit" are used for all three. When someone asks for one, ask which of the three they mean.

Bias Audit vs Risk Assessment vs Model Monitoring

An AI bias audit is one input to two neighboring jobs. It is not a replacement for either.

Why an AI Bias Audit Matters Before a Customer or Regulator Asks

In May 2026, Stanford researchers published a study of about 4 million job applications scored by AI hiring tools from one vendor. They found that 26% of Black applicants and 15% of Asian applicants applied to positions where the AI disadvantaged their racial group. The study covers one vendor, so it does not describe every tool. Its finding is the useful part: the problem showed up job by job, and the pooled average hid it.

That is what an AI bias audit is for. The exposure also reaches beyond city law:

• The toolmaker can be sued, not only the employer. In Mobley v. Workday (N.D. Cal.), the court granted preliminary certification of a collective action under the Age Discrimination in Employment Act on 16 May 2025, on a theory that the software vendor acted as an agent of the employers using it. The case is still pending in October 2026.
• The employer carries the duty even when it buys the tool. New York's enforcement agency, the Department of Consumer and Worker Protection (DCWP), states in its FAQ that the vendor is not responsible for the bias audit and the employer is.
• Buyers ask for the result. Enterprise buyers in HR tech, lending and insurance now ask vendors for the audit summary in security questionnaires.

Real practitioners put the underlying question plainly. A Hacker News commenter wrote in May 2025: "How could you prove discrimination if you can't audit the decision making system?" The audit is how a team shows its own system was examined.

Which Laws and Standards Require an AI Bias Audit (Status as of October 2026)

AI bias audit requirements are narrower than most summaries imply, and the requirements differ by region. Very few rules say "run a bias audit" in those words. Most say "examine", "evaluate and document" or "assess impact", and the difference decides what you must actually do.

NYC Local Law 144 Bias Audit: What It Must Contain

Local Law 144 has been in effect since 1 January 2023, and DCWP began enforcing it on 5 July 2023. It covers employers and employment agencies that use an automated employment decision tool in the city to screen candidates for hiring or promotion. A tool counts when it uses machine learning, statistical modeling, data analytics or AI and substantially assists or replaces discretionary decision-making.

The NYC bias audit has specific AI bias audit requirements, taken from DCWP's automated employment decision tools page and FAQ and the city Administrative Code:

• Independent auditor. Not the employer, not the vendor, and nobody involved in using, developing or distributing the tool. DCWP publishes no approved-auditor list.
• Minimum content. Selection or scoring rates and impact ratios by sex, by race and ethnicity, and by intersectional categories of the two.
• Data. Historical data from the tool must be used. Test data is allowed only when historical data is insufficient for a statistically significant audit. Imputed or inferred demographic data cannot be used.
• Small categories. Categories under 2% of the data may be excluded, with an explanation.
• Timing. The audit must be done within one year before the tool is used, so it repeats annually.
• Publication. A summary of the most recent audit and the date the tool began to be used must be posted publicly.
• Notice. Candidates and employees who live in the city get notice at least 10 business days before the tool is used.

Penalties run up to $500 for a first violation and $500 to $1,500 for each subsequent violation, and each day of use is a separate violation, under the city's Administrative Code (section 20-872).

What the law does not require is any specific fix. A low impact ratio does not trigger a duty to change the tool, but other anti-discrimination law still applies.

EU AI Act: Bias Examination, Mitigation and Special-Category Data

For providers of high-risk AI systems, Article 10(2)(f) requires examination of datasets "in view of possible biases" that are likely to affect health and safety or lead to prohibited discrimination. Article 10(2)(g) requires appropriate measures to detect, prevent and mitigate them. Article 10(3) adds that datasets must be relevant, sufficiently representative and, to the best extent possible, free of errors and complete.

After the Digital Omnibus on AI, the rule for using special-category data to find bias moved into a new Article 4a. It exceptionally allows processing that data where strictly necessary for bias detection and correction, with safeguards, and it does not create a duty to run the detection. Annex III high-risk obligations now apply from 2 December 2027. For scope, see the EU AI Act compliance guide and the EU AI Act risk classification guide.

Colorado and the US State Picture

Colorado's 2024 AI Act never took effect. SB 26-189, signed on 14 May 2026, repeals and replaces it, with most provisions effective 1 January 2027. The replacement drops the impact assessments and the risk management program. It keeps notice, explanations of adverse decisions, three-year records and liability allocated by relative fault, which contracts cannot shift.

A federal lawsuit challenging the older law, X.AI LLC v. Weiser, was paused by stipulation in April 2026 and was still pending in October 2026. Many summaries still describe the old act.

Which AI Systems Need an AI Bias Audit

The scoping rule is simple. A decision about a person that a tool makes or substantially shapes is in scope. A spell-checker is not.

Scope the AI bias audit to what you use, not only what you build. You own the use of a tool even if you did not build it. If you buy it, DCWP's FAQ puts responsibility for the audit on you as the employer.

If you sell it, expect the audit request from your customers. Keep the model record from the AI model governance guide so you can say what the system is, what it was tested on and who owns it.

How to Conduct an AI Bias Audit in Seven Steps

An AI bias audit is mostly decisions made before any code runs: what you test, which groups, which metric and what threshold. Tool-first approaches skip the part an auditor checks first.

Step 1: Scope the System and the Groups

Write down the system, the decision it makes, the population it affects, the protected attributes that matter and the likely proxies for them. State the fairness objective before you test anything, and name an owner. This page is the first thing a reviewer asks to see.

Step 2: Audit the Data Before the Model

A model inherits the problems of its data. Check representation by group, the history behind the labels (past decisions encode past bias), missing demographic fields, the sampling method and time period, and features that proxy for protected attributes. Record where the data came from. ISO 42001's data controls, titled quality of data for AI systems (A.7.4) and data provenance (A.7.5), cover the same ground.

Step 3: Choose the AI Fairness Metrics and Thresholds in Advance

AI fairness metrics measure different things: equal outcomes, equal error rates, or equal accuracy of scores. The audit picks one on purpose and writes down why, before seeing results.

You cannot satisfy every metric at once. Kleinberg, Mullainathan and Raghavan showed in 2016 that three natural fairness conditions generally cannot hold together except in highly constrained cases, so the audit records which metric was chosen and why. NIST's AI RMF Playbook adds that "systems in which harmful biases are mitigated are not necessarily fair," which is a reason to document the context and not only the number.

Step 4: Run the Tests for an AI Bias Audit, Including Counterfactual and Intersectional Cuts

AI bias testing has four parts: compare outcomes by group, run counterfactual tests (change one demographic signal such as a name and hold everything else constant), run intersectional cuts (sex and race together, as NYC requires), and handle small samples honestly. A group with 12 people can show a dramatic ratio that means little. Report the count beside the ratio.

Testing LLM and Generative Features

Do LLMs and generative AI features need a bias audit? If they write about, to or for people, test them. Use name-swap prompts, compare refusals and tone across groups, and look at representation in outputs. Public benchmarks help as a start: BBQ, from Parrish and colleagues at ACL 2022 Findings, tests social bias in question answering across nine social dimensions. Benchmarks test the base model, not your prompt and retrieval layer, so test your own task.

Step 5: Analyze Gaps and Decide What They Mean

A gap is not a verdict. Separate the size of the gap, whether it is statistically significant and whether it is practically meaningful. The four-fifths rule at 29 CFR 1607.4(D) is a screening heuristic: a selection rate below four-fifths of the highest group's rate will generally be regarded by federal agencies as evidence of adverse impact, but smaller differences can still be adverse impact, and larger ones may not be where numbers are small.

Practitioners argue about exactly this. In a June 2026 Hacker News thread on the Stanford study, one commenter described the rule as "a fast, coarse evaluation" for deciding whether something is worth investigating further. Another objected that the study used it "as a measurement of discrimination, instead of as a flag that triggers further investigation". Both are describing the same limit: the ratio tells you where to look.

Then record a decision: investigate, fix, accept with a written rationale, or stop using the system.

Step 6: Mitigate and Re-Test

Mitigation comes in four families: fix the data or reweigh it before training, constrain the model during training, adjust decision thresholds after the fact, or remove a proxy feature. Human review is a fifth option when the stakes justify it. Whatever you choose, re-run the same tests and log the residual disparity. A fix with no re-test is a claim, not evidence.

Step 7: Document, Sign Off and Schedule the Next Run

Write down what a reviewer will need (the checklist below lists it). The owner who ran the audit and the approver who signs it off should be different people. Set the next run date and list the changes that trigger an earlier one.

Testing Without Demographic Data: What the Rules Allow

Can you run a bias audit without collecting demographic data? Often yes, with trade-offs. The options differ in what they allow and what they cost.

In the EU, Article 4a exceptionally allows providers and deployers to process special-category data strictly for bias detection and correction, with safeguards. It does not create a duty to test, and GDPR still applies. Decide the data source and the lawful basis before you collect anything, and involve privacy counsel.

A Worked Example: Impact Ratios for a Resume-Screening Feature

The numbers below are illustrative and invented for teaching. Picture a 25-person SaaS company whose resume-ranking feature advanced candidates in a hiring round. There were 400 applicants in three groups.

The disparate impact ratio is a one-line formula: impact ratio = a group's selection rate divided by the highest group's selection rate. Group B's ratio is 28% divided by 40%, which is 0.70.

Group B falls below 0.80, so the team investigates. It checks whether the difference is statistically and practically meaningful, looks for proxy features in the model, runs a counterfactual test and writes down its decision. Group C, at 0.90, is above the line but still worth a note.

A published NYC Local Law 144 bias audit would report all three groups with these ratios. No fix is mandated by that law, though discrimination law may apply to the result.

AI Bias Audit Tools: What Each One Actually Does

AI bias audit tools compute metrics. They do not decide scope, thresholds or sign-off. For a team with one or two engineers, one library plus a spreadsheet of results and a decision log is enough.

AI Bias Audit Checklist and the Evidence Pack

A customer, auditor or regulator asks to see the same handful of artifacts. Use this AI bias audit checklist as you go, and the audit is mostly paperwork by the end.

Before you call an audit done, run this self-review:

1. The system, decision and groups are written down.
2. The metric and threshold were chosen before testing.
3. The data was checked for representation, labels and proxies.
4. Results show counts, not only ratios.
5. Counterfactual and intersectional cuts were run where they apply.
6. Every gap has a recorded decision.
7. Mitigations were re-tested.
8. A second person signed off, and the next run is scheduled.

The audit maps onto the frameworks like this:

• NIST AI RMF MEASURE 2.11: fairness and bias "are evaluated and results are documented."
• ISO 42001: control titles that apply include assessing AI system impact on individuals or groups (A.5.4), AI system verification and validation (A.6.2.4) and quality of data for AI systems (A.7.4).
• EU AI Act: Article 10 evidence lives in the provider's technical documentation.
• SOC 2: it does not test fairness, as the does SOC 2 cover AI guide explains.

For the controls themselves, see the ISO 42001 checklist and what is ISO 42001.

Who Can Run an AI Bias Audit: Internal, Third-Party or Vendor

The right auditor depends on why you are doing the audit.

New York's test for independence is specific. The auditor cannot work for the employer or the vendor, cannot have been involved in using, developing or distributing the tool, and cannot have a direct or material indirect financial interest in either.

The rule for a small team: if a law requires the audit, hire an independent auditor. If a customer or framework asks for evidence, an internal audit with a second reviewer is usually the starting point.

How Often to Run an AI Bias Audit

New York requires the audit within one year before the tool is used, so it is annual. Everywhere else the answer is: when something changes.

For the monitoring ladder and change triggers that sit around this, see AI model governance.

Mistakes That Make an AI Bias Audit Fail

These are easy to make and expensive to discover during a customer review.

• Choosing the metric after seeing the results. It looks like cherry-picking because it is.
• Testing single attributes only. Intersectional groups can hide gaps that sex or race alone do not show.
• Treating 0.80 as a legal pass mark. It is a screening heuristic, and smaller gaps can still be adverse impact.
• Inferring demographics from names or zip codes. New York bars it, and the errors make results unreliable.
• Trusting the vendor's audit without checking its data. DCWP's FAQ is explicit that the employer stays responsible.
• Auditing once at launch. A model that changes needs a repeat test.
• Writing the record but no decision. A gap with no recorded decision is the first thing a reviewer questions.
• Ignoring the prompt and retrieval layer in LLM features. A clean base model does not make your product fair.

FAQs

What Is an AI Bias Audit?

It is a structured test of whether an AI system produces systematically different outcomes for different groups of people, followed by documented results, a decision on what to fix and a repeat test when the system changes. The term covers a legal audit in New York City, a data-governance duty in the EU AI Act and a voluntary internal fairness test.

Is an AI Bias Audit Legally Required?

Sometimes. New York City requires an annual independent audit for employers using automated hiring tools. The EU AI Act requires providers of high-risk systems to examine and mitigate bias in datasets. Most other rules ask for evaluation, notice or documentation, not an audit by that name.

What Is the Four-Fifths Rule in AI Bias Testing?

It is a federal screening guideline (29 CFR 1607.4(D)). If one group's selection rate is below 80% of the highest group's rate, agencies will generally treat it as evidence of adverse impact. It is a flag to investigate, not proof of discrimination and not a legal safe harbor.

How Do You Test an AI Model for Bias?

Define the decision and groups, check the data, pick fairness metrics in advance, compare outcomes by group, run counterfactual and intersectional tests, decide what each gap means, mitigate and re-test. The seven steps above walk through each one.

How Often Should You Run an AI Bias Audit?

Annually where New York's law applies, and whenever the model, the data, the population or the prompt changes elsewhere. A complaint or a regulation change is also a trigger.

Who Can Conduct an AI Bias Audit?

For a legal audit under Local Law 144, an independent auditor with no role in using, developing or distributing the tool and no financial interest in it. For customer evidence or framework alignment, your own team with a second reviewer is a reasonable start.

Can You Run a Bias Audit Without Collecting Demographic Data?

Yes, in some cases. You can use another party's historical data of the same tool, test data with an explanation, or synthetic and counterfactual data for an internal test. New York does not allow inferred demographics in a legal audit. In the EU, Article 4a offers an exceptional basis to process special-category data for bias detection, with safeguards.

What Is the Difference Between an AI Bias Audit and an AI Risk Assessment?

A risk assessment scores the overall risk of an AI system across many harms. A bias audit produces fairness evidence for one of them. The audit feeds the assessment. See the AI risk assessment guide.

Do LLMs and Generative AI Features Need a Bias Audit?

If they write about, to or for people, test them. Use name-swap prompts and compare tone and refusals across groups. Benchmarks such as BBQ test the base model, so also test your own prompts and retrieval layer.

Related Reading

• AI risk assessment: scoring the risks of one AI system, including unfair outcomes
• AI model governance: the model record, monitoring and change triggers around a model
• AI governance framework: the company-wide program, roles and inventory
• AI governance policy: approved-tool rules and a policy you can adapt
• EU AI Act compliance guide: who the Article 10 duties apply to, and when
• EU AI Act risk classification: how to tell whether a system is high-risk
• NIST AI risk management framework: the voluntary framework behind MEASURE 2.11
• What is ISO 42001: the standard behind the impact and data controls
• ISO 42001 checklist: a step-by-step path to certification
• Does SOC 2 cover AI?: where a SOC 2 report stops on fairness