What Is an AI Bias Audit? Definition, Process, and Why Colorado Requires It

An **AI bias audit** is a systematic, documented evaluation of whether an artificial intelligence system produces unfair or discriminatory outcomes for individuals based on protected characteristics — race, gender, age, disability, ethnicity, religion, or other legally protected classes. The audit measures whether the AI's decisions (approvals, denials, scores, recommendations, classifications) differ across demographic groups at rates that indicate discrimination. The primary statistical tool is **disparate impact analysis**, most commonly measured using the **four-fifths rule** (80% rule). Under Colorado SB 24-205, AI bias audits are a legal requirement for any business using AI systems that make or substantially assist in "consequential decisions." Enforcement begins June 30, 2026.

Related: step-by-step bias audit guide · practical bias audit process · algorithmic bias explained

**Selection rates by group:** What percentage of each demographic group receives a favorable outcome (approved, hired, funded) versus an unfavorable outcome (denied, rejected, flagged)? **Disparate impact ratio:** The ratio of the lowest group's selection rate to the highest group's rate. If below 0.80 (80%), adverse impact is indicated. **Statistical significance:** Whether observed differences are likely due to the AI's behavior rather than random chance (typically p < 0.05). **Error rate disparities:** Whether false positive rates and false negative rates differ across groups. A system that falsely denies one group at higher rates than another shows bias even if overall accuracy appears balanced. **Feature importance analysis:** Which input variables drive the AI's decisions, and whether any serve as proxies for protected characteristics (e.g., zip code correlating with race). **Intersectional analysis:** Testing not just single protected classes but intersections — e.g., Black women may experience different outcomes than Black men or white women, even if each individual category appears balanced.

Any Colorado business using AI for **consequential decisions** must conduct bias audits. Consequential decisions include: - **Employment:** Hiring, promotion, termination, compensation, task assignment - **Financial services:** Lending, credit, insurance underwriting, pricing - **Healthcare:** Diagnosis, treatment recommendations, triage, coverage determinations - **Housing:** Tenant screening, rental pricing, mortgage approvals - **Education:** Admissions, grading, financial aid, disciplinary actions - **Legal services:** Risk assessment, case evaluation, settlement recommendations **Both developers and deployers** have obligations. If you build AI that others use for consequential decisions, you must provide bias testing data. If you deploy AI for those decisions, you must conduct ongoing audits. The law applies regardless of business size. A 5-person startup using AI hiring tools has the same audit obligation as a Fortune 500 company.

**Step 1: System identification** — Document which AI systems make consequential decisions and which protected classes they affect. **Step 2: Data collection** — Gather decision outcome data broken down by demographic group. If direct demographic data isn't available, use proxy methods or synthetic testing. **Step 3: Disparate impact analysis** — Calculate selection rates per group and apply the four-fifths rule. A ratio below 0.80 indicates adverse impact. **Step 4: Statistical significance testing** — Run chi-square, z-tests, or Fisher's exact tests to confirm observed disparities aren't due to chance. **Step 5: Root cause investigation** — When bias is found, determine why: training data bias, proxy variables, feedback loops, or deployment context. **Step 6: Remediation** — Fix identified issues: retrain models, remove proxy features, adjust thresholds, or add human review. **Step 7: Re-testing** — Run the audit again post-remediation to verify the fix worked. **Step 8: Documentation** — Record all findings, remediation actions, and results in your compliance evidence trail. SB 24-205 requires 3+ years of retention.

**SB 24-205 doesn't specify a frequency**, but industry best practice and NIST AI RMF guidance suggest: - **Monthly automated checks** for high-risk systems (hiring, lending, healthcare) - **Quarterly full audits** with documented statistical analysis - **Annual comprehensive review** including impact assessments and policy updates - **Event-triggered audits** whenever you change training data, modify model parameters, deploy to a new population, or receive a complaint AI systems drift over time. Data distributions shift, user behavior changes, and models degrade. A system that passed bias testing in January may fail by June. Continuous monitoring isn't gold-plating — it's the only way to maintain compliance between formal audits.

**Legal consequences:** - $20,000+ per violation per incident under SB 24-205 - Each affected consumer is a separate violation - No affirmative defense available without documented bias auditing - AG investigation and Civil Investigative Demands (CIDs) **Business consequences:** - Reputational damage if algorithmic discrimination is discovered - Loss of enterprise contracts (procurement teams increasingly require AI compliance documentation) - Insurance implications for unmanaged AI risk - Litigation from affected individuals **The math:** A single AI system affecting 1,000 customers without proper bias auditing could create $20 million+ in potential violations. The cost of auditing: $199/month with CO-AIMS.