How to Audit AI for Bias: A Practical Guide for Colorado Businesses

Colorado SB 24-205 makes AI bias audits a legal requirement for any business using AI in consequential decisions — hiring, lending, insurance, healthcare, housing, and education. Enforcement begins June 30, 2026. But "audit your AI for bias" is vague advice. What does a bias audit actually look like? What data do you need? How do you measure bias? And what do you do when you find it? This guide breaks the entire process into concrete, repeatable steps that any Colorado business can follow.

Related: step-by-step bias audit guide · what is an AI bias audit · real examples of AI discrimination

Before you can audit for bias, you need to know what you're auditing. Most businesses are surprised to discover how many AI systems they actually run. **Common hidden AI systems:** - Applicant Tracking Systems (ATS) with resume screening - CRM platforms with AI lead scoring - Chatbots handling customer routing or triage - Automated underwriting or lending tools - Insurance risk scoring algorithms - Healthcare scheduling or triage AI - Predictive analytics dashboards For each system, document: what decisions it makes, who it affects, what data it uses, and whether those decisions are "consequential" under SB 24-205.

SB 24-205 requires testing across all protected classes defined in Colorado anti-discrimination law: - Race and ethnicity - Color - Sex (including gender identity and sexual orientation) - Religion - Age - Disability - National origin - Veteran status Your audit must test whether your AI produces different outcomes — approval rates, scores, recommendations, or denials — for any of these groups at statistically significant levels.

You need decision outcome data broken down by protected class. This typically requires: **Input data:** The features your AI model uses to make decisions **Output data:** The decisions themselves (approved/denied, hired/rejected, scored high/low) **Demographic data:** Protected class membership of the individuals affected **Important:** If you don't collect demographic data directly, you may need to use proxy methods (e.g., BISG — Bayesian Improved Surname Geocoding) or synthetic testing with representative datasets. CO-AIMS can generate synthetic test populations for systems where real demographic data isn't available.

The industry-standard test for algorithmic bias is the **four-fifths rule** (also called the 80% rule), originally from EEOC employment guidelines but now applied broadly to AI: **The rule:** If the selection rate for any protected group is less than 80% (four-fifths) of the selection rate for the group with the highest rate, adverse impact is indicated. **Example:** - Your AI approves 60% of Group A applicants - Your AI approves 42% of Group B applicants - Ratio: 42/60 = 0.70 (70%) - 70% < 80% → **Adverse impact detected** Run this analysis for every protected class, for every consequential decision your AI makes. Document every result — both passing and failing — in your audit trail.

The four-fifths rule is a screening tool, not a final verdict. You also need statistical significance testing to confirm that observed differences aren't due to chance: - **Chi-square test** for categorical outcomes (approved/denied) - **Z-test for proportions** when comparing two group rates - **Fisher's exact test** for small sample sizes A result is typically considered statistically significant at p < 0.05. Document both the four-fifths ratio and the p-value for each comparison. Courts and regulators expect both.

When bias is detected, you need to understand *why* before you can fix it. Common root causes: **Training data bias:** Historical data reflects past discrimination (e.g., hiring data from decades of biased practices) **Feature selection bias:** Proxy variables that correlate with protected classes (zip code → race, name → ethnicity) **Label bias:** The outcome variable itself reflects historical bias **Sampling bias:** Underrepresentation of certain groups in training data **Feedback loop bias:** The model's own outputs reinforce patterns over time Document the root cause for every instance of detected bias. SB 24-205 requires not just detection, but demonstrated effort toward remediation.

Remediation strategies depend on the root cause: - **Rebalance training data** — oversample underrepresented groups or apply synthetic data augmentation - **Remove proxy variables** — drop features that correlate strongly with protected classes - **Adjust decision thresholds** — apply group-specific thresholds to equalize outcome rates - **Retrain the model** — with debiased data and fairness constraints - **Add human review** — implement human-in-the-loop gates for edge cases After remediation, re-run the full audit. Document before/after metrics. This creates the remediation evidence trail that SB 24-205 demands — and that qualifies you for affirmative defense under the law.

A one-time audit isn't enough. SB 24-205 requires ongoing compliance, and AI systems drift over time as data distributions change. **Recommended cadence:** - Monthly automated bias checks for high-risk systems - Quarterly full audits with documented results - Annual comprehensive review with updated impact assessments CO-AIMS automates this entire cycle — from system inventory through bias detection, remediation tracking, and evidence bundle generation. Every audit result is timestamped, versioned, and court-ready.

Manually auditing AI for bias requires data science expertise, statistical knowledge, and 10-20 hours per AI system per month. CO-AIMS compresses this into an automated workflow: 1. **Register your AI systems** in the dashboard (5 minutes per system) 2. **Configure audit parameters** — protected classes, decision types, thresholds 3. **Run automated bias audits** on your schedule or on-demand 4. **Review results** with plain-English explanations and four-fifths ratios 5. **Generate remediation plans** when bias is detected 6. **Export evidence bundles** for legal defense, AG notification, or client sharing Plans start at $199/month. Free 14-day trial with full audit capability.