How to Conduct AI Bias Audits for Medical Devices Under Colorado and FDA Rules

Bias audits for medical device AI are fundamentally different from standard algorithmic fairness testing for three reasons: **1. The stakes are clinical.** A biased hiring algorithm causes discriminatory employment outcomes. A biased diagnostic AI causes missed diagnoses, delayed treatment, or inappropriate care — outcomes that can directly harm patient health. **2. The protected classes overlap but are not identical.** FDA evaluates device performance across "clinically relevant subgroups" (age, sex, race, disease severity). Colorado SB 24-205 protects against discrimination based on a broader set: race, color, national origin, sex, sexual orientation, gender identity, religion, disability, age, and veteran status. **3. The evidence standards differ.** FDA accepts clinical validation data from device submissions. Colorado requires ongoing monitoring evidence with timestamps, methodology documentation, and remediation records. A one-time clinical trial is not sufficient for Colorado compliance. A properly designed bias audit for medical device AI must satisfy both frameworks in a single process.

Related: general bias audit guide · medical device compliance guide · SB 24-205 compliance guide

Start by creating a unified matrix that covers both FDA and Colorado requirements: **FDA clinically relevant subgroups:** - Age groups (pediatric, adult, geriatric) - Sex (male, female, non-specified) - Race/ethnicity (per FDA demographic categories) - Disease severity or stage - Comorbidities relevant to device performance **Colorado SB 24-205 protected classes:** - Race, color, national origin - Sex, sexual orientation, gender identity - Religion - Disability (physical and mental) - Age - Veteran status **Your unified audit matrix** should test device performance across every intersection that is both clinically relevant and legally protected. For example: a dermatology AI must be tested across skin tones (clinically relevant for FDA, race-related for Colorado) with documented results for each subgroup.

Define what constitutes unacceptable performance disparity before you run the audit. This is critical — without pre-defined thresholds, your audit results are subjective. **Recommended thresholds:** - **Four-Fifths Rule (80% rule):** If the device's accuracy/sensitivity/specificity for any protected subgroup falls below 80% of the highest-performing subgroup, flag it as a finding. This is the EEOC standard adapted for AI. - **Statistical significance:** Use chi-squared or Fisher's exact test to determine whether performance differences are statistically significant (p < 0.05). - **Clinical significance:** Even statistically insignificant differences may be clinically significant if they affect diagnostic sensitivity for serious conditions. Document your threshold methodology. During enforcement, the AG will ask not just "did you test for bias?" but "what standard did you use to determine acceptability?" A documented, pre-defined methodology is far stronger evidence of reasonable care than post-hoc analysis.

**Data requirements:** - Use real-world or representative clinical data that reflects the patient population the device serves - Ensure sufficient sample size per subgroup to detect meaningful disparities (minimum 30 per subgroup, ideally 100+) - Document data sources, collection methods, and any known limitations or gaps **Test categories:** - **Accuracy by subgroup:** Overall accuracy, sensitivity, specificity, positive predictive value, negative predictive value — broken out by each protected class - **False negative disparity:** Particularly critical for diagnostic AI — a higher false negative rate for a specific demographic means that group is disproportionately receiving missed diagnoses - **Confidence calibration:** Does the AI's confidence score correlate equally with actual outcomes across subgroups, or is it overconfident for some groups? - **Edge case performance:** How does the device perform at decision boundaries (e.g., borderline findings) across subgroups? **Document everything:** The methodology, the data sources, the sample sizes, the statistical tests, and every finding — regardless of pass/fail status. Comprehensive documentation of passing results is just as important as findings documentation.

Every audit should produce documentation that satisfies both regulatory regimes simultaneously: **For FDA:** - Performance metrics by clinically relevant subgroup - Comparison against device labeling claims - Any findings that could affect device safety or efficacy - Post-market surveillance report format **For Colorado SB 24-205:** - Algorithmic discrimination analysis across all protected classes - Methodology documentation with pre-defined disparity thresholds - Timestamped audit report with integrity verification - NIST AI RMF control mapping showing which Measure and Manage controls this audit satisfies - Remediation plans for any findings above threshold **Integration point:** Map each audit finding to the specific NIST AI RMF control it relates to. This creates a direct evidence chain: Bias audit finding → NIST control → Remediation action → Verification audit. This chain is the foundation of your legal defense under Colorado law.

A single bias audit is insufficient for either framework. Both FDA and Colorado expect ongoing monitoring: **FDA:** Post-market surveillance obligations require monitoring real-world device performance. For AI/ML devices with Predetermined Change Control Plans, this includes monitoring for performance drift. **Colorado:** "Reasonable care" implies continuous, not one-time, bias monitoring. Monthly automated audits provide the strongest evidence of ongoing diligence. **Recommended cadence:** - Monthly automated bias audits against all protected classes - Quarterly deeper analysis with updated real-world performance data - Annual comprehensive review with methodology updates - Triggered audits after any device update, training data change, or reported adverse outcome Each audit creates a new timestamped evidence record. Over time, this builds an unbroken chain of compliance evidence that predates any potential enforcement action — which is exactly what the 60-day cure window requires you to produce.