EquityPanel audit report

Key numbers

Small groups (fewer than 30 readmitted or not-readmitted patients; treat their numbers as uncertain): Race: Asian, Race: Other, Sex: Unknown.

Summary

AI-generated summary. An AI model wrote these findings from the report's aggregate numbers only; it saw no individual patient records. Software confirmed that each number it cites matches a value in the tables below. The wording, comparisons and conclusions were not checked and may be wrong or misleading, and no person has reviewed this summary. Rely on the tables, and have a qualified person review before acting on anything here.

• Younger patients are missed far more often. The false negative rate is the share of patients who were readmitted but were not flagged for follow-up. It is 87.0% for under-50s and 85.2% for 50-59s, against 74.5% for the 70-79 reference group. Those gaps are +12.6 percentage points (95% CI 7.1 to 18.1) and +10.8 percentage points (CI 5.2 to 16.6). Both intervals sit entirely above zero. Younger patients are also flagged much less often: 4.3% of under-50s and 4.7% of 50-59s, versus 14.7% of 70-79s. Those are gaps of -10.5 and -10.1 percentage points. [age_band.<50.fnr, age_band.50-59.fnr, age_band.70-79.fnr, age_band.<50.fnr_gap, age_band.50-59.fnr_gap, age_band.<50.flag_rate, age_band.50-59.flag_rate, age_band.70-79.flag_rate, age_band.<50.flag_rate_gap, age_band.50-59.flag_rate_gap]
• Overall the model ranks patients only modestly well. AUC measures how often the model gives a readmitted patient a higher risk score than a non-readmitted one; a coin toss would score one half. Across 20,997 test patients the AUC is 0.64. The readmission rate is 9.0%, and 9.9% of patients are flagged. Ranking is weakest for the oldest patients: 0.58 for 80+ (CI 0.56 to 0.61) versus 0.61 for 70-79 (CI 0.59 to 0.64). Those two intervals overlap, so that difference is uncertain. [model.auc_test, model.n_test, model.prevalence_test, model.flag_share_test, age_band.80+.auc, age_band.70-79.auc]
• Patients recorded as 'Other' race are flagged as small_group, so their numbers are uncertain: 375 patients, only 25 of them readmitted. Within that caution, 96.0% of their readmitted patients were not flagged (CI 87.0% to 100.0%), compared with 77.7% for White patients (CI 75.4% to 79.7%). The gap is +18.3 percentage points (CI 9.0 to 24.3). Their AUC is 0.55, close to chance level, with a CI of 0.44 to 0.65. [race.Other.n, race.Other.n_pos, race.Other.fnr, race.White.fnr, race.Other.fnr_gap, race.Other.auc]
• Black patients are flagged less often than White patients: 8.0% versus 10.6%, a gap of -2.6 percentage points (CI -3.5 to -1.6). The share of readmitted patients who were missed is similar: 79.6% for Black patients (CI 75.2% to 83.7%) and 77.7% for White patients (CI 75.4% to 79.7%). These intervals overlap, and the gap of +1.9 percentage points has a CI of -2.8 to 6.6. [race.Black.flag_rate, race.White.flag_rate, race.Black.flag_rate_gap, race.Black.fnr, race.White.fnr, race.Black.fnr_gap]
• Calibration asks whether the predicted number of readmissions matches the real number. It is measured as observed/expected (O/E): a value near one means they match, and a higher value means more readmissions happened than predicted. O/E is 1.06 for Black patients (CI 0.96 to 1.16) and 1.00 for White patients (CI 0.95 to 1.05). These intervals overlap. For patients of unknown race O/E is 0.77 (CI 0.54 to 1.01), meaning fewer readmissions than predicted, but this is uncertain. [race.Black.o_e, race.White.o_e, race.Unknown.o_e]
• Men are flagged less often than women: 8.9% versus 10.8%, a gap of -1.9 percentage points (CI -2.7 to -1.1). The share of readmitted patients who were missed is 80.1% for men (CI 77.4% to 82.6%) and 76.9% for women (CI 74.5% to 79.5%). These intervals overlap. Only 2 patients have unknown sex, flagged small_group, so their results cannot be interpreted. [sex.Male.flag_rate, sex.Female.flag_rate, sex.Male.flag_rate_gap, sex.Male.fnr, sex.Female.fnr, sex.Unknown.n]
• Asian patients are flagged small_group: 153 patients, only 8 of them readmitted. Their numbers are very uncertain. Of their readmitted patients, 87.5% were not flagged, but the CI runs from 57.1% to 100.0%. Their AUC of 0.68 has a CI of 0.50 to 0.86. [race.Asian.n, race.Asian.n_pos, race.Asian.fnr, race.Asian.auc]
• Kidney function (eGFR) results come from synthetic patients, not real-world rates. They compare the older race-based equation with the newer race-free one. Among 542 synthetic Black patients, 111 moved to a worse kidney stage under the race-free equation. The number at or below the line where transplant waiting time can start rose from 60 to 89, with 29 newly below it. Among 4,963 synthetic non-Black patients, the number at or below that line fell from 1,142 to 911, with 231 newly above it. [egfr.Black.n, egfr.Black.n_lower, egfr.Black.waitlist_le20_2009, egfr.Black.waitlist_le20_2021, egfr.Black.waitlist_le20_newly_below, egfr.non-Black.n, egfr.non-Black.waitlist_le20_2009, egfr.non-Black.waitlist_le20_2021, egfr.non-Black.waitlist_le20_newly_above]

By race

Each cell shows the estimate and its 95% confidence interval. Gaps are differences from the reference group, White (the largest); gaps in rates are in percentage points (pts).

Gaps vs White

By sex

Each cell shows the estimate and its 95% confidence interval. Gaps are differences from the reference group, Female (the largest); gaps in rates are in percentage points (pts).

Gaps vs Female

By age band

Each cell shows the estimate and its 95% confidence interval. Gaps are differences from the reference group, 70-79 (the largest); gaps in rates are in percentage points (pts).

Gaps vs 70-79

eGFR: race-based (2009) vs race-free (2021) kidney formula

Synthetic patients (Synthea). Not real-world rates. The direction of the change matches published real-world studies; the sizes do not estimate real prevalence.

Until 2021 the standard kidney-function formula (CKD-EPI 2009) multiplied a Black patient's result by 1.159, making their kidneys look healthier. Counts below are patients below each clinical line under 2009 → under 2021. The equations are computed only to count how this cohort moves; they are not for estimating any individual's kidney function.

Methods and limitations

Flagging rule

A patient is flagged when their risk score is at or above the threshold (0.147, set on training patients to flag about 10%).

Confidence intervals

95% stratified bootstrap (1000 resamples, seed 0): patients are resampled within each group and outcome.

Reference group

The largest group in each column; gaps are group minus reference.

Small groups

Flagged, not dropped, when they have fewer than 30 readmitted or fewer than 30 not-readmitted patients.

Summary model

claude-opus-5-5 via the Anthropic API, given only aggregate numbers (2 attempt(s)). Small-cell suppression before sending: off (public data).

Metrics

AUC: how well scores rank readmitted above not-readmitted patients (0.5 = chance). Brier: average squared error of the risk (lower is better). O/E: observed ÷ expected readmissions (1 = calibrated; above 1 = risk underestimated). FPR: flagged but not readmitted. FNR: readmitted but not flagged. PPV: flagged patients who were readmitted.

What this cannot tell you

Why a gap exists, whether the labels themselves are biased (who gets readmitted depends on access to care), or how the model performs on other hospitals' patients. Race, sex and age are audit variables, not model inputs.