What is False negative?

A qualified applicant incorrectly screened out by the system.

What is False positive?

An unqualified applicant incorrectly advanced by the system.

What is Proxy variable?

A feature that carries information about another characteristic, even when that characteristic is not directly provided.

What is Selection rate?

The proportion of people in a group who receive a favorable outcome, such as advancing to an interview.

What is Historical bias?

Patterns from past inequalities or decisions that become encoded in training data.

What is Group-level evaluation?

Measuring system performance separately across relevant groups instead of relying only on overall averages.

The Resume Filter That Learned Yesterday’s Bias — Responsible AI: From Hidden Bias to Accountable Deployment

Start With the Training Records

The Dataset Is a Record of Decisions

A resume filter trained on past hiring decisions does not learn merit directly. It learns relationships between resume features and earlier outcomes, including patterns created by unequal access or biased judgment.

History Can Look Like Evidence

If successful applicants usually attended a small group of schools, the model may treat those schools as proof of ability. But the pattern may reflect opportunity and recruiting habits rather than job performance.

Name the Mechanism

Historical bias enters when past inequalities or decisions are encoded in the training examples. The model can reproduce them even when nobody explicitly tells it to discriminate.

A Pattern That Looks Efficient

The Visible Rule

The system rewards uninterrupted employment because that pattern appeared often among people previously hired. On a dashboard, the rule may look neutral and predictive.

The Hidden Context

A gap in employment can have many causes. Treating every gap as weak commitment ignores the different conditions under which candidates built their experience.

The Uneven Effect

A single rule can reduce scores for one group more often than another. The model may never use a protected attribute explicitly, yet still produce systematically uneven outcomes.

Find the Proxy Variables

Correlation Is Not the Same as Relevance

A feature can predict past hiring without measuring job ability. A postal code might correlate with hiring outcomes because of transportation, school access, or recruiting geography.

Use a Three-Part Test

For each feature, ask what it represents, whether that representation matters for the job, and whether its effect differs across groups. This turns suspicion into a testable investigation.

Do Not Use Blindness as Proof

Removing race or gender from the input does not guarantee fairness. Other variables may preserve much of the same information and continue the same pattern.

Trace One Resume Feature

Choose one feature a resume filter might use: school attended, employment gap, commute distance, writing style, or previous job title.

Write a short chain:

Feature -> possible proxy -> historical pattern -> affected group -> possible consequence

Example: commute distance -> household resources or transportation access -> fewer nearby applicants historically hired -> candidates from distant or underserved areas -> qualified applicants receive lower scores.

Now ask: what job-related evidence could replace the shortcut? A skills assessment, structured work sample, or verified experience may measure the requirement more directly.

Accuracy Can Hide Unequal Errors

Two Different Mistakes

A qualified person rejected by the filter is a false negative. An unqualified person advanced is a false positive. These errors have different consequences for candidates and employers.

The Average Can Mislead

Overall accuracy combines everyone into one number. A model may look reliable while screening out qualified candidates from one group much more often than from another.

Measure the Pipeline

Compare group-level results at several stages: screening, interview selection, offers, and hiring. A fairness problem may appear after the initial score rather than inside it.

When the Threshold Creates the Gap

Same Cutoff, Different Consequence

A score threshold of 70 treats groups identically on paper. But if 35 percent of qualified candidates in one group fall below it, the practical effect is unequal access to interviews.

Numbers Make the Harm Visible

Among 100 qualified candidates, a 20 percent false-negative rate rejects 20 people. A 35 percent rate rejects 35. The gap is easier to see than a single accuracy statistic.

Thresholds Are Value Choices

Lowering or raising a cutoff changes false positives and false negatives. Selecting the threshold requires context, evidence, and an explicit judgment about acceptable risk.

Build a Group-Level Evaluation Table

Compare Outcomes, Not Just Scores

A group-level table links model behavior to real decisions. Include selection, false-negative, and false-positive rates, then follow the process through interviews and offers.

Record Uncertainty

Small groups or missing qualification data can make rates unstable. Document sample sizes and uncertainty instead of presenting every difference as equally reliable.

Explain the Difference

A gap is a signal for investigation, not the end of the analysis. Trace it back to training labels, proxy features, missing data, or decision thresholds.

Check Understanding

Choose the explanation that best traces the source of an uneven outcome.

A resume filter never receives gender data, but it heavily penalizes employment gaps. Qualified caregivers are rejected more often than other qualified applicants. What is the strongest explanation?

The employment-gap feature may act as a proxy for unequal social conditions, and the penalty creates a higher false-negative rate for that group.
The model is fair because gender was not included as an input.
The result proves that caregivers are less qualified for the job.

Show Answer

Answer: A) The employment-gap feature may act as a proxy for unequal social conditions, and the penalty creates a higher false-negative rate for that group.

The feature may encode a proxy for unequal circumstances. The next step is group-level evaluation: measure false-negative rates, inspect the training labels, and test whether the feature is genuinely job-related.

Key Terms

False negative: A qualified applicant incorrectly screened out by the system.
False positive: An unqualified applicant incorrectly advanced by the system.
Proxy variable: A feature that carries information about another characteristic, even when that characteristic is not directly provided.
Selection rate: The proportion of people in a group who receive a favorable outcome, such as advancing to an interview.
Historical bias: Patterns from past inequalities or decisions that become encoded in training data.
Group-level evaluation: Measuring system performance separately across relevant groups instead of relying only on overall averages.