AP CSP Algorithmic Bias

AP CSP Topics › Algorithmic Bias

AP CSP Algorithmic Bias: Complete Guide (2025‑2026)

Algorithmic bias occurs when a computer algorithm produces systematically unfair or discriminatory outcomes for certain groups. Bias can enter through training data that reflects historical inequalities, data that underrepresents some populations, or through the assumptions baked into the algorithm’s design. Algorithmic bias is a key AP CSP topic because computing algorithms are increasingly used to make high-stakes decisions about hiring, lending, criminal justice, and healthcare.

Contents

  1. Sources of Algorithmic Bias
  2. Real-World Examples
  3. Harms and Accountability
  4. Common Exam Pitfalls
  5. Check for Understanding (10 Questions)
  6. AP Exam Pattern
  7. FAQ
3Main sources: biased training data, unrepresentative data, biased problem framing
0Times an algorithm is intentionally biased for most cases — bias is usually unintentional
100%Accurate does not mean unbiased — an algorithm can be accurate on average but biased for subgroups

Sources of Algorithmic Bias

Sources of Algorithmic Bias Biased Training Data Historical data reflects past discrimination, which algorithm learns and perpetuates Example: resume screener trained on historically male-dominated hiring data Most common source Non-representative Data Training data doesn’t represent all groups equally (sampling bias in data) Example: facial recognition trained mostly on lighter-skinned faces Underrepresentation Biased Problem Framing What the algorithm is designed to optimize encodes assumptions Example: optimizing for credit score proxies correlated with race Design choices matter

Bias usually enters through data, not code. An algorithm that accurately learns from biased historical data will faithfully replicate that bias at scale.

Scenario — Resume Screener

A tech company builds an AI resume screener trained on 10 years of hiring data. The company historically hired mostly men for technical roles. The AI learns patterns associated with successful hires.

How does this algorithm become biased even though the engineers never intended it to be biased?

Answer

The algorithm learns that certain features correlate with past hires: attended certain schools, have certain hobbies, use certain language patterns. These correlations reflect historical hiring patterns that favored men. The algorithm penalizes resumes that mention women’s sports teams or clubs with female-dominated membership, because such applicants were rarely hired historically. The engineers never wrote ‘downrank women;’ but the algorithm learned this pattern from data. The bias is in the training data, not the code.

Real-World Examples

Facial Recognition Bias
Non-representative training data
  • Trained predominantly on lighter-skinned faces
  • Higher error rates for darker-skinned individuals
  • Error rate disparity documented in research
  • Deployed in law enforcement and security
  • Has led to documented wrongful arrests
Predictive Policing Bias
Historical data perpetuates patterns
  • Trained on historical arrest patterns
  • Historical over-policing of certain neighborhoods
  • Algorithm recommends more policing in same areas
  • Creates self-fulfilling feedback loop
  • More arrests → more training data → more predictions

Harms and Accountability

Scenario — Credit Scoring Algorithm

A bank uses an algorithm to decide who gets approved for loans. The algorithm uses ZIP code as a feature. Certain ZIP codes have predominantly minority populations. Applicants from those ZIP codes are denied at higher rates.

Is this bias? Who is harmed? Who is responsible?

Answer

Yes, this is algorithmic bias. ZIP code is a proxy for race — using it introduces racial disparities in lending decisions, which may constitute discrimination even if race is never explicitly used. Harms: qualified applicants are denied credit, reducing wealth-building opportunities for already-disadvantaged communities; the bank reinforces historical redlining patterns. Responsibility: the engineers who chose the features, the managers who approved the system, and the organization deploying it. The AP CSP exam emphasizes that developers have responsibility to test for bias across subgroups, not just overall accuracy.

Common Exam Pitfalls

1
Algorithmic bias is usually unintentional but still harmful

Most biased algorithms were not designed to be biased. The harm is real regardless of intent. The AP exam focuses on how bias enters and how to detect/prevent it.

2
High overall accuracy does not mean unbiased

An algorithm can be 95% accurate overall but 70% accurate for one demographic group. Group-level accuracy disparities are the key measure of bias.

3
Removing protected attributes (race, gender) does not eliminate bias

Proxy variables (ZIP code, school attended, hobbies) can encode the same information. Removing race from inputs does not guarantee a race-neutral outcome.

4
Bias can create feedback loops

An algorithm that over-predicts risk for a group leads to actions (more policing, loan denials) that generate data that further confirms the prediction. The bias compounds over time.

Check for Understanding

1. Algorithmic bias most commonly originates from:

  • Intentional discrimination by programmers.
  • Training data that reflects or amplifies historical human biases.
  • Errors in mathematical calculations within the algorithm.
  • Insufficient computing power to process data accurately.
Most algorithmic bias is not intentional. It arises because historical data reflects past human biases, and algorithms learn from that data.

2. A facial recognition system has a 99% accuracy rate overall but a 75% accuracy rate for darker-skinned faces. This is:

  • Acceptable — 99% overall accuracy is excellent.
  • An example of algorithmic bias — the algorithm performs significantly worse for a demographic group.
  • Not bias because it is not intentional.
  • Unavoidable due to technical limitations.
High overall accuracy masking poor performance for a subgroup is the definition of algorithmic bias. The disparity indicates the training data underrepresented darker-skinned faces.

3. A hiring algorithm never uses gender as an input feature. However, it penalizes resumes that list women’s sports teams. This is:

  • Not bias because gender is not in the input data.
  • Bias through proxy variables — women’s team membership encodes gender indirectly.
  • Acceptable because the algorithm treats all hobbies equally.
  • A programming error, not algorithmic bias.
Proxy variables encode protected characteristics indirectly. Removing the protected attribute itself does not prevent bias when correlated proxies remain.

4. Consider: I. Removing race from an algorithm’s inputs guarantees a race-neutral outcome. II. Bias in training data can cause an algorithm to perpetuate historical discrimination. III. Algorithmic bias is always the result of intentional discrimination.

  • I only
  • II only
  • I and III only
  • I, II, and III
II is correct. I is false — proxy variables can re-introduce bias. III is false — most bias is unintentional.

5. A predictive policing algorithm recommends more police patrols in neighborhoods where more arrests have historically occurred. This may be problematic because:

  • Algorithms should not be used by law enforcement.
  • The algorithm creates a feedback loop: more patrols → more arrests → more predictions of crime → more patrols.
  • The algorithm cannot process geographic data accurately.
  • Historical arrest data is always inaccurate.
Feedback loops in biased systems compound over time: the algorithm’s outputs generate data that confirms the algorithm’s predictions, regardless of actual underlying crime rates.

6. What is the most effective way to test whether an algorithm is biased?

  • Review the code for discriminatory logic.
  • Analyze accuracy and error rates across different demographic groups.
  • Ensure the training dataset is very large.
  • Remove all personal information from the training data.
Testing for demographic disparities in accuracy and error rates is the primary method for detecting algorithmic bias.

7. A loan approval algorithm uses ZIP code as a feature. ZIP codes in the training area correlate strongly with race due to historical residential segregation. This feature:

  • Is acceptable because ZIP code is a neutral geographic variable.
  • Introduces racial bias through a proxy variable, even though race is not explicitly used.
  • Is not biased because the algorithm treats all ZIP codes identically.
  • Only causes bias if the programmer intended it to.
ZIP code is a proxy for race in areas with residential segregation. Using it introduces racial disparities even when race is excluded.

8. Who bears responsibility for addressing algorithmic bias?

  • Only the company’s legal team.
  • Developers who build the algorithm, organizations that deploy it, and policymakers who regulate it.
  • Only the individuals whose data was used for training.
  • No one — algorithms make objective mathematical decisions.
Responsibility is shared: developers must test for bias, organizations must audit deployed systems, and policymakers establish accountability standards.

9. A medical diagnosis algorithm is trained mostly on data from male patients. For which patients might this algorithm underperform?

  • Older male patients only.
  • Female patients, because the algorithm learned patterns predominantly from male physiology.
  • Patients with rare conditions.
  • All patients equally.
Underrepresentation in training data leads to poorer performance for underrepresented groups. Female patients present symptoms differently for many conditions; an algorithm trained on male data may miss these patterns.

10. An algorithm’s designers argue: “We cannot be held responsible for bias because we did not intend to discriminate.” The strongest counterargument is:

  • Intentions are what matter legally.
  • The harm from biased outcomes is real regardless of intent, and developers have a responsibility to test for and mitigate bias before deployment.
  • Any algorithm that is mathematically correct cannot be biased.
  • Bias only matters when it affects a majority population.
Impact matters more than intent in evaluating algorithmic harm. Developers bear responsibility to proactively audit systems for discriminatory impact.

How the AP Exam Tests This

  • Identify the source of algorithmic bias in a given scenario (training data, proxy variables, feedback loops)
  • Explain why removing a protected attribute does not guarantee unbiased outcomes
  • Determine whether a described outcome constitutes algorithmic bias
  • I/II/III: which statements about algorithmic bias are correct
  • Describe a feedback loop created by a biased algorithm

FAQ

Can algorithmic bias be completely eliminated?
In practice, no. All training data reflects the world in which it was collected, and the world contains human bias. The goal is to detect, measure, and minimize bias — not to claim a bias-free system.
What is fairness in the context of algorithms?
Multiple mathematical definitions of fairness exist and can conflict. Equal accuracy across groups, equal false positive rates, and proportional outcomes are three different definitions that cannot all be satisfied simultaneously in most real-world cases.
← Digital DivideCopyright & Creative Commons →

Get in Touch

Whether you're a student, parent, or teacher — I'd love to hear from you.

Just want free AP CS resources?

Enter your email below and check the subscribe box — no message needed. Students get daily practice questions and study tips. Teachers get curriculum resources and teaching strategies.

Typically responds within 24 hours
✓

Message Sent!

Thanks for reaching out. I'll get back to you within 24 hours.

🏫 Welcome, fellow educator!

I offer curriculum resources, practice materials, and study guides designed for AP CS teachers. Let me know what you're looking for — whether it's classroom materials, a guest speaker, or Teachers Pay Teachers resources.

✉

Email

[email protected]

📚

Courses

AP CSA, CSP, & Cybersecurity

⏱

Response Time

Within 24 hours

Prefer email? Reach me directly at [email protected]