Where Bias Comes From

Computing innovations can reflect existing human biases through two primary pathways:

1. Biases written into algorithms -- The rules or decision-making logic of the algorithm encodes assumptions that disadvantage certain groups. Example: a hiring algorithm that penalizes gaps in employment history may disadvantage women who took maternity leave more than men.

2. Biases in training data -- Machine learning systems learn patterns from historical data. If that data reflects historical discrimination or unequal representation, the system learns to reproduce those patterns. Example: a criminal sentencing risk-assessment tool trained on historical conviction data may systematically overestimate recidivism risk for Black defendants because the historical data reflects policing patterns with racial disparities.

Bias Can Be Embedded at Any Level

The CED states explicitly: “Biases can be embedded at all levels of software development.” This means bias isn’t just a data problem or just an algorithm problem -- it can appear at every stage:

• Problem framing: Deciding what to optimize for (maximize engagement vs minimize harm) reflects values and priorities that may disadvantage some groups
• Data collection: Who provides training data, which populations are represented, what historical patterns are included
• Feature selection: Which variables are included in the model and which are excluded can encode discrimination (using zip code as a proxy for race)
• Algorithm design: The mathematical rules that define how decisions are made
• Testing and evaluation: If testing populations don't represent all users, problems affecting underrepresented groups aren't discovered
• Deployment context: Applying a system in contexts it wasn't designed for can amplify biases

Real-World Examples of Computing Bias

What Responsible Programmers Should Do

The CED states: “Programmers should take action to reduce bias in algorithms used for computing innovations as a way of combating existing human biases.”

Practical steps:

• Audit training data for representation -- ensure all affected groups are proportionally represented
• Test across demographics -- don't just test with the most common users; test with edge cases and underrepresented groups
• Examine proxy variables -- zip code, names, and other seemingly neutral variables can encode protected characteristics
• Include diverse perspectives in development teams -- people from different backgrounds catch biases that homogeneous teams miss
• Monitor after deployment -- bias may emerge in production that wasn't visible in testing

Practice MCQs

Predict your answer before clicking. These questions match AP exam difficulty and phrasing.