Lesson 4.1: Ethical and Social Issues Around Data Collection

Unit 4 · Lesson 4.1 · Ethics & Society

Lesson 4.1: Ethical and Social Issues Around Data Collection

🕑 25–30 min · 8 Practice Questions · AP Exam Concept + Mastery

What You'll Learn

  • 4.1.A: Explain the risks to privacy from collecting and storing personal data on computer systems.
  • 4.1.B: Explain the importance of recognizing data quality and potential bias in data sets.
  • Distinguish between personally identifiable information (PII) and non-PII data.
  • Identify sources of bias in data collection and their downstream effects on algorithms.
  • Explain why legal compliance does not equal ethical behavior in data practices.

Key Vocabulary

Term Definition
PII Personally Identifiable Information — any data that can identify a specific individual, such as name, SSN, email, or location.
digital footprint The trail of data a person generates through online activity, including searches, purchases, and location history.
bias Systematic error introduced when a data set does not accurately represent the population it is meant to model.
informed consent A person's knowing agreement to have their data collected, understanding what it will be used for.
data aggregation Combining multiple data sources or records so that non-sensitive data points together reveal sensitive information.
representation bias Bias that occurs when certain groups are over- or under-represented in a data set, causing a trained model to perform poorly for underrepresented groups.

Privacy Risks of Storing Personal Data (4.1.A)

Every time a program stores data about a user, it creates a potential privacy risk. The AP CSA exam expects you to be able to identify what constitutes PII, explain why storing it creates risk, and recognize when combining non-PII data creates a new privacy problem.

What Is PII?

PII is any data that, alone or combined with other data, can identify a specific person. Examples include:

  • Name, address, phone number, email
  • Social Security number, driver's license, passport number
  • Biometric data (fingerprints, face scans)
  • IP address, GPS coordinates, device identifiers
  • Medical and financial records

📌 The Aggregation Problem

Each of the following alone seems harmless: your first name, the city you live in, the college you attended, your employer. But combined, they likely identify you uniquely. This is data aggregation — one of the most commonly tested concepts in this topic on the AP exam.

Why Stored Data Creates Risk

Stored data can be breached, sold, subpoenaed, or used in ways users didn't anticipate. Key risks the AP exam focuses on:

  • Data breaches: Unauthorized access exposes stored PII.
  • Re-identification: Anonymized data can be de-anonymized when combined with other public data.
  • Surveillance: Governments or corporations can track behavior using digital footprints.
  • Purpose creep: Data collected for one reason is later used for another without consent.

⚠️ AP Exam Trap: Legal ≠ Ethical

The AP exam frequently tests whether students can distinguish between what is legal and what is ethical. A company can legally collect and sell your browsing history in many jurisdictions — but that doesn't make it ethical. Watch for MCQ answer choices that conflate the two.

Data Quality and Bias (4.1.B)

When a data set is biased, any algorithm trained on it or run against it will produce biased results — even if the code is written perfectly. Recognizing bias is a software design skill, not just a social awareness skill.

Types of Bias in Data

  • Representation bias: The data set underrepresents some groups. A facial recognition model trained mostly on light-skinned faces performs poorly on darker-skinned faces.
  • Historical bias: Past data encodes past discrimination. A hiring algorithm trained on historical résumés may perpetuate the same patterns.
  • Measurement bias: The tool used to collect data is inaccurate for some groups. A sensor calibrated for one demographic may give systematically wrong readings for another.
  • Feedback loop bias: An algorithm's decisions create more data that reinforces its original bias.

✅ Example: Biased Training Data

A loan-approval algorithm is trained on 10 years of historical approvals. Because past practices denied loans to certain zip codes, the algorithm learns to penalize those zip codes — even if race is not an explicit variable. The data quality problem (biased historical decisions) produces a biased algorithm.

Data Quality Factors

Beyond bias, data quality also depends on:

  • Accuracy: Does the data correctly reflect reality?
  • Completeness: Are there missing values that skew results?
  • Timeliness: Is the data current enough for the use case?
  • Consistency: Is data collected the same way across all records?

Real-World Connection

📌 Why This Section Opens Unit 4

Unit 4 is about arrays, ArrayLists, searching, sorting, and algorithms — powerful tools for processing data at scale. Topic 4.1 comes first to establish a principle: what you do with data has consequences. Every array you process in this unit could represent real people. The AP exam tests whether future programmers think about that before they write the loop.

The Programmer's Responsibility

When you write code that collects, stores, or processes user data, you take on ethical obligations:

  • Collect only what you need (data minimization).
  • Be transparent about what data is collected and why.
  • Secure stored data appropriately.
  • Examine training data for bias before deploying algorithms.
  • Consider who might be harmed by your program's outputs.

Summary

  • PII is any data that can identify a specific individual; combining non-PII data can also create PII through aggregation.
  • Storing personal data creates risks: breaches, re-identification, surveillance, and purpose creep.
  • Legal data practices are not automatically ethical — the AP exam tests this distinction explicitly.
  • Biased data produces biased algorithms; types include representation, historical, measurement, and feedback loop bias.
  • Programmers share responsibility for the ethical use of data their code handles.

Practice Questions

MCQ 1
A website collects users' zip codes, ages, and job titles separately. Each piece of data is publicly available. Which of the following best describes the privacy concern?
Predict the answer before reading the options.
A There is no concern because each data point is public.
B The website is violating federal law by storing this data.
C Combining the data points may allow identification of specific individuals.
D Zip codes are PII and cannot be collected without consent.
✓ C. This is data aggregation — individually harmless data points combined can uniquely identify a person. A is wrong (aggregation creates new risk). B is wrong (no law is stated to be broken). D is wrong (zip codes alone are not PII).
MCQ 2
A company's terms of service allow it to sell user browsing data to advertisers. Users who sign up agree to these terms. Which statement is most accurate?
Predict the answer before reading the options.
A The practice is both legal and ethical because users consented.
B The practice may be legal but could still be considered unethical.
C The practice is illegal because browsing data is PII.
D The practice is ethical only if the company is nonprofit.
✓ B. Burying consent in a terms-of-service agreement may be legally compliant but is widely considered ethically problematic because users often don't meaningfully understand what they're agreeing to. The AP exam frequently tests legal vs. ethical distinction.
MCQ 3
A medical AI is trained on patient data from hospitals in one region of the country. When deployed nationally, it produces less accurate diagnoses for patients from other regions. This is most likely an example of:
A Representation bias
B Feedback loop bias
C Purpose creep
D Data aggregation
✓ A. Representation bias — the training data only represented one region, so the model performs poorly for underrepresented groups. B (feedback loop) occurs when the model's own decisions generate new biased data. C (purpose creep) is using data for a different purpose than collected. D is combining data from multiple sources.
MCQ 4
Which of the following is NOT an example of PII?
Predict the answer before reading the options.
A A person's GPS coordinates logged every 10 seconds
B A user's IP address combined with their login timestamp
C A Social Security number stored in a database
D The average high temperature in Kansas City in July
✓ D. Weather data about a city is not linked to any individual and cannot identify anyone — it is not PII. A, B, and C all can identify or locate specific individuals.
Tier 3 · AP Mastery

Mastery: Ethical and Social Issues Around Data Collection

MCQ 5
A city deploys a predictive policing algorithm trained on 20 years of arrest records. Critics argue the algorithm is biased. Which of the following best explains the concern?
Predict the answer before reading the options.
A Arrest records are PII and cannot be used without individual consent.
B The algorithm is illegal because it uses historical data.
C Historical biases in policing are encoded in the training data, causing the algorithm to reinforce those biases.
D The algorithm has no bias because it uses real data.
✓ C. This is historical bias — past discriminatory practices create skewed training data, and the algorithm perpetuates those patterns. D is a classic misconception: real data can absolutely be biased if it reflects historical inequity.
MCQ 6
A fitness app stores users' step counts and sleep data. The company later begins selling this data to health insurance companies without updating its privacy policy. This scenario most directly illustrates:
Predict the answer before reading the options.
A Data aggregation
B Purpose creep
C Representation bias
D Feedback loop bias
✓ B. Purpose creep — data collected for one purpose (tracking fitness) is being used for a different purpose (insurance evaluation) that users didn't anticipate or consent to. This is a common real-world privacy concern that the AP exam tests.
MCQ 7
Which of the following best represents the principle of data minimization?
Predict the answer before reading the options.
A A shopping app requests only the user's shipping address, not their full contact list.
B A company encrypts all stored user data.
C A database deletes records after 30 days.
D A website displays a privacy policy on its homepage.
✓ A. Data minimization means collecting only what is necessary for the stated purpose. B is about security, C is about retention policy, D is about transparency — all good practices but not specifically data minimization.
MCQ 8
A software team is building an algorithm to predict student success. A team member notices that the training data contains very few records from rural schools. What is the most appropriate next step?
Predict the answer before reading the options.
A Deploy the algorithm and monitor for complaints after launch.
B Exclude rural schools from the algorithm's predictions entirely.
C The lack of rural data is not a concern because the algorithm is mathematically correct.
D Seek additional data from rural schools before training to reduce representation bias.
✓ D. The correct ethical response is to address the data quality problem before deployment. A is reckless (harm occurs before correction). B discriminates. C confuses mathematical correctness with ethical correctness — a well-written algorithm on biased data still produces biased results.

Get in Touch

Whether you're a student, parent, or teacher — I'd love to hear from you.

Just want free AP CS resources?

Enter your email below and check the subscribe box — no message needed. Students get daily practice questions and study tips. Teachers get curriculum resources and teaching strategies.

Typically responds within 24 hours
✓

Message Sent!

Thanks for reaching out. I'll get back to you within 24 hours.

🏫 Welcome, fellow educator!

I offer curriculum resources, practice materials, and study guides designed for AP CS teachers. Let me know what you're looking for — whether it's classroom materials, a guest speaker, or Teachers Pay Teachers resources.

✉

Email

[email protected]

📚

Courses

AP CSA, CSP, & Cybersecurity

⏱

Response Time

Within 24 hours

Prefer email? Reach me directly at [email protected]