AP CSP Big Idea 2 Data Collection
AP CSP Data Collection, Sampling & Bias: Complete Guide (2025‑2026)
Data is only useful if it accurately represents the population you are studying. Sampling bias occurs when the group you collect data from is not representative of the broader population — making your conclusions wrong even if your analysis is perfect. AP CSP tests the causes of sampling bias, how to recognize it in scenarios, and the difference between correlation and causation. Biased data produces biased conclusions regardless of how sophisticated the analysis is.
Contents
Representative vs. Biased Sampling
A biased sample produces conclusions that cannot be generalized to the full population. The Literary Digest poll surveyed car and phone owners — wealthier demographics — and predicted the wrong winner.
A social media company surveys users about their satisfaction with the platform by posting the survey in the app. 85% of respondents report being satisfied. The company claims ‘85% of users are satisfied.’
What sampling bias is present? Who is underrepresented?
Self-selection and availability bias. Only users who are actively engaged with the app see and respond to the survey. Users who are dissatisfied and have reduced or stopped using the app are systematically underrepresented — the most unhappy users left and therefore cannot be surveyed. The 85% satisfaction figure describes engaged users, not all users. The company’s generalization from the sample to all users is invalid.
Correlation vs. Causation
- Ice cream sales and drownings both rise in summer
- Nicolas Cage films and pool drownings correlate over years
- Shoe size and reading level correlate in children
- City size and crime count correlate globally
- Pattern exists but no causal link
- Smoking causes increased lung cancer risk
- Exercise causes improved cardiovascular health
- Lack of sleep causes reduced cognitive performance
- Causation requires controlled experiments
- Correlation alone is never sufficient to establish causation
A study finds that cities with more coffee shops per capita have higher average incomes. A coffee chain uses this data to argue: ‘Opening more coffee shops in low-income areas will raise incomes there.’
What is the logical error? What likely explains the correlation?
Confusing correlation with causation. The correlation likely has a confounding variable: wealthy cities with higher incomes attract more coffee shops (because people can afford them) AND have higher incomes. Coffee shop density does not cause higher incomes — both are caused by a third variable (economic activity / wealth). Opening coffee shops in low-income areas will not raise incomes because the causal arrow points in the opposite direction from what the chain claims.
Data Collection Methods
- Surveys with fixed response options
- Sensor readings at regular intervals
- Transaction logs from a system
- Lab measurements with standard protocols
- Easy to analyze: consistent format
- Open-ended survey responses
- Social media posts and comments
- Images, audio, video
- Email and document text
- Requires processing before analysis
A school wants to know if students prefer in-person or online learning. They email a survey to all students and receive 340 responses out of 1,200 students enrolled. 78% of respondents prefer in-person learning. The principal announces: ‘Our students overwhelmingly prefer in-person learning.’
What sampling concerns limit this conclusion?
Two concerns: (1) Non-response bias — only 28% responded. Students who feel strongly (in either direction) are more likely to respond; those with moderate opinions may have ignored the survey. (2) Mode of collection — email requires digital access and engagement. Students with less reliable tech access are less likely to see and respond. The 78% figure describes willing email respondents, not all students. The principal’s generalization is overconfident.
Common Exam Pitfalls
This is the most common Big Idea 2 error. Even strong correlations (r close to 1.0) do not establish causation. Always ask: is there a confounding variable? Does the causal arrow point in the assumed direction?
The Literary Digest sent 10 million surveys and received 2.4 million responses — and predicted the wrong winner. Sample size does not fix sampling method. A biased sampling method applied at massive scale produces massively biased results.
Voluntary surveys attract people who feel strongly about the topic. Passive data collection (transaction logs, sensor data) is often more representative than opt-in surveys.
Data available for analysis was collected for some purpose and reflects that collection method’s biases. Wikipedia is written by people who edit Wikipedia. Twitter data reflects Twitter users. Always ask who produced the data and who is excluded.
Check for Understanding
1. A researcher surveys gym members about their exercise habits and finds they exercise 5 days per week on average. She concludes the average American exercises 5 days per week. What is wrong with this conclusion?
- The sample size is too small to draw any conclusion.
- The sample is biased: gym members are not representative of the general population’s exercise habits.
- The researcher should have used a 10-day week for the survey.
- Exercise frequency cannot be measured through surveys.
2. A study finds a strong positive correlation between students’ shoe sizes and their reading test scores. The most likely explanation is:
- Larger feet indicate faster brain development, improving reading.
- Reading practice strengthens the same neural pathways that control foot growth.
- Both shoe size and reading score increase with age, making age the confounding variable.
- The study was conducted at a school that selects students based on foot size.
3. Consider statements about sampling:
I. A biased sampling method applied to a larger sample will still produce biased results.
II. Random sampling helps ensure the sample is representative of the broader population.
III. Online surveys always produce unbiased data because anyone can respond.
Which are correct?
- I only
- I and II only
- II and III only
- I, II, and III
4. A data analyst finds that cities with more hospitals have higher death rates. She concludes hospitals cause deaths. What concept explains why this conclusion is wrong?
- Sampling bias, because rural areas were excluded.
- Correlation does not imply causation — sick people go to cities with hospitals, not the reverse.
- The data was collected incorrectly using structured instead of unstructured methods.
- The hospital count was measured in different units across cities.
5. A company surveys customers who called their help line to measure product satisfaction. The survey finds 60% are satisfied. Which limitation is most significant?
- Phone surveys have lower response rates than email surveys.
- Satisfied customers rarely call help lines, so the sample is biased toward dissatisfied users.
- The 40% dissatisfied customers should have been excluded from the analysis.
- 60% satisfaction is always below industry standards and indicates a problem.
6. A researcher collects data showing ice cream sales and drowning incidents both peak in July. The best conclusion is:
- Ice cream consumption causes drowning by impairing swimming ability.
- Drowning causes increased ice cream consumption as a stress response.
- Both variables are correlated because both increase in summer heat, not because one causes the other.
- The correlation proves that beach concession stands should not sell ice cream.
Frequently Asked Questions
Get in Touch
Whether you're a student, parent, or teacher — I'd love to hear from you.
Just want free AP CS resources?
Enter your email below and check the subscribe box — no message needed. Students get daily practice questions and study tips. Teachers get curriculum resources and teaching strategies.
Message Sent!
Thanks for reaching out. I'll get back to you within 24 hours.
Prefer email? Reach me directly at [email protected]