AP CSP Big Idea 2 Data Cleaning

AP CSP Topics › Data Cleaning

AP CSP Cleaning & Filtering Data: Complete Guide (2025‑2026)

Data cleaning is the process of identifying and correcting problems in a dataset before analysis. Raw data collected from real-world sources almost always contains errors: missing values, duplicates, outliers, inconsistent formatting, and irrelevant records. If you analyze dirty data, your conclusions are wrong regardless of how sophisticated your analysis is. The principle: garbage in, garbage out. AP CSP tests your ability to identify what cleaning is needed and why filtering changes what conclusions can be drawn.

Contents

  1. Common Data Problems
  2. Cleaning Strategies
  3. Filtering and Its Effects
  4. Common Exam Pitfalls
  5. Check for Understanding (6 Questions)
  6. Frequently Asked Questions
80%Of a data scientist’s time is spent cleaning data, not analyzing it
5Common data problems: missing, duplicate, outlier, inconsistent format, irrelevant records
1Rule: decisions about how to clean data always affect the conclusions you can draw

Common Data Problems

Raw Data vs. Cleaned Data Raw Data (Problems) Name,Age,Score Alice,25,87 ALICE,25,87 ← duplicate Bob,,91 ← missing age Carol,999,74 ← outlier Dave,30, ← missing score CLEAN Cleaned Data Name,Age,Score Alice,25,87 ← kept removed duplicate Bob: imputed or removed Carol: outlier investigated Dave: removed (no score)

Every cleaning decision — impute, remove, cap outliers — affects the resulting dataset. The choice must match the analysis goal.

Scenario — Identify the Problems

A dataset of student test scores contains: (1) a student with age = 999, (2) two identical rows for student “Alice”, (3) a student with no test score recorded, (4) some scores entered as “87%” and others as “0.87”, and (5) records from students who withdrew before taking the test.

List each problem type and the appropriate cleaning action for each.

Answer

(1) Outlier: age 999 is impossible — investigate if it is a data entry error, then correct or remove. (2) Duplicate: remove one copy of the Alice record, keeping only one. (3) Missing value: either remove the record or impute (e.g., use the class average) depending on the analysis goal. (4) Inconsistent format: standardize all scores to the same format (e.g., convert all to decimal 0.87 or all to percentage 87). (5) Irrelevant records: filter out withdrawn students if the analysis is about students who completed the course.

Cleaning Strategies

Handling Missing Values
Three main approaches
  • Remove the record entirely (simplest, loses data)
  • Impute: fill in with mean, median, or most common value
  • Flag as missing: keep the record, note the gap
  • Choice depends on: how much data is missing and why
  • If missing is random: impute. If missing is systematic: investigate
Handling Outliers
Four main approaches
  • Verify: is it a data entry error or a real value?
  • Remove if clearly an error (age = 999)
  • Cap: replace with a maximum valid value
  • Keep if real: extreme values may be important findings
  • Never remove outliers just because they are inconvenient
Scenario — Choose the Cleaning Strategy

A salary dataset has 10,000 records. 200 records have no salary listed. Analysis shows missing salaries are concentrated among part-time employees. A data analyst removes all 200 records and proceeds with analysis.

What problem does this cleaning decision create?

Answer

Systematic exclusion bias. The missing values are not random — they are concentrated among part-time employees. Removing these records means the analysis no longer represents the full workforce; it represents only employees with recorded salaries (disproportionately full-time). Any conclusions about average salaries or compensation patterns will overstate full-time salaries and miss part-time compensation patterns entirely. The cleaning decision changed who is represented in the dataset.

Filtering and Its Effects

Filtering Changes What You Can Conclude
Effect on analysis
  • Filtering to adults only: conclusions apply to adults only
  • Filtering to one region: cannot generalize nationally
  • Filtering to paid subscribers: conclusions about free users are invalid
  • Each filter reduces the population your conclusions describe
  • Document every filter applied to the dataset
When Filtering Is Appropriate
Valid reasons to filter
  • Remove records outside your study population (withdrew before test)
  • Remove known data entry errors after verification
  • Focus analysis on a specific subgroup (by design)
  • Remove test records or seed data from production
  • Every filter must be documented and justified
Scenario — Filtering Effects on Conclusions

A music streaming service analyzes which genres are most popular. They filter out free-tier users ‘because premium users generate more reliable data.’ Their analysis shows classical music is the most popular genre. The marketing team launches a classical-focused campaign.

What went wrong? What conclusion was actually valid?

Answer

The filter changed who the analysis describes. Premium subscribers skew older and higher-income — demographics that prefer classical music at higher rates than the general user base. The valid conclusion is: classical music is most popular among premium subscribers. The invalid generalization is that it is most popular overall. Every filtering decision restricts the population your conclusions can describe.

Common Exam Pitfalls

1
Thinking cleaning data always improves analysis

Cleaning decisions can introduce bias. Removing records with missing values is only safe if missingness is random. If it is systematic, removal creates a biased dataset.

2
Assuming outliers should always be removed

Outliers may be the most important data points. A pharmaceutical trial should not remove patients with extreme reactions — those are precisely the safety signals being studied.

3
Missing that filtering restricts valid conclusions

Every filter you apply to a dataset restricts the population your conclusions can describe. Conclusions from filtered data cannot be generalized to the excluded groups.

4
Treating data cleaning as objective

Decisions about how to handle missing values, outliers, and duplicates involve judgment. Different reasonable choices lead to different datasets and different conclusions. Data cleaning is not a neutral, purely technical process.

Check for Understanding

1. A dataset of customer ages contains the value 150. The most appropriate first step is:

  • Remove the record immediately to prevent it from skewing the analysis.
  • Replace it with the dataset average age.
  • Investigate whether it is a data entry error or a valid extreme value before deciding.
  • Keep it as-is because outliers should never be modified.
Always investigate outliers before acting. A value of 150 is almost certainly a data entry error, but the correct process is to verify first, then correct or remove. Removing without investigation could eliminate a valid record.

2. A researcher filters a national health dataset to include only users of a specific fitness app before analyzing average daily steps. Her conclusion: ‘Americans average 9,200 steps per day.’ What is the problem?

  • Fitness apps do not accurately measure steps.
  • The filtered sample (fitness app users) is not representative of all Americans, so the conclusion cannot be generalized.
  • 9,200 steps per day is outside the valid range for human activity.
  • The dataset should have been filtered to exclude outliers before analysis.
Fitness app users self-selected as people interested in tracking activity. They are not representative of all Americans. The conclusion can only validly describe fitness app users, not Americans generally.

3. Consider statements about data cleaning:
I. Removing records with missing values always improves analysis quality.
II. How data is cleaned can affect what conclusions can be drawn from it.
III. Outliers should always be removed before analysis.

Which are correct?

  • I only
  • II only
  • I and III only
  • I, II, and III
Statement II is correct — cleaning decisions affect which records remain and therefore what conclusions are valid. Statement I is false — removing missing values introduces bias when missingness is systematic. Statement III is false — outliers may be important findings and should be investigated, not automatically removed.

4. A dataset includes duplicate records for 5% of customers. A data analyst removes all duplicates before analysis. This cleaning step most directly prevents:

  • Sampling bias from unrepresentative data collection.
  • Confounding variables from affecting the analysis.
  • Certain customers being counted multiple times and distorting frequency calculations.
  • Missing value imputation from introducing errors.
Duplicate records inflate the count and weight of duplicated entities. Removing duplicates ensures each real-world entity is counted once, producing accurate frequency and statistical calculations.

5. A company analyzes only ‘high-value’ customer transactions (purchases over $500) to understand customer behavior. Their conclusion: ‘Customers primarily buy luxury goods.’ What error is present?

  • They should have filtered for transactions under $50 instead.
  • The filtering restricted conclusions to high-value transactions; the finding cannot be generalized to all customer behavior.
  • Transactions over $500 are always outliers and should have been removed.
  • The analysis should have included returns as well as purchases.
Filtering to high-value transactions defines the analysis population as ‘customers making purchases over $500.’ Of course this group primarily buys expensive items — that is the filter criterion. The conclusion cannot be generalized to all customers.

6. Which principle is best illustrated by the phrase ‘garbage in, garbage out’?

  • Computer programs produce random errors when input is not validated.
  • Data analysis conclusions are only as reliable as the quality of the data being analyzed.
  • Recycling programs require proper sorting of materials.
  • Memory leaks cause programs to slow down over time.
Garbage in, garbage out: if the input data is flawed (biased, dirty, incomplete), the analysis output will be flawed regardless of how sophisticated the analysis method is. Data quality is a prerequisite for valid conclusions.

Frequently Asked Questions

What is imputation and when is it appropriate?
Imputation fills in missing values with estimated values — commonly the mean, median, or most frequent value for that variable. It is appropriate when values are missing at random and the dataset would be significantly reduced by removing missing records. It is inappropriate when missing values are systematic (e.g., high-earners skip income questions — imputing mean income underestimates top incomes).
How does the AP exam test data cleaning?
Expect scenarios presenting a dataset with identified problems, asking you to: name the problem type, describe the appropriate cleaning action, or identify how a specific cleaning decision affects what conclusions can be drawn. Questions often test whether students understand that cleaning decisions are not neutral — they affect the analysis.
What is the difference between cleaning and filtering?
Cleaning corrects problems within records (fixing errors, handling missing values, standardizing formats). Filtering removes entire records from analysis based on criteria (only include adults, only include completed surveys). Both affect the dataset, but filtering explicitly restricts the population your conclusions describe.
← Data Collection & SamplingData Visualization →

Get in Touch

Whether you're a student, parent, or teacher — I'd love to hear from you.

Just want free AP CS resources?

Enter your email below and check the subscribe box — no message needed. Students get daily practice questions and study tips. Teachers get curriculum resources and teaching strategies.

Typically responds within 24 hours
✓

Message Sent!

Thanks for reaching out. I'll get back to you within 24 hours.

🏫 Welcome, fellow educator!

I offer curriculum resources, practice materials, and study guides designed for AP CS teachers. Let me know what you're looking for — whether it's classroom materials, a guest speaker, or Teachers Pay Teachers resources.

✉

Email

[email protected]

📚

Courses

AP CSA, CSP, & Cybersecurity

⏱

Response Time

Within 24 hours

Prefer email? Reach me directly at [email protected]