Computing Bias

Computing bias happens when a system produces unfair results for certain people or groups. It often comes from biased data or biased design choices, not from the computer "deciding" to be unfair. Because humans build and train these systems, human bias can be built in.

This topic is part of Big Idea 5: Impact of Computing. You will learn how bias enters through training data and design, how feedback loops can amplify it, and how diverse data, testing, and transparency help.

Systems that learn from data depend on training data. If that data does not fairly represent everyone, the system learns a skewed view. Bias can enter at several points: during data collection, in the design choices of the algorithm, and in how results are interpreted and used.

A dangerous pattern is the feedback loop. A biased system makes biased decisions, those decisions shape the real world, and the new data that results carries the same bias back into the system, reinforcing it over time.

Bias is reduced through diverse, representative data, testing outputs across different groups, and transparency about how the system works. Fairness is a goal that must be actively designed for, not assumed.

Visual Feedback Loop

Click to enlarge

Biased data trains an algorithm whose decisions create new data, looping the bias back.

Follow the loop from historical data to algorithm to decision to real-world impact and back to new data. The labels show where bias can enter: data collection, design choices, and interpretation. On the exam, explain both how bias enters and how the loop reinforces it.

• Biased data: data that unfairly represents some groups.
• Algorithmic bias: unfair results produced by a system's logic or data.
• Training data: the data used to teach a system patterns.
• Representation: how fairly groups appear in the data.
• Feedback loop: when biased outputs create biased new data.
• Fairness: producing equitable results across groups.
• Transparency: openness about how a system works.

Be ready to:

• Explain how bias enters through data and design.
• Identify who may be missing from a dataset.
• Describe how a feedback loop reinforces bias.
• Suggest fixes like diverse data, testing across groups, and transparency.

The exam expects you to see bias as a human and data issue, not just a code error, and to propose concrete improvements.

A company builds a hiring algorithm trained mostly on data from past employees who came from one background. How can bias enter, and what is the effect?

• Where bias enters: the training data overrepresents one background, so the system learns that profile as the "ideal" candidate. This is biased data feeding the model.
• Algorithmic effect: the algorithm favors applicants similar to past hires and scores others lower, even if they are well qualified.
• Impact on applicants: qualified people from underrepresented groups are unfairly screened out, reducing diversity.
• Feedback loop: the new hires again come from one background, so the next round of training data is just as skewed, reinforcing the bias.

To reduce the bias, the company should use more diverse and representative training data, test outcomes across different groups, and be transparent about how the algorithm decides. The key insight is that the algorithm was not malicious; the data and design carried human bias into automated decisions.

• Thinking computers cannot be biased. Systems reflect the data and choices of the people who build them.
• Blaming only the algorithm. Biased data is often the real source.
• Ignoring who was missing. Underrepresented groups in the data lead to unfair results.
• Assuming more data removes bias. More biased data just reinforces the bias.
• Failing to test across groups. A system can work well for one group and poorly for another.

When analyzing a biased system, state where the bias entered (data, design, or interpretation) and one concrete fix (diverse data, testing across groups, or transparency). Pairing cause with solution gives complete answers and shows the deeper understanding the exam rewards.

Computing also lets large groups solve problems together. In the next topic, Crowdsourcing, you will learn how many people's contributions combine into powerful shared resources, and how verification keeps the quality high.