Prejudice is a human trait in which different types of people are treated differently. There have been countless laws and amendments to the Constitution to address it.
The last place you would expect to find discrimination is in the cold, dispassionate digits of a computer algorithm.
That's why researchers at Carnegie Mellon University were flabbergasted when their experiments showed that when male and female computer users searched on Google, men were shown a lot more ads promising help in getting high-paying jobs than women were.
The researchers used a tool called AdFisher that conducts experiments with simulated user profiles. Anupam Datta, associate professor of computer science and of electrical and computer engineering, says the results are clear – online gender discrimination is real.
“Out of the blue”
"This just came out of the blue," Datta said of the gender discrimination finding, which was part of a larger study of the operation of Google's Ad Settings Web page, formerly known as Ad Preferences.
He says the bigger question is “why.” Was it the preference of advertisers or was it the unintended consequence of machine learning algorithms that drive online recommendation engines?
AdFisher created hundreds of simulated users, enabling researchers to run browser-based experiments to identify various effects from changes in preferences or online behavior. The program uses a set of other machine learning tools to analyze the results and perform statistical analyses.
"Many important decisions about the ads we see are being made by online systems," Datta said. "Oversight of these 'black boxes' is necessary to make sure they don't compromise our values."
In the case of the job ads, the differences were profound. Specifically, the experiment looked at who was shown ads for high-paying jobs when visiting employment sites.
AdFisher was used to create 1,000 simulated users - half male, half female - and had them visit 100 top job-listing sites. AdFisher then reviewed the ads that were shown to the simulated users and found the site most strongly associated with the male profiles was a career coaching service for executive positions paying more than $200,000.
The researchers say male users were shown the high-paying job ads about 1,800 times, compared to female users who saw those ads about 300 times. Drilling deeper, the researchers found the ads most associated with female profiles were for a generic job posting service and an auto dealer.
Looking inside the black box
"We can't look inside the black box that makes the decisions, but AdFisher can find changes in preferences and changes in the behavior of its virtual users that cause changes in the ads users receive," said Michael Carl Tschantz, a researcher at the International Computer Science Institute in Berkeley, Calif.
The Carnegie Mellon researchers are quick to point out that they have no evidence that Google is doing anything illegal or that it violates its own policies. While they say Adfisher can identify discrepancies, it can't explain why they occur without a look inside the black box.
However, they add that the discrepancies they found could come from the advertiser or Google's system targeting males.