Algorithm Bias

For all their differences, big tech companies agree on where we’re heading: into a future dominated by smart machines. Google, Amazon, Facebook, and Apple all say that every aspect of our lives will soon be transformed by artificial intelligence and machine learning, through innovations such as self-driving cars and facial recognition. Yet the people whose work underpins that vision don’t much resemble the society their inventions are supposed to transform. WIRED worked with Montreal startup Element AI to estimate the diversity of leading machine learning researchers, and found that only 12 percent were women.

Health systems rely on commercial prediction algorithms to identify and help patients with complex health needs. We show that a widely used algorithm, typical of this industry-wide approach and affecting millions of patients, exhibits significant racial bias: At a given risk score, Black patients are considerably sicker than White patients, as evidenced by signs of uncontrolled illnesses. Remedying this disparity would increase the percentage of Black patients receiving additional help from 17.7 to 46.5%. The bias arises because the algorithm predicts health care costs rather than illness, but unequal access to care means that we spend less money caring for Black patients than for White patients. Thus, despite health care cost appearing to be an effective proxy for health by some measures of predictive accuracy, large racial biases arise. We suggest that the choice of convenient, seemingly effective proxies for ground truth can be an important source of algorithmic bias in many contexts.

Highlights •The Statistical Learning Theory to study learning guarantees of Decision Trees;•The numerical formulation of the Shattering coefficient for Decision Trees;•Complexity assessment of Decision-Tree models using the Generalization Bound.

As organizations increasingly replace human decision-making with algorithms, they may assume these computer programs lack our biases. But algorithms still reflect the real world, which means they can unintentionally perpetuate existing inequality. A study published Thursday in Science has found that a health care risk-prediction algorithm, a major example of tools used on more than 200 million people in the U.S., demonstrated racial bias—because it relied on a faulty metric for determining need.