Skip to content

Cross Post: Biased Algorithms: Here’s a More Radical Approach to Creating Fairness

  • by

Written by Dr Tom Douglas

File 20190116 163283 1s61b5v.jpg?ixlib=rb 1.1

Our lives are increasingly affected by algorithms. People may be denied loans, jobs, insurance policies, or even parole on the basis of risk scores that they produce.

Yet algorithms are notoriously prone to biases. For example, algorithms used to assess the risk of criminal recidivism often have higher error rates in minority ethic groups. As ProPublica found, the COMPAS algorithm – widely used to predict re-offending in the US criminal justice system – had a higher false positive rate in black than in white people; black people were more likely to be wrongly predicted to re-offend.

Corrupt code.
Vintage Tone/Shutterstock

Findings such as these have led some to claim that algorithms are unfair or discriminatory. In response, AI researchers have sought to produce algorithms that avoid, or at least minimise, unfairness, for example, by equalising false positive rates across racial groups. Recently, an MIT group reported that they had developed a new technique for taking bias out of algorithms without compromising accuracy. But is fixing algorithms the best way to combat unfairness?

It depends on what kind of fairness we’re after. Moral and political philosophers often contrast two types of fairness: procedural and substantive. A policy, procedure, or course of action, is procedurally fair when it is fair independently of the outcomes it causes. A football referee’s decision may be fair, regardless of how it affects the game’s outcome, simply because the decision was made on the basis of an impartial application of the rules. Or a parent’s treatment of his two children may be fair because it manifests no partiality or favouritism, even if it has the result that one child’s life goes much better than the other’s.

Making a call.

By contrast, something that is substantively fair produces fair outcomes. Suppose a football referee awards a soft penalty to a team that is 1-0 down because she thinks the other team’s lead was the result of pure luck. As a result, the game finishes in a 1-1 draw. This decision seems procedurally unfair – the referee applies the rules less stringently to one team than the other. But if a draw reflects the relative performance of the two teams, it may be substantively fair.

Alternatively, imagine that a mother and father favour different children. Each parent treats the disfavoured child unfairly, in a procedural sense. But if the end result is that the two children receive equal love, then their actions may be substantively fair.

Playing favourites.

What’s fair?

AI researchers concerned about fairness have, for the most part, been focused on developing algorithms that are procedurally fair – fair by virtue of the features of the algorithms themselves, not the effects of their deployment. But what if it’s substantive fairness that really matters?

There is usually a tension between procedural fairness and accuracy – attempts to achieve the most commonly advocated forms of procedural fairness increase the algorithm’s overall error rate. Take the COMPAS algorithm for example. If we equalised the false positive rates between black and white people by ignoring the predictors of recidivism that tended to be disproportionately possessed by black people, the likely result would be a loss in overall accuracy, with more people wrongly predicted to re-offend, or not re-offend.

We could avoid these difficulties if we focused on substantive rather than procedural fairness and simply designed algorithms to maximise accuracy, while simultaneously blocking or compensating for any substantively unfair effects that these algorithms might have. For example, instead of trying to ensure that crime prediction errors affect different racial groups equally – a goal that may in any case be unattainable – we could instead ensure that these algorithms are not used in ways that disadvantage those at high risk. We could offer people deemed “high risk” rehabilitative treatments rather than, say, subjecting them to further incarceration.

Alternatively, we could take steps to offset an algorithm’s tendency to assign higher risk to some groups than others – offering risk-lowering rehabilitation programmes preferentially to black people, for instance.

Aiming for substantive fairness outside of the algorithm’s design would leave algorithm designers free to focus on maximising accuracy, with fairness left to state regulators, with expert and democratic input. This approach has been successful in other areas. In medicine, for instance, doctors focus on promoting the well-being of their patients while health funders and policymakers promote the fair allocation of healthcare resources across patients.

Doctors focus on individual well-being, others focus on overall health.
Stuart Jenner/Shutterstock

In substance or procedure

Of course, most of us would be reluctant to give up on procedural fairness entirely. If a referee penalises every minor infringement by one team, while letting another get away with major fouls, we’d think something had gone wrong – even if the right team wins. If a judge ignores everything a defendant says and listens attentively to the plaintiff, we’d think this was unfair, even if the defendant is a jet-setting billionaire who would, even if found guilty, be far better off than a more deserving plaintiff.

We do care about procedural fairness. Yet substantive fairness often matters more – at least, many of us have intuitions that seem to be consistent with this. Some of us think that presidents and monarchs should have the discretion to offer pardons to convicted offenders, even though this applies legal rules inconsistently – letting some, but not others, off the hook. Why think this is justified? Perhaps because pardons help to ensure substantive fairness where procedurally fair processes result in unfairly harsh consequences.

Many of us also think that affirmative action is justified, even when it looks, on the face of it, to be procedurally unfair, since it gives some groups greater consideration than others. Perhaps we tolerate this unfairness because, through mitigating the effects of past oppression, affirmative action tends to promote substantive fairness.

If substantive fairness generally matters more than procedural fairness, countering biased algorithms through changes to algorithmic design may not be the best path to fairness after all.The Conversation

Tom Douglas, Senior Research Fellow, University of Oxford

This article is republished from The Conversation under a Creative Commons license. Read the original article.

Share on

5 Comment on this post

  1. It seems to me you assume you know what substantive fairness looks like. I think I disagree with that. People could choose different career paths (for instance) for a great many reasons to do with personal taste, personal fulfilment and so on. It isn’t at all obvious that we know what the right outcomes look like, so I don’t think we know where the algorithms should aim for.

    My mum was one of NZ’s first women commercial pilots. About 50 years ago she was told by a senior guy at Air New Zealand that she couldn’t get a job flying jets because the customers wouldn’t get on a flight with a woman pilot. She faced a lot of barriers which are hard to imagine today. Yet she remained hostile to affirmative action on meritocratic grounds: in the long run, choosing the best pilot, irrespective of demographics, should yield fewer accidents and should lead to better outcomes. I find that pretty persuasive reasoning, and in that case utilitarian arguments point in the same direction as procedural fairness. Given that my mother’s career was pretty formative for me, I guess I don’t share your intuition that “substantive fairness often matters more.”

    Also, I would be interested in your take on what happens to trust/social capital/social cohesion where referees are allowed to (or obliged to) referee one team more harshly than another. My take is that it creates bad blood and disharmony, and undermines trust in the foundational institutions of the sport in question. (e.g. Likewise society, in my view – if you gerrymander the process to obtain pre-determined outcomes you may win the short game, but you’re unlikely to bring people with you, and in the long run democracies are about carrots and not sticks. (…I’d be interested if you would include the profound lack of political diversity in universities as a substantive justice issue.)

  2. great issues altogether, you just received a logo
    new reader. What could you recommend about your post that
    you made a few days ago? Any positive?카지노사이트

  3. An impressive share! I’ve just forwarded this onto a coworker who had been conducting a little homework on this.
    And he actually ordered me breakfast due to the fact that I discovered it for him…
    lol. So allow me to reword this…. Thanks for the meal!!
    But yeah, thanks for spending time to talk about this topic here on your web site.카지노사이트

Comments are closed.