It is no secret that many businesses are increasing their use of algorithmic decision-making powered by AI and machine learning. These developments are not only impacting those at the cutting edge of tech but are also permeating day to day business from customer chat bots and customer relationship management to AI powered CV screening tools. So, of course, regulators and lawmakers are increasing their focus on how businesses are managing the new risks that this brings. This in turn means that lawyers and boards will find themselves considering how to manage these emerging risks in the way that regulators are expecting. Many lawyers, and boards, even those that specialise in data, are not experts in data science. We have put together a list of 5 things that lawyers and boards should know about mitigating algorithmic bias and what to do about them below.

This is an area which is changing rapidly, just last week in the UK, the Digital Regulation Cooperation Forum (which is formed of the key digital regulators including the ICO, FCA and CMA) announced a call for views on algorithmic processing (which is one of their four priority areas) to inform their regulatory approach. Anyone interested can contribute their views until 8 June 2022 (see more here). In the US, the Federal Trade Commission, has indicated that algorithmic bias is a priority area for enforcement investigations. Whilst many of the issues are not new, the scale of the challenge is increasing as the capacity of technology to use data has increased exponentially.

Use of algorithmic decision-making can occur in any sector from logistics to recruitment and financial services, and the risks will impact businesses even if they are buying in the technology and not developing it themselves. So even if you are not working in a tech focused sector it is likely you will encounter these issues at some point over the coming years.

1. Mitigation of bias cannot be treated as a purely technical issue 

You can be forgiven for thinking that identification and mitigation of bias is mainly an issue for data scientists, and ethicists. With many discussions focusing on statistical and technical solutions or abstract concepts of equality it is not always clear what a lawyer’s, or board’s, role is.    

However, emerging across all sectors and jurisdictions recommendations of how to manage risk of algorithmic bias include key areas such as governance, decision making, transparency, explainabilty and accountability which are areas which will be familiar to many lawyers and boards.

2. Algorithms are trained on statistical fairness but there are limitations as to what statistical fairness can achieve  

Understanding the way algorithms work is important. An algorithm is simply a set of instructions to solve a problem or complete a task. Usually, therefore, if we want the algorithm to contain a definition of fairness, we need to tell the model what that definition is. 

The problem arises when you consider that there is no single definition of fairness and numerous approaches to defining it. Furthermore, different constructs of fairness can lead to different results eg procedural fairness or outcome based fairness. So, businesses must first consider what concept of fairness to work with, and then consider how that can be translated into a concept a computer can understand. This is where it is important to understand the limitations of statistical fairness as not all concepts are capable of translation into mathematical formula. 

3. There is no “one size fits all” tool to address bias, context matters 

The tools that will be most effective will depend on an understanding of the contextual background. This includes considerations of sector, organisation, individuals, purposes etc. There is a clear role here for lawyers to work with their tech and data teams to inform the contextual background so the most appropriate tools are selected.

It is also important to understand that equality laws are not the same around the world so a tool that is developed in one jurisdiction may not meet the required standards in another.

Choosing between different tools will require trade-offs. For example, one solution may lead to more accurate results but could mean a greater intrusion of privacy. Where trade-offs are driven by technical drivers, they will not always be obvious to lawyers and boards. 

4. Familiarise yourself with some of the common techniques used by data scientists to address bias

Knowing that there are different techniques used to mitigate bias is unlikely to be enough. Lawyers and boards being familiar with some of the common techniques and how they can vary in effectiveness and in trade-offs can facilitate robust governance.

Some examples are:

  • Fairness through unawareness excludes protected characteristics. However, data points that remain can still strongly correlate with protected characteristics (eg postcodes as proxy for race) so the bias reduction can be limited depending on the data set.

  • Reject option classification takes points which the model isn't sure about and then assigns a favourable outcome to the disadvantaged protected class and a negative outcome to the advantaged class. This intervention at the margins is intended to balance overall outcomes. Whilst this intervention is viewed as relatively straightforward it does sacrifice accuracy more than some other methods.

  • Outcome parity this assesses if the model gives equal numbers of positive or negative outcomes to different groups.

5. Bias mitigation is not a one-time issue it must be considered at all stages of the life-cycle

Bias can enter a system at various points including in training data and model design through to how the output is acted on. Issues can be compounded where there is a lack of transparency in the model, uncertainty over the quality of training data, complexities in the supply chain and automation.

Take for example a loan rate repayment model with training data that contained more data on men than women. This underrepresentation could cause a model to predict lower loan repayment rates for women, even if they are on average more likely to repay their loans than men. Knowing whether there are under or overrepresented subsets of the population allows implementation of techniques to mitigate the issue.

What steps could lawyers and boards be taking

  • Review approaches to governance ensuring it includes appropriate review of potential bias in algorithmic decision making throughout the lifecycle and documentation of the decision making process including in relation to tools/technique selection and trade-offs.

  • Ensure those involved in key decision making have appropriate levels of training and awareness. This may involve training lawyers and members of the board on the techniques used by data scientists for bias mitigation.

  • Create multi-disciplinary teams at all levels involving tech, data, business and legal experts.

  • Identify if any protected characteristics are being used, and why. If not, identify what steps have been taken to identify and mitigate against potential proxies.

  • Ask questions to understand the quality of the training data (eg accuracy, potential for historic bias, relevant, recent, generalisable and appropriate) and to ensure it is representative of the population the system will be applied to.

  • Seek to understand the model design. This should include seeking to understand what steps have been taken to ensure design fairness.  This may include variance tolerances as well as when outputs will be escalated and when issues will be investigated.

  • Ask questions about the output, how fair outcomes are tested, and how users are trained to implement decisions responsibly.