Kristian Lum will lead a discussion of Alicia Solow-Niederman, YooJung Choi, and Guy Van den Broeck‘s The Institutional Life of Algorithmic Assessment on Friday, April 12, at 10:15 a.m. at #werobot 2019.
Alicia Solow-Niederman
On August 28, 2018, California passed the California Money Bail Reform Act, also known as Senate Bill 10 (SB 10), and eliminated the state’s system of money bail, replacing it in part with a risk assessment algorithm. Though SB 10 has been temporarily stayed, pending resolution of a 2020 ballot referendum, we cannot stay the bigger picture questions about risk assessment tools in the criminal justice system. Building from a long-standing critique of actuarial assessments in criminal justice, a rapidly-growing legal and technical literature recognizes that risk assessment algorithms are not automatically unbiased. Research to date tends to focus on fairness, accountability, and transparency within the tools, urging technologists and policymakers to contend with the normative implications of these technical interventions. While questions such as whether these instruments are fair or biased are normatively essential, this Essay contends that looking at these issues in isolation risks missing a critical broader point. Automated risk assessment systems do not operate in a vacuum; rather, they are deployed within complex webs of new and preexisting policy and legal structures.
Guy Van den Broeck
This Essay’s detailed analysis of SB 10 concretely illustrates how algorithmic risk assessments statutes and regulations require tradeoffs and tensions between global and local authority. Specifically, using SB 10 as a not-so-theoretical hypothetical reveals a tension between, on one hand, a top-down, global understanding of fairness, accuracy, and lack of bias and, on the other, a tool that is well-tailored to local considerations. There is a general conceit in the law that a principle like fairness is universal. SB 10’s text and legislative history support this globally-applicable perspective, calling for “validated risk assessment tools” that are “demonstrated by scientific research to be accurate and reliable.”
Kristian Lum
Concepts like accuracy, reliability, and non-discrimination are fixed principles from such a legal and policy standpoint. But there is a tension between such global principles and the validation and deployment of a tool in particular jurisdictions (typically at the county level). Anytime there is both a more centralized body that sets overarching guidelines about the tool and a risk assessment algorithm that must be tailored to reflect local jurisdictional conditions, this algorithmic federalism will give rise to a global-local tension. This tension results in a number of technical challenges, including questions about the treatment of proxies, Simpson’s paradox, and thresholding choices. Accordingly, we call for increased attention to the design of algorithmic risk assessment statutes and regulations, and specifically to their allocation of decision-making responsibility and discretion. Only then can we begin to assess the impact of these risk assessment algorithms when it comes to core criminal justice decisions about life and liberty.
