YSM Issue 93.2

FEATURE
Computer Science
AN ALGORITHMIC
JURY
PREDICTING
RECIDIVISM
RATES WITH
ARTIFICIAL
INTELLIGENCE
BY MIRILLA ZHU
Over the last two decades,
predictive risk assessment tools
have been used to determine
the fates of millions of individuals in the
criminal justice system, deciding whether
a defendant will be detained or released
based on an algorithmic calculation of
risk. This technology has been embraced
by courts and policymakers alike, with one
Congressional bill going as far as to call
for the implementation of risk assessment
systems in every federal prison. But in
2018, researchers Julia Dressel and Hany
Farid published a surprising result: a
commonly used risk assessment tool
named COMPAS was incorrect almost
half the time. With the accuracy rate of
COMPAS only a few percentage points
higher than that of humans with no
judicial experience, some judges were left
wondering whether they would be better
off not using algorithms at all.
When Stanford graduate student
Zhiyuan Lin heard about Dressel and
Farid’s study, he was equally surprised at its
findings—although for a different reason
than the public. As a computer scientist
in Stanford’s Computational Policy Lab,
Lin had encountered dozens of studies
demonstrating that algorithms performed
better than humans, and he was puzzled
why Dressel and Farid had found otherwise.
Together with a team of researchers from
Stanford and Berkeley, Lin decided to see
whether he could fill in the missing pieces
to understand what was going on.
Lin and his colleagues began by
attempting to replicate the 2018 study,
giving over six hundred participants
the same set of profiles that Dressel and
Farid used and asking them to predict
whether the defendants would recidivate.
When they provided participants with
immediate feedback after each response,
they found that the participants guessed
correctly sixty-four percent of the time,
compared to the sixty-two percent
accuracy rate reported in the 2018 study.
The COMPAS algorithm’s accuracy rate
of sixty-five percent matched the 2018
study exactly.
Next, the researchers investigated
whether these results would hold if they
modified the experiment to resemble the
real world more closely. They did so in
three ways: providing the respondents
with more detailed criminal profiles,
lowering the average recidivism rates
to reflect the rate of violent crime,
and most significantly, not telling the
respondents whether they were right
or wrong. “Receiving this kind of
immediate feedback is something that
rarely happens in reality, because when
the judges are making bail decisions,
they don’t find out whether a defendant
22 Yale Scientific Magazine September 2020 www.yalescientific.org