TY - JOUR

T1 - Auditing Risk Prediction of Long-Term Unemployment

AU - Seidelin, Cathrine

AU - Moreau, Therese

AU - Shklovski, Irina

AU - Holten Møller, Naja

N1 - Funding Information: We thank our collaborators from the Danish Agency for Labour Market and Recruitment, especially Carsten Søren Nielsen, and Zetland’s Frederik Kulager besides Peter Maarbjerg Dønvang – as well as colleagues Asbjørn Ammitzbøll Flügge, Trine Rask Nielsen, and Thomas T. Hildebrandt for providing feedback. This research has been supported by the Innovation Fund Denmark (EcoKnow: award number 7050-00034A) and the Independent Research Fund Denmark (PACTA: award number 8091-00025b). Funding Information: This work is supported by the Innovation Fund Denmark, under grant 7050-00034A and the Independent Research Fund Denmark, under grant 8091-00025b. Author’s addresses: C. Seidelin, I., Shklovski and N. Holten Møller, Department of Computer Science, University of Copenhagen, Sigurdsgade 41, 2200 Copenhagen, Denmark; T. Moreau, Zetland, Njalsgade 19D, 1st floor, 2300 Copenhagen, Denmark. Publisher Copyright: © 2022 Owner/Author.

PY - 2022

Y1 - 2022

N2 - As more and more governments adopt algorithms to support bureaucratic decision-making processes, it becomes urgent to address issues of responsible use and accountability. We examine a contested public service algorithm used in Danish job placement for assessing an individual's risk of long-term unemployment. The study takes inspiration from cooperative audits and was carried out in dialogue with the Danish unemployment services agency. Our audit investigated the practical implementation of algorithms. We find (1) a divergence between the formal documentation and the model tuning code, (2) that the algorithmic model relies on subjectivity, namely the variable which focus on the individual's self-assessment of how long it will take before they get a job, (3) that the algorithm uses the variable "origin"to determine its predictions, and (4) that the documentation neglects to consider the implications of using variables indicating personal characteristics when predicting employment outcomes. We discuss the benefits and limitations of cooperative audits in a public sector context. We specifically focus on the importance of collaboration across different public actors when investigating the use of algorithms in the algorithmic society.

AB - As more and more governments adopt algorithms to support bureaucratic decision-making processes, it becomes urgent to address issues of responsible use and accountability. We examine a contested public service algorithm used in Danish job placement for assessing an individual's risk of long-term unemployment. The study takes inspiration from cooperative audits and was carried out in dialogue with the Danish unemployment services agency. Our audit investigated the practical implementation of algorithms. We find (1) a divergence between the formal documentation and the model tuning code, (2) that the algorithmic model relies on subjectivity, namely the variable which focus on the individual's self-assessment of how long it will take before they get a job, (3) that the algorithm uses the variable "origin"to determine its predictions, and (4) that the documentation neglects to consider the implications of using variables indicating personal characteristics when predicting employment outcomes. We discuss the benefits and limitations of cooperative audits in a public sector context. We specifically focus on the importance of collaboration across different public actors when investigating the use of algorithms in the algorithmic society.

KW - accountability

KW - algorithm

KW - audit

KW - job placement

KW - public services

U2 - 10.1145/3492827

DO - 10.1145/3492827

M3 - Journal article

AN - SCOPUS:85123316786

VL - 6

SP - 112

JO - Proceedings of the ACM on Human-Computer Interaction

JF - Proceedings of the ACM on Human-Computer Interaction

SN - 2573-0142

IS - GROUP

M1 - 8

ER -