DIKU Bits: Measuring, Understanding and Modelling Human Label Variation in Legal NLP

DIKU Bits Graphic with picture of Shanshan Xu

Speaker

Shanshan Xu, PhD Fellow in the NLP (Natural Language Processing) section.

Title

Measuring, Understanding and Modelling Human Label Variation in Legal NLP

Abstract

Human Label Variation (HLV) refers to the differences in human annotations, which reflects the plurality of human opinions. In this talk, Shanshan will present our investigations into naturally occurring HLV in legal NLP, focusing on split votes among judges in court decisions, and respondents’ different answer to political survey. These cases reveal the challenges of modeling pluralistic human values in legal NLP and emphasize the importance of responsible data curation and robust evaluation for pluralistic Human-AI alignment.