This repo contains the code and documentation for "Assessing Annotator Identity Bias via Item Response Theory: A Case Study in a Hate Speech Corpus", by P. S. Sachdeva, R. Barreto, C. von Vacano, and C. J. Kennedy.
This work builds off "Constructing interval variables via faceted Rasch measurement and multitask deep learning: a hate speech application" by Kennedy et al. (2020).
To install the code used in the paper, clone this repository to your local machine, and navigate to the project. Install the package using pip:
pip install -e .
You should then be able to access the hatespeech package, which will allow you to run the analyses and the notebooks for the figures.
The data used in this work is available on HuggingFace.