The General Data Protection Regulation and personal data
Detecting and mitigating bias and discriminatory impact of AI hiring tools often requires processing personal data. In the EU, every instance of collecting, storing or using personal data has to comply with the General Data Protection Regulation (GDPR). This regulation sets the requirements for processing personal data, and it applies to all organisations operating within the EU or handling the data of EU citizens.
Personal data are defined broadly in the GDPR, as “any information relating to an identified or identifiable natural person (‘data subject’); an identifiable natural person is one who can be identified, directly or indirectly, in particular by reference to an identifier such as a name, an identification number, location data, an online identifier or to one or more factors specific to the physical, physiological, genetic, mental, economic, cultural or social identity of that natural person”. The GDPR does not apply to truly anonymous information, which refers to data that has been anonymised in such a way that the individual can no longer be identified. However, given the broad definition of personal data, simply removing names or addresses from documents such as CVs is rarely sufficient to achieve full anonymisation.
In addition, the GDPR categorises data on racial or ethnic origin, religion, philosophical beliefs, health, or sexual orientation as special categories of data. (commonly referred to as sensitive data). The processing of sensitive data is generally prohibited (Art. 9(1) GDPR), with strict conditions for exceptions (Art. 9(2) GDPR), see further below.
The GDPR requires a legal basis for processing any personal data (Art. 6 GDPR). There are only six possible legal bases. The two most relevant in the recruitment context are ‘freely given consent’ by a job applicant and ‘legitimate interests’.
Every time you need to process personal data, you have to clearly define the purpose of that use within the recruitment process and comply with relevant GDPR rules, including core principles of lawfulness, fairness, transparency, purpose limitation, data minimisation, accuracy, storage limitation, security and accountability (Art. 5 GDPR). When data processing is likely to result in a high risk to the rights and freedoms of individuals (Art. 35 GDPR), especially when using new technologies, a Data Protection Impact Assessment (DPIA) is required. We recommend that AI developers conduct a DPIA. When conducting a DPIA, the organisation should consult the data protection officer (Art. 35(2) GDPR).
In the recruitment context, processing of personal data will most likely fall under one of these purposes:
(i) Developing, training and testing an AI model, which often involves using historical candidate data:
→ If you want to develop an AI model using personal data, have a look at the recent opinion of the European Data Protection Board (EDPB)1 which analyses in detail the use of personal data for training AI. The opinion covers three main points: when AI models can truly be seen as anonymous, whether companies can rely on ‘legitimate interest’ to use non-sensitive personal data for training AI systems, and what the consequences are if an AI model is built using personal data that was collected or used unlawfully.
(ii) Generating candidate rankings, focusing on selecting candidates who best match the job requirements:
→ To generate candidate rankings, the recruitment system might use different types of a candidate’s data, including structured data (extracted from the candidate’s recruitment materials, e.g. a CV), unstructured data (free text of the CV or other documents such as a covering letter), and behavioural data (e.g. how the candidate interacted with the recruitment platform, e.g. clicks on job openings).
(iii) Monitoring the AI system’s outcomes for potential discrimination in post-deployment that may involve access to sensitive personal data:
→ To ensure that AI hiring tools do not produce discriminatory results, deploying organisations should monitor how these systems perform once they are in use. To evaluate whether job candidates were treated unfairly based on characteristics protected under EU non-discrimination law, including age, gender, ethnicity, religion, health, or sexual orientation, it may be necessary to analyse personal data of job candidates, including age, gender, ethnicity, religion, health status, or sexual orientation. This creates a dilemma. → see chapter #7
Recommendations:
Every act of processing personal data needs to comply with the GDPR. When processing personal data, developers and designers of recruitment systems should ensure the following:
Transparent Communication with Candidates
- Clearly and visibly explain what data is processed and why.
- Use plain, accessible language; avoid burying information in long texts or fine print.
- Explicitly inform candidates that an automated or AI-based system is being used, even if it involves third-party tools.2
- Notify candidates when there are updates or changes to the system or how their data is processed.
Disclosure of Evaluation Criteria
- Platforms should require employers to disclose the criteria used for candidate evaluation.3
- Differentiate between criteria used by automated ranking systems and those applied by human recruiters. Candidates should be aware of both.
Mitigation of Bias and Discrimination
- Delay the exposure of sensitive data or proxy information (e.g. name, age, photo) until later stages of the process. Avoid displaying such data in rankings, previews or profiles.4
- Use text encoders and behavioral data (e.g. clicks or interaction patterns) with caution. They may reinforce societal or position biases. Bias should be detected and corrected proactively.
- Do not use sensitive attributes in decision-making. Be aware that other data (e.g. years of experience) may unintentionally act as proxies for protected characteristics.
Data Minimisation and Security
- Collect only data necessary for the hiring process (e.g. qualifications, work experience, education). Avoid irrelevant personal details (e.g. marital status).
- Document what data is collected, why, and how it is pre-processed before being used by the system.
- Discard data that is no longer relevant after system updates, and inform candidates accordingly.
- Allow candidates to access, correct and delete their personal data at any time. Prompt them periodically to review and update their profile.
- Ensure that personal data is stored securely and that only relevant, anonymised data is shared with employers or third parties. Inform candidates about the data you share with others.5
- Opinion 28/2024, European Data Protection Board (EDPB)
- Paksy et al. (2023).
- Lin et al. (2023).
- Paksy et al. (2023); Lin et al. (2023).
- César et al. (2023).
