Fairness-enhancing interventions in the ranking pipeline can reduce ranking bias

Fairness interventions can be used within a ranking system’s pipeline to prevent the system from generating unfair predictions for groups defined by some attributes such as gender, age, or nationality. These can be categorised as pre-processing (changes to training data), in-processing (changes to the way in which the ranking module works) and post-processing (changes to the output of the ranking module).

Pipeline of a ranking system and fairness interventions.

Having limited access to the sensitive information of candidates makes existing fairness interventions hard to use in practice:¹

Post-processing approaches require sensitive information in order to be able to re-rank the candidates to satisfy a minimum representation of the disadvantaged group at the top of the ranking.
In-processing approaches require constant access to sensitive information in an online environment, where the ranking model continuously learns from the interactions between the platform and the recruiters/candidates.
Pre-processing approaches need access to sensitive information during training time but not during inference time.

Taking into account that a system needs to limit access to sensitive information, pre-processing fairness interventions are more suitable to be used in practice when trying to ensure the fair ranking of candidates within a recruitment system. Pre-processing methods can be trained offline on a training set containing the ‘real’ sensitive information of job candidates, which was acquired in compliance with the GDPR and the AI Act, or on a synthetic dataset.

→ FINDHR semi-synthetic data set, (developed on the basis of a public data donation campaign) is available for researchers in established EU academic institutions under a license agreement.

During inference time, the pre-processing method can be applied without access to sensitive information of candidates. Notably, the pre-processing fairness methods aim to debias the data. However, they don’t guarantee that the data is fully debiased, as there can always be hidden biases that were not taken into account.

Fairness interventions focusing on intersectionality

Accounting for intersectional groups poses some additional challenges concerning fairness interventions:²

As the number of sensitive attributes increases, the number of intersectional sub-groups grows exponentially, making optimisation computationally expensive and potentially difficult to find an optimal solution for all groups.
Some intersectional groups may have very small data sets, which could undermine the reliability of fairness improvements and assessments for these groups.
Redundant or unnecessary comparisons between certain sub-groups can lead to inefficiencies without contributing to better fairness outcomes.

Existing fairness interventions considering intersectionality³ focus on achieving fairness for all possible intersectional sub-groups. This can be viewed as simplifying the complex dynamics of intersectionality, but it provides a starting point.

Out of the existing pre-processing fairness interventions used for ranking tasks, CIF-Rank⁴ is one of the few methods explicitly designed to handle intersectional groups. However, it requires access to sensitive information at inference time, limiting its applicability for recruitment contexts.

Within FINDHR we have developed a Fairness API, which provides an example of how to implement a full pipeline, composed of data pre-processing, fairness, explainability and monitoring, as well as the accompanying documentation. For easy integration, the following Python package https://github.com/findhr/findhrAPI supports the implementation of pre-processing fairness intervention methods (CIFRank, LFR, iFair), as well as extensions for supporting multiple non-binary groups, including intersectional groups.

→ for more information on mitigation strategies and on how support for intersectional groups enables optimisation approaches, consult the FINDHR Software Development Guide, section 2.8.2

This graphic shows how unfair ranking can be improved through fairness interventions.

Recommendations:

Guidelines with respect to prevention and mitigation of discriminatory behaviour

As a preventative measure, proactively reflect on system design, potential biases and mitigation strategies early in development.
Create a system card and a system map to document the end-to-end workflow, surface potential biases and ensure legal and ethical compliance.
As a mitigation step, prefer pre-processing fairness interventions. Considering the limited access to sensitive information, they are more suitable in practice.
When selecting a fairness method, evaluate its transparency, impact on candidate diversity, intra-group fairness and ability to account for intersectional bias.
Consider using CIF-Rank,⁶ one of the few methods explicitly designed to address intersectionality.
Keep in mind that pre-processing fairness interventions do not guarantee full elimination of biases – remain vigilant for hidden biases throughout the system’s life cycle.

Rus et al. (2024)
Wang, A., Ramaswamy, V. V., Russakovsky, O. (2022)
Gohar, U., Cheng, L. ( 2023)
Yang, K., Loftus, J. R., Stoyanovich, J. (2020): it aims to achieve group fairness by estimating the causal effect of the sensitive data (e.g. gender) on the non-sensitive data (e.g. features/scores of the candidates). Using this estimate it aims at correcting the bias in the data.
Yang, K., Loftus, J. R., Stoyanovich, J. (2020). see above.