Experiences of navigating opaque hiring processes

“I applied for a product manager role at a tech company in Berlin. I submitted my application in the afternoon, and the next day I received a rejection email around midnight. This made me think an AI tool was likely used to screen applications, as no human resources team would assess candidates’ CVs at such a late hour.” – Ming, a jobseeker interviewed by FINDHR¹

To collect real-world experiences of algorithmic discrimination, FINDHR researchers conducted Participatory Action Research (PAR) sessions, actively involving members of underrepresented groups across seven countries.

Participants shared experiences of both human and algorithmic discrimination in recruitment, often based on gender, ethnicity and age, which are often visible characteristics that are hard to hide. Discrimination was experienced throughout the hiring pipeline, up to being hired and quickly fired, or offered a demoted position. Also, many highly qualified individuals shared stories of being told by human recruiters that they were either overqualified, did not fit the team, or, in the case of AI-based hiring processes, received no response at all despite meeting the criteria. A recurring frustration was the lack of adequate explanation and feedback when applying through an AI tool. Many reported not even receiving replies to applications.

“I sent out 100 résumés in 1 month but did not get any interview call-back. The only few interviews I managed to get were through referrals.” – Ju, a jobseeker interviewed by FINDHR²

When considering how jobseekers experience discrimination, we must remain aware of two key facts:

• Algorithmic systems are often deployed in secrecy, without informing people that an automated tool made or contributed to a decision affecting their lives, let alone informing them about important parameters, such as how the system works and reaches a decision. This makes algorithmic discrimination hard to detect and avoid for people who have been subjected to it.
• Some individuals completely lack access to online job portals to apply for jobs, not having the required resources (including Internet access and safe housing) or digital literacy and proficiency in reading and writing. For this vulnerable group, entry barriers to the job market are already high in offline application processes, and get even higher when algorithmic hiring mechanisms are used.

More transparency in online hiring processes, explanations on how systems work and reach their decisions and training for jobseekers on how to navigate and perform in online applications are urgently needed. To address the latter, FINDHR developed a free training manual on AI hiring tools as a starting point to support jobseekers in online hiring processes.

→ read and share our free Training Manual on AI Hiring Tools

Personal data and proxies – experiments with applicant tracking systems (ATS)

Algorithmic hiring tools can add additional barriers to the ones already present in the lived job-search-reality of underrepresented groups in the labour market. In our research, we found that these include:

• barriers in access to information;
• shortcomings in digital access, literacy and trust;
• challenges in curating one’s digital persona and feeling pressured to assimilate one’s profile to compete in the AI hiring context (e.g. changing names into western-sounding names for better ‘readability’ by algorithms or omitting vs. adding a photo to the résumé)³;
• stylistic biases and ‘cultural fit’.

“I had a picture in my CV, but I decided to make some tweaks and remove my picture after getting feedback from a friend’s friend that this will help remove the bias…that people won’t see that you are a woman applying for a developer position. I also removed the mention that I was from Colombia. And after that, it seems to really change, and I got more successful from hearing back to move on to the next round.” – Tala, a jobseeker interviewed by FINDHR⁴

An experiment⁵ presented in one of our FINDHR expert reports⁶ and conducted in the UK sought to understand how manual and automatic testing of applicants might vary. Numerous résumés that were graded as ‘A-Candidates’ by human recruiters were given lower scores than other applicants by an automated system, despite possessing relevant experience and closely aligning with the job description’s requirements.

The accuracy and efficiency of a résumé’s parsability, meaning, the system’s ability to effectively read and extract the pertinent keywords and information emerged as a key factor influencing the scoring outcomes.

A further experiment⁷ also revealed clear differences in the evaluation of CVs. It was found that another algorithm assigned a higher score to ‘native’ (in this case British) applications than to international CVs, in some cases giving British applicants an advantage over non-natives that was not based on qualification.

Names and particularly photos in CVs ­— which are not typically included in CVs in some countries, for example the UK, but are routinely found in CVs in other countries, for example Germany — can act as indications of gender and ethnicity. This has led some jobseekers to even change their legal names. A recent study by FINDHR researchers⁸ found that foreign names can negatively affect candidates’ chances of being selected. Recruiters often make assumptions about perceived relocation costs, unfamiliarity with the employer, the relevance of the candidate’s education and potential issues with the local language or local regulations.

Algorithms used for ranking candidates in some Applicant Tracking Systems (ATS) might consider demographic characteristics of candidates (e.g. nationality, age or gender) even if they are not explicitly mentioned in CVs. While these might be explicitly excluded or redacted from data, there are subtle clues in job applications, so-called proxies, that can provide indications of these demographic characteristics. These can include for example language skills, work experience or educational background. Such proxies make bias and discriminatory outcomes difficult to detect. Along these lines, the experiment mentioned above⁹ revealed that studying abroad resulted in lower AI-provided scores for 80% of applicants. Ultimately, this can lead to applicants who did not grow up and study in the country where the job is advertised being treated differently from local applicants. This is consistent with findings from our other FINDHR research¹⁰, which demonstrates that names of candidates’ affiliations can signal different degrees of ‘foreignness’, reinforcing existing biases and influencing selection decisions.

One of the key features of recruitment software are ranking algorithms used to present a list of candidates to the recruiters. Our research explored the position bias that sorted lists introduce. Given recruiters’ limited time and resources, in combination with the large number of CVs received, recruiters usually review CVs in a top-down manner (similarly to what we all tend to do when searching for information online), until a suitable candidate is found. Top-listed candidates are therefore more likely to be reviewed, and candidates that are not in the first page of the list are unlikely to be given the same level of attention. In addition, in the same study, algorithms without anti-discriminatory measures have been found to significantly advantage candidates with European-sounding names, while the anti-discriminatory ranking was found to reduce the imbalance, but without advantaging candidates with non-European sounding names. It is thus important to ensure that fair and anti-discriminatory ranking algorithms are used in ATS to provide visibility to candidates from underrepresented groups.

These and further experiments suggest that biases can arise based on both the data an algorithmic system was trained on and the data it interacts with in real time. They show that algorithmic discrimination is happening – and unfortunately often affects individuals and communities that are already discriminated against in the job market.