Pre-employment tests: science or hocus pocus?

It used to be the case that when you applied for a job, you usually faced an application, interview, and reference checks. Today, such a screening process almost seems sparse next to the various tests employers often issue.

Talent Board found in a 2022 study that 56% of North American employers are using pre-employment assessments/selection tests. [1] Those tests may vet:

• Job specific skills
• Personality
• Cognitive ability
• Competency
• Cultural fit
• Job simulation performance
• Case studies performance
• Scenario judgements

Readers may have heard or felt concerns about tests of ‘softer’ traits like personality, perceiving them to be “pseudoscience”. This article investigates what the science has to say about these tests, particularly personality tests.

Studies shed a positive light. A major analysis by the University of Iowa looked at several decades of studies, finding positive correlations between various tests and job performance. [2]

A study by ODU (Old Dominion University, Norfolk, Virginia) on the retention of hundreds of staffers in a mix of industries found those who were hired following an application process that included tests kept their positions longer than those who were not. [3]

On to personality tests specifically, an analysis of 117 studies found conscientiousness correlates with performance for all jobs. [4] Personalities that are good for certain occupations (e.g., extraversion likely helps if you are in sales) were correlated with performance in such occupations. Other meta-analyses as well as the earlier Iowa study found similar results. [5,6,7]

So, can we conclude that pre-employment tests are the way forward?

Those studies on personality tests did find positive correlations consistently, but not strong ones. A strong correlation is thought to be 70% or above. These studies usually found around 20% at best (albeit norms in HR research set a low bar for correlations relative to other fields of study). Casting further doubt is an analysis of 33 studies suggesting people often lie on personality tests to improve their odds. [8]

Another issue is that a strong correlation does not mean a strong effect. Staff selected via personality tests may be consistently better at carrying out their jobs, but how much better? Is the improvement worth the resources it takes to run the tests? Would different tests produce better results?

The studies do not answer those questions. However, high-performing employees are generally retained longer than those who do not perform so well. As such, the ODU study [3] found that 87% of those hired via testing did not leave the company a year later, in contrast from 80% of the non-tested employees who did leave within a year. However, whether or not a seven percentage point differential is a strong enough indicator of the value of personality testing may depend on the firm. E.g., a high staff-turnover industry may see less value in this than firms who must invest more in their staff.

Of course, an employer needs some screening method – and an effective one since getting the wrong staff is so costly. A range of 2017-2023 studies link staff turnover to reduced product and service quality in various industries and countries. [9,10,11] In 2019 Culture Amp estimated that turnover costs as much as 200% of salary with similar estimates found by Work Institute and Oxford Economics. [12,13,14]

Research suggests the best results are from testing cognitive and practical skills, as well as multiple-faceted tests. The Iowa study found the strongest correlations for cognitive ability tests at 65%, and for multi-measure tests (cognitive ability + personality + interests) at 71%. [2] Other research finds similar results for ability tests, assessment centres, and simulating the work an employee would do (work sampling). [15] The earlier study on employee retention found the best results for multi-measure tests and mathematical tests, though not for other measures like reading, listening, and teamwork. [3]

This is not to say a firm can set up work sampling or an assessment centre and call it a day. Naturally, the selection process must be appropriately designed if it is to accurately select candidates.

Moreover, employers should also consider self-disqualifying candidates, as researchers fear that firms with poor application processes may miss good employees. [15] Strong candidates, less desperate to find work, may perceive a selection process as inappropriate and poorly designed. Seeing this as a reflection of the wider firm, they may not bother to apply. Worse still, they may tell others of their negative experience and hurt the company’s reputation. Self-disqualifying candidates is no trifling matter given how quickly word can spread via the internet and current skill shortages. [16] This was even reflected by surveyed employers who welcomed the improvements of testing but also feared accidentally filtering out good candidates. [3]

On a subjective note, there is a risk that firms, pressed for time and resources, may place too much reliance on testing and too little on the importance of assessing suitability based on the exercise of judgement informed by carefully structured personal interviews. A case in point is a financial services firm which primarily used maths tests to hire for a programming role. Indeed, being skilled at maths might be a good indicator, but is that enough? Could it miss someone who is bad at maths, yet great at programming and who would fit the company’s culture? Conversely could the firm come to suffer someone who is great at maths yet bad at programming? Is there much maths done day-to-day? Does a talent for maths reliably indicate a current or potential talent in programming, or in the soft skills like dealing with colleagues and coworkers, in managing projects, etc., also needed to fit seamlessly into the firm’s culture and to support its strategy?

To sum up, the research suggests that some tests have limited utility in selecting candidates, providing no silver bullets. The research literature identifies the following to help firms design a selection methodology that works for their business: [14,15,17]

• Identify what abilities are needed for the job: ensure that all necessary skills are assessed and avoid testing ‘nice to have’ but irrelevant skills
• Ensure the method relates to a legitimate business purpose: see that the skill being assessed demonstrably relates to a duty of the role. This doubles in importance for undoing structural discrimination against minorities*
• Design tests which can genuinely confirm or deny evidence of these abilities: ideally a test will examine the skill itself, rather than a related but different skill
• Design standards and metrics which score these abilities: allow for objective confirmation or denial of skill instead of subjective hunches
• Trial the assessment method and adjust as necessary: trial the method on current staffers, and verify if it reflects their performance, or would it have filtered out someone known to be a good employee
• Write clear, fair, and consistent instructions for candidates: the method should not compound as a test of one’s ability to read poor English
• Avoid processes that are intimidating and unnecessarily hurt performance: like the previous point, the method should not unintentionally double as a test of handling stress under the interviewers’ gaze. For example, consider if an assessment test must be done in the presence of an interviewer, or if it would still be valid if a candidate can complete it with time and privacy
• Select and train assessors who can run the assessment effectively: a method could be undermined by assessors who are poor at explaining, organising, are biased, etc.

The ideal selection process might prove hard to define – perhaps even more so for smaller firms lacking the time, resources, and cash to design, test, and run multi-dimensional assessment centres. It is likely to combine appropriate testing with (arguably more important) personal interviews and will need to be refined and adapted as the business and its requirements develop.

Crucially, having identified, selected, and appointed the right candidates, firms will need to retain them through appropriately designed remuneration structures that reward the desired behaviours and outcomes required to support the firm’s culture and business strategy.

Please do not hesitate to contact James Sharp for further information or discussion on this article and MM&K’s HR and remuneration advisory services.

*A prominent example for further reading being the US Supreme Court ruling in Griggs v. Duke Power Co.

Sources:

1. https://thetalentboard.wpenginepowered.com/wp-content/uploads/2023/01/2022-TB-NA-CandE-Candidate-Experience-Benchmark-Research-Report-FINAL.pdf
2. https://www.researchgate.net/publication/232564809_The_Validity_and_Utility_of_Selection_Methods_in_Personnel_Psychology and 2013 update to 1998 work https://greatpeoplemanagement.com/wp-content/uploads/2022/07/Problem-with-using-personality-tests-for-hiring.pdf
3. https://digitalcommons.odu.edu/cgi/viewcontent.cgi?article=1186&context=efl_etds
4. https://gwern.net/doc/psychology/personality/conscientiousness/1991-barrick.pdf
5. https://www.researchgate.net/publication/7453309_The_Great_Eight_Competencies_A_Criterion-Centric_Approach_to_Validation/link/02e7e52d7ffa01188f000000/download
6. https://www.researchgate.net/publication/14098123_The_Five_Factor_Model_of_Personality_an_Job_Performance_in_the_European_Community
7. https://www.researchgate.net/publication/12202552_Personality_and_Job_Performance_The_Big_Five_Revisited 14
8. https://www.academia.edu/4029762/A_Meta_Analytic_Investigation_of_Job_Applicant_Faking_on_Personality_Measures?auto=download
9. https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3568792
10. https://www.kenaninstitute.unc.edu/wp-content/uploads/2018/03/A-Structural-Estimation-Approach-to-Agent-Attrition-Emadi.pdf
11. https://www.hbs.edu/ris/Publication Files/16-039_a0651b04-18be-4a91-9bf9-a1d4cb12c540.pdf
12. https://www.cultureamp.com/blog/cost-of-employee-turnover
13. https://cdn2.hubspot.net/hubfs/478187/2017 Retention Report Campaign/Work Institute 2017 -Retention Report.pdf
14. https://www.oxfordeconomics.com/wp-content/uploads/2023/05/cost-brain-drain-report.pdf
15. https://pdfs.semanticscholar.org/e86e/ac142735e1d2e5a2929dcbc49478eb65db94.pdf
16. https://www.ons.gov.uk/employmentandlabourmarket/peopleinwork/employmentandemployeetypes/bulletins/jobsandvacanciesintheuk/july2023
17. https://journals.sagepub.com/doi/abs/10.1177/009102609202100202

Back to News