A shortlist can look efficient on a dashboard and still create a serious fairness risk. Adverse impact testing helps employers examine whether a selection process disproportionately screens out people from a particular group, even where every candidate was assessed using the same rule. For Australian employers, it is a practical way to test whether recruitment decisions are consistent with the organisation’s commitment to fair, evidence-based hiring.
The purpose is not to assume discrimination whenever groups achieve different results. Differences can arise for several reasons, including small candidate numbers, role requirements, labour-market patterns or the design of the assessment itself. The point is to identify patterns early, investigate them properly and make defensible improvements where required.
What is adverse impact testing?
Adverse impact testing compares selection outcomes across different demographic groups. It asks a straightforward question: is one group progressing through a hiring stage at a materially lower rate than another group?
Selection stages can include application screening, cognitive ability testing, skills assessments, video interviews, assessment centres and final shortlisting. Testing can also be applied to internal promotion, graduate programs, leadership selection and talent development decisions.
A common starting measure is the selection rate. If 60 out of 100 applicants in one group progress, their selection rate is 60 per cent. If 24 out of 100 applicants in another group progress, their selection rate is 24 per cent. Comparing those rates reveals whether the process is producing substantially different outcomes.
Many organisations use the four-fifths rule as an initial screening indicator. Under this approach, the selection rate for one group is compared with the rate of the group with the highest selection rate. A ratio below 80 per cent may warrant closer review. It is useful because it is simple, but it is not a legal safe harbour and should not be treated as a final finding of bias.
The result needs context. A ratio based on 12 applicants is far less reliable than one based on 1,200. Statistical significance, practical impact, the role’s genuine requirements and the quality of the selection measure all matter.
Why adverse impact testing matters in Australian recruitment
Hiring teams are under pressure to move quickly, particularly in high-volume campaigns. Automated ranking, online testing and asynchronous interviews can improve speed and consistency, but they also make it easier to scale a poorly designed decision rule. If a cut-off score is inappropriate, the issue is repeated across every candidate rather than corrected through individual judgement.
Regular testing gives HR and talent acquisition leaders evidence to challenge assumptions. A process may feel objective because it applies the same assessment to everyone. Objectivity, however, depends on more than equal administration. The assessment must be relevant to the role, interpreted appropriately and used alongside a fair process.
This is also a governance issue. Employers need to be able to explain why they assess particular capabilities, how scores influence decisions and whether the process is working as intended. Clear evidence is particularly valuable in government, healthcare, education and other sectors where recruitment decisions may face close scrutiny.
How to run adverse impact testing effectively
Start with a defined hiring decision
Testing is most useful when the decision point is clear. Define exactly what counts as success at each stage: meeting an application screen, reaching a benchmark score, being invited to interview, making the final shortlist or receiving an offer.
Avoid combining unrelated stages into one measure. If a demographic difference appears at offer stage, it may have originated in the initial application filter, the assessment threshold or interviewer decisions. Stage-by-stage analysis helps locate the source rather than treating the whole recruitment process as a black box.
Collect and manage data carefully
You need accurate data on candidate progression and, where appropriate and lawfully collected, demographic information. Participation in demographic data collection should be handled transparently and respectfully. Data should be separated from the hiring decision, access should be restricted, and reporting should focus on aggregated patterns rather than identifiable individuals.
The demographic categories to review will depend on your workforce, applicant pool and legal obligations. Gender, age, disability, cultural background, Aboriginal and Torres Strait Islander status and other relevant characteristics may be considered. Intersectional analysis can add useful insight, although small group sizes can limit what can be reported reliably.
Where sample sizes are low, do not force a conclusion from unstable data. Pooling data across comparable campaigns or reviewing results over time may provide a clearer picture. Equally, do not use small samples as a reason to ignore a recurring concern.
Compare rates, then investigate the pattern
Calculate progression rates for each group at each selection stage, then compare the results. Look for recurring gaps, not just one unusual campaign. Trends across locations, hiring managers, job families and assessment formats can point to whether the issue sits in the tool, its implementation or a particular operational practice.
Numbers identify where to look. They do not tell you why the difference exists. A lower progression rate may be connected to an assessment that is poorly aligned to the role, an unnecessarily high cut-off score, inaccessible instructions, inconsistent interviewer scoring or a sourcing strategy that produces unequal candidate preparation.
For example, a cognitive assessment may be relevant for a role involving complex problem solving. But setting the benchmark at the score of a current high performer without validating the required level of ability can exclude capable candidates without improving job performance. The answer is not automatically to remove the assessment. It is to validate the requirement and set a threshold that reflects the role.
Review the job relevance of every selection measure
The strongest defence against unfair outcomes is job relevance. Each assessment should measure a capability that matters for performance, be administered consistently and contribute to a decision model that makes sense for the role.
This means starting with a clear job analysis. What knowledge, skills, abilities and behavioural requirements predict success? Which are essential at entry, and which can reasonably be learned on the job? The answers should guide assessment selection, weighting and minimum standards.
Validated psychometric and skills assessments can provide a more consistent basis for comparison than unstructured interviews alone. They are not a substitute for judgement, though. A sound process combines relevant assessment data with structured interviews, clear scoring criteria and trained decision-makers.
Test changes before rolling them out widely
When results suggest potential adverse impact, resist the urge to make a cosmetic change. Removing a question or lowering a threshold without examining job requirements may weaken quality of hire while doing little to improve fairness.
Instead, test practical alternatives. This may include revising a benchmark, improving candidate instructions, providing a reasonable adjustment pathway, replacing an unstructured interview with a structured format, or using multiple relevant measures rather than relying heavily on one score. Monitor the outcomes after each change.
For organisations using assessment technology at scale, this work should be part of ongoing quality assurance, not a one-off compliance exercise. RightPeople supports this approach by combining scientifically validated assessments with psychologist input that helps employers interpret results in the context of the role and hiring decision.
Common mistakes that reduce the value of testing
The first mistake is treating the four-fifths rule as the entire analysis. It is a useful flag, not a complete answer. A process can meet that threshold and still deserve review, particularly where the consequence of exclusion is substantial or a pattern is repeated.
The second is focusing only on the final hiring outcome. By then, the candidate pool may already have been narrowed significantly. Reviewing each stage gives employers a better chance to correct issues before they affect offers.
The third is assuming technology is neutral by default. Algorithms, automated rankings and digital assessments reflect their design, the data used to develop them and the rules set by the employer. Technology can strengthen consistency, but it still requires validation, monitoring and human accountability.
Finally, avoid separating fairness from commercial performance. A process that rejects capable candidates without a valid job-related reason is not only a risk issue. It reduces access to talent, narrows the shortlist and can undermine the quality of the eventual hire.
Build a process that can stand up to scrutiny
Adverse impact testing works best when it sits alongside clear assessment governance. Document the job requirements, the purpose of each measure, cut-off decisions, reasonable adjustment processes, assessor training and review dates. Keep decision records that explain how evidence was weighed, particularly for senior, specialist or regulated roles.
The right response to a concerning result will depend on the evidence. Sometimes a deeper review confirms that the measure is relevant and the difference is not statistically meaningful. Sometimes it reveals a threshold, interview practice or access barrier that should change. What matters is having the discipline to test the process rather than relying on intuition.
Fair hiring is not achieved by removing standards. It is achieved by setting standards that are genuinely connected to the work, applying them consistently and checking that they lead to better decisions for both the organisation and its candidates.
