Before You Trust an AI Interview Score: Five Checks for UK Agencies

By Nhi (Yiyi) Vu, Senior Operations Specialist, Flowmingo

Key takeaways

• Make sure every AI interview score can be traced to the candidate's own answer and a written criterion before it sorts a shortlist.
• Treat dropped audio and unfinished interviews as technical problems to review, not as low scores.
• If nobody reads the answers before a rejection goes out, treat the stage as automated and give candidates the safeguards that come with it.

On 31 March, the ICO published Recruitment rewired, its findings from conversations with more than 30 employers about automation in hiring. Many of them believed their tools were only supporting decisions. The regulator's view was that many were likely making solely automated decisions, without the safeguards the law expects.

The timing matters. Since 5 February, the Data (Use and Access) Act 2025 has written a test into UK GDPR: a decision is solely automated when there is no meaningful human involvement in it. The reform swapped the old general restriction on such decisions for a duty to provide safeguards, with tighter limits where health or other special category data is used. The ICO's evidence predates the change, but the test it applied is now the law.

For an agency using AI first-round interviews, the place to look is the ranked list of scores a consultant opens on a Monday morning, and what happens to the people at the bottom of it.

I work at Flowmingo, which makes an AI interview tool, so read this with that in mind. The figures below come from interviews on our platform across all markets, not just the UK. None of the checks depend on which tool you use.

Key figures

• 73.5% of AI interviews begun on a phone were finished, against 89.2% on a desktop or laptop (24 June–20 September 2026).
• 1 in 5 candidates in a typical role scored within half a point above that role's median score (6 July–30 September 2026).
• 1.6% of candidates who finished an AI interview received a written comment from the hiring team in the platform (6 July–20 September 2026).

Flowmingo platform data, all markets. Flowmingo conducted the analysis; it has not been independently audited. The figures show associations, not causes.

1. Make sure every score can be traced to an answer

A score on its own cannot be explained to a client, or to a candidate who asks why they were not taken forward. Candidates can ask: the ICO's draft guidance on shortlisting and interviewing confirms that interview notes fall within the right of access. Where a significant decision is solely automated, Article 22C of UK GDPR adds safeguards: candidates must be told, and be able to make representations, ask for a person and contest it.

Before relying on any tool, pick three candidates from one role: the top scorer, one from the middle and the lowest. Read or watch their answers, mark them yourself against the client's criteria, and only then open the report. A traceable report shows the question, the part of the answer it relied on and a reason you can check against the recording.

If the reason says "lacked confidence" or "answer felt rehearsed", stop there. Those are impressions of the interview, not evidence about the job. It is also worth knowing that AI scoring is not equally reliable on every question. In a 2023 study, an AI rater closely matched trained human raters on question types it had been trained on, and much less closely on unfamiliar ones.

2. Check what the tool is actually marking

Each criterion should come from the client's brief. Generic criteria such as "communication" or "presence" tend to reward candidates who are comfortable on camera, which is rarely what a client is paying you to find in, say, a credit controller.

This has legal weight for agencies, which are covered as employment service-providers under sections 55 and 56 of the Equality Act 2010. A criterion that marks down an accent or second-language grammar can put candidates of particular races or national origins at a disadvantage. A fluency score can do the same to a candidate who stammers.

Many tools score a transcript rather than the recording, and speech recognition is not equally accurate for every voice. A 2020 study of five commercial systems found average word error rates of 35% for Black US speakers, against 19% for white US speakers. When a transcript reads badly, listen to the recording before drawing any conclusion about the candidate.

3. Separate technical failures from performance

Candidates complete AI interviews in their own time and on their own equipment, and that is where scores pick up noise. Of 49,323 AI interviews begun on our platform between 24 June and 20 September 2026, 89.2% of those begun on a desktop or laptop were finished, against 73.5% on a phone. Device choice tracks other differences between candidates, so this is an association rather than a cause, but the gap is large.

Ask the tool to flag dropped audio, camera failures and unfinished interviews separately from scores, and have someone look at each one before anyone is ranked. An answer with no usable audio should be marked "not assessed", never zero, and the candidate offered a retake.

An answer with no usable audio should be marked "not assessed", never zero.

Nhi (Yiyi) Vu, Senior Operations Specialist, Flowmingo

Section 55 also requires agencies to make reasonable adjustments for disabled candidates, and the government's Responsible AI in Recruitment guidance recommends planning them before a tool goes live. One line in the interview invitation does much of the work: tell candidates how to ask for extra time, a different format or a retake.

4. Make the human review real

The ICO's worked examples are specific. An automatic rejection below a pass mark is solely automated. So is ranking candidates, inviting the top ones to interview and rejecting the rest without anyone considering every application. Its findings say the same of a manager who focuses on "green" candidates and only glances at "red" ones before rejecting them.

Cut-offs deserve particular care, because scores bunch together. On our platform, the median AI interview score from July to September was 6.4 out of 10, and in a typical role about one in five candidates scored within half a point above that role's median. Move a cut-off by half a point and you can move a fifth of the list.

Reviews also tend to leave little trace. Hiring teams added a written comment in the platform for 1.6% of candidates who finished an AI interview. Many will keep notes elsewhere, but if you ever need to show that a review was meaningful, a written reason is what you will reach for.

That leaves two honest routes for each stage. Either a consultant reads the answers of every candidate they reject and records a reason, or you accept that the stage is automated, tell candidates so in advance and give them a way to make representations, obtain human intervention and contest the outcome. That second route is much narrower where the decision draws on health data, for example from an adjustment request. This is general information on UK law, not legal advice.

5. Test it against your own consultants first

Before a tool sorts anything on its own, run it alongside your team on one live role. Consultants shortlist as normal, the tool scores the same candidates, and you compare the two lists. The candidates who appear on only one list are the useful part: read the AI's reasons for each, and look for patterns, such as second-language speakers being marked down.

Disagreement is not automatically the tool's mistake. In our data, the candidate with the top interview score also had the top CV score in only 11.2% of 349 hiring projects. Both are AI scores, so treat that as a sign that the two methods rank people differently, not as proof of which is right.

Before the trial, get written answers to these:

• What bias testing have you done, and can we see it? The ICO expects employers to ask developers about their bias testing and to run trials of their own.
• Can every score be traced to an answer and a criterion? If not, you cannot explain it.
• How are technical failures handled? They should never quietly lower a score.
• How long are recordings and transcripts kept? Ideally, you set the period.
• Is candidate data used to train your models? It affects what you need to tell candidates.
• Who is the controller and who is the processor? The ICO's 2024 audit of AI recruitment tools found several providers that had wrongly described themselves as processors.

The wider point

AI interviews let an agency give every applicant a first conversation that a busy desk never could. The score is a faster way to read those conversations, not a reason to stop reading them. Agencies that can explain each score to a client and to a candidate will find the ICO's direction of travel much easier to live with.

About the author