The Interview Prep Problem Every Recruiter Ignores

The preparation problem is simple to state. Nobody agrees the criteria, nobody writes the questions, and nobody divides the panel, so four interviewers improvise four different conversations and the shortlist debate afterwards is about who was most articulate. The fix is about 20 minutes of work per role: rank 5 criteria, write 6 questions, assign them in the calendar invites. Almost everything blamed on interviewer skill is a preparation failure wearing a better suit.

I ran a recruitment agency before I built Pickr, and I was as guilty of this as anyone I criticised. I would open the CV in the 4 minutes between calls, find something to ask about, and improvise for 45 minutes. It felt like judgement. What it produced was a pleasant conversation and an opinion I could not defend two weeks later.

Why do unstructured interviews predict so little?

The ordering has been stable across decades of meta-analysis: a structured interview predicts on-the-job performance roughly twice as well as an unstructured one. When the field revised its validity estimates downwards in the early 2020s, structured interviews came down with everything else and still finished at about double. Unstructured interviews are not worthless. They are just a weak instrument being asked to carry a decision worth tens of thousands.

Three mechanisms do the damage, and all three are preparation failures rather than skill failures.

Nothing is comparable. If candidate A talked about a migration and candidate B talked about a team conflict, you are not comparing two candidates. You are comparing two conversations. The shortlist debate then turns on who happened to be more articulate about whatever they happened to be asked.

The judgement forms early. Interviewers reach a view in the first few minutes and spend the rest of the slot collecting support for it. The specific number of minutes you see quoted is folklore; the pattern is not. A prepared question set does not switch the instinct off. It does force the interviewer to gather evidence on criteria they did not choose in the moment.

Nobody owns the hard part. Every role has one or two things it will actually break on. Without an explicit division, everyone assumes somebody else is covering it, and the loop ends with four opinions about communication and nothing about the thing that matters.

Why does interview preparation never happen?

Watch what really happens. The invite lands, 30 or 45 minutes get blocked, and the interviewer opens the CV somewhere between 4 minutes before the call and 2 minutes into it. Nobody has written down what this particular interview is for. Nobody has told the four people in the loop who covers what. So all four open with "walk me through your background", the candidate delivers the same rehearsed 6 minutes four times, and a large share of every slot goes on re-establishing facts already on page one.

Price that out. A 4-stage loop at 30 to 45 minutes a stage is about 3 hours of the candidate's time and the same again in interviewer time, before anybody writes anything down. Add the writing-up and the debrief and you are at 5 to 6 hours per candidate. Across a shortlist of 4 that is 20 to 24 hours per role, the better part of three working days, spent producing evidence you cannot compare because no two candidates were asked the same things.

The preparation that would fix it costs about 20 minutes. Once, per role. Not per candidate.

That is the whole argument and it is not a technology problem, which is an awkward thing to write as somebody who sells software.

What does structured interview preparation look like?

Four steps. None of them need a product.

1. Criteria before questions. Five or six criteria, drawn from the brief and ranked, written down before anyone drafts a single question. If you cannot say which two matter most, your brief is the problem rather than your interview, and that is a different job covered in how to write a job brief.

2. One or two questions per criterion, asked of everyone, in the same words. Consistent wording matters more than clever wording. Variation in phrasing is variation in the test.

3. A written division of labour. Assign criteria to interviewers in the calendar invite, not in a corridor ten seconds beforehand.

4. An agreed evidence standard. Decide in advance what a strong answer contains: a specific example, the candidate's own actions rather than the team's, a real outcome. Without that, "strong" quietly comes to mean "confident".

A panel plan for a mid-level engineering role fits on a napkin:

Interviewer	Owns	Asks about
Recruiter, 30 min	Motivation and constraints	Why now, what they are moving away from, notice period
Hiring manager, 45 min	The two top-ranked criteria	Two worked examples, pushed down to specifics
Peer engineer, 45 min	Depth and adjacent skills	One hard problem the role will genuinely hit
Cross-functional, 30 min	Collaboration and handovers	A disagreement they lost, and what happened next

Nobody asks anyone to walk through their CV. It is already in the record.

How do you write interview questions that produce evidence?

Three rules cover most of it.

One criterion per question. A question that asks about ownership and communication at once returns an answer you cannot file under either, and the interviewer scores an impression instead of a criterion.

Ask about something that happened, not something that might. "How would you handle a disagreement with a staff engineer" tests how well somebody theorises about conflict. "Tell me about a disagreement you lost" tests what they did. Hypotheticals reward fluency, which is exactly the trait an interview already over-weights.

Write the follow-up ladder in advance, because the first answer is almost always a summary. Two follow-ups get you to specifics: what was your part of it, and what happened after. Interviewers who improvise the follow-ups tend to run out of them early and fill the rest of the slot with rapport.

The trade-off is real and worth stating. A fixed question set makes it harder to chase the interesting thread a candidate opens at minute 12, and rigidly delivered it can feel like an interrogation rather than a conversation. Fixed questions are a floor, not a script: ask the agreed set, then use whatever time is left however you like.

Do AI-generated interview questions help?

The reason structured interviewing loses to improvisation is not that people disagree with it. It is that building a question set for every role is real work landing on a recruiter with six other things due today. So the intention survives and the practice does not.

That is the part worth automating. A question set drafted from the role's requirements, sitting in the invite before the call, changes the default from improvisation to a draft somebody edits. In my experience editing a decent draft takes about 5 minutes and writing one from a blank page takes 30 to 40, which is precisely why one happens for every role and the other happens for the role you had a quiet Tuesday for.

Two limitations, stated plainly. A generated question about a skill the role does not actually need is worse than no question, so a human has to read the set before it goes out. And consistency is not quality. A consistently shallow interview is still shallow. It is simply now shallow in a way you can see and correct.

Pickr is the AI-native recruiting platform that scores candidates on evidence of skills rather than keyword matches, including adjacent and transferable ones, and those scored criteria are the same criteria the interview features exist to test. Interviews are transcribed, and the scorecard arrives pre-filled with evidence mapped back to each criterion, so the interviewer edits a draft instead of facing an empty form on Thursday about a conversation they had on Monday. Interviewer and hiring-manager seats are free, which is deliberate rather than generous: the moment feedback costs a licence, somebody caps the licence count, and the evidence starts arriving as hallway conversations that never reach the record.

How do you measure interview consistency?

Interview quality is hard to measure. Interview consistency is not. Three numbers tell you nearly everything:

• What share of interviews had ranked criteria attached before the call.
• What share produced a completed scorecard within 24 hours.
• What share of your ranked criteria were covered by somebody, anybody, across the loop.

If fewer than half your interviews produce a scorecard within a day, you do not have a scorecard problem. You have a preparation problem surfacing one stage later, because a scorecard can only be filled in 6 minutes if the criteria it asks about were agreed before the call. Read those three numbers next to what actually happened to the people you hired, which is the outcome data Pickr's recruiting analytics keep against every placement. Distributed panels make all three numbers worse, which is why structured interviews matter most in remote hiring.

You can count all three by hand for one month with a spreadsheet and a calendar export. That is a fair first step and it costs nothing.

What to change before your next interview loop

Take your next open role and spend 20 minutes on it. Rank 5 criteria. Write 6 questions. Assign them across the panel inside the calendar invites. Ask every candidate the same set. Require the scorecard before the interviewer's next call rather than before the debrief.

You need no vendor for any of that. It is the cheapest change available to most hiring teams and the one least likely to get done, because it is nobody's job. Software earns its place at the next step: drafting the question set so preparation happens for every role instead of the occasional one, and turning the interview into a scorecard draft while the evidence is still warm. The sequence matters, though. Prepare first, automate the preparation second. No interviewer has ever regretted walking into a call already knowing what they were there to find out.