Hiring Engineers Is Broken. Here's What Actually Works.
Years with a framework, brand-name employers and whiteboard puzzles do not predict who ships. What does, and what take-home tests get wrong.
Engineering hiring mostly measures the wrong things. Years with a named framework, the logo on someone's last badge and a whiteboard puzzle nobody performs at work survive because they are easy to sort, not because they predict who ships. Three things do predict it: evidence of work the candidate actually built, reasoning under a realistic constraint watched live, and a structured technical conversation scored the same way for every candidate. The rest of this is how to swap the proxies for those three, what the swap costs you, and where it still does not work.
I ran a recruitment agency before I built Pickr, and I sold those proxies for years. Clients asked for 5 years of React, so I searched for 5 years of React. It is fast, it is defensible in a meeting, and it is close to worthless.
Why engineering hiring runs on three weak proxies
Years with a named framework. A job description asking for more years of a technology than the technology has existed is a recurring industry joke, but the ordinary version is worse because nobody laughs at it. Time spent near a framework tells you nothing about the shape of the work. 5 years of React can mean owning a design system used by 40 engineers, or maintaining one form for a team that shipped twice a year.
Brand-name employers. A big logo tells you that a hiring committee said yes to this person some years ago, under conditions you cannot inspect, at a company probably growing headcount faster than it could assess it. It overvalues people who maintained one narrow service inside a large machine, and undervalues the engineer who carried a whole product somewhere you have not heard of.
The whiteboard exercise. Solving a graph problem with no editor, no documentation, no compiler and no colleague measures interview practice and composure under artificial observation. Both are real skills. Neither is the one you are paying for.
To be fair to all three: they are not stupid, they are cheap. 200 applications have to become 8 conversations somehow, and the proxies are the only filter most systems can apply in seconds. The answer is not to sort more slowly. It is to sort on something better.
What actually predicts engineering performance
Evidence of shipped work. Public repositories with a commit history, an internal migration the candidate can walk you through end to end, a conference talk, a well-argued bug report against an upstream project. What matters is not polish, it is specificity: which constraint, which trade-off, what broke afterwards, what they would do differently. People who did the work answer that in concrete nouns. People who watched it answer in adjectives.
Reasoning under a realistic constraint. Take a problem from your own backlog, shrink it until it fits in an hour, and work through it with the candidate using their own editor, the real documentation and search. You are watching them decide what not to build. That decision is most of senior engineering, and it is invisible in every puzzle format.
A structured technical conversation, scored identically for everyone. Same core questions, same criteria, same scale, every candidate, written down before anyone meets anyone. Structured interviews outperforming unstructured ones is one of the more replicated findings in selection research, and it is still ignored by most engineering panels, which are typically 4 people improvising in sequence and then comparing feelings in a room. Structured interviews and pre-filled scorecards are the unglamorous half of this, and the half that moves your hit rate. If your panel resists, the honest framing is in the hiring manager scorecard guide: the scorecard is not there to police interviewers, it is there so the fourth person in the loop is not overwriting the first three from memory.
What take-home tests get wrong for engineering roles
Take-homes were the well-intentioned fix for the whiteboard, and they broke in four specific ways.
They tax the wrong people. A 10 to 15 hour unpaid exercise selects for slack in someone's life. The parent working full time drops out; the 24-year-old with no dependents finishes it on Saturday. You have run a filter on free time and called it a filter on ability.
They are graded inconsistently. Most teams have no rubric and no calibration. The same submission gets a yes from the reviewer who likes tests and a no from the reviewer who likes brevity, and nobody notices because nobody compares.
They measure the easiest phase of the job. Greenfield code in an empty repository is the most pleasant programming there is. The actual job is reading code you did not write, in a system with history and constraints you have to discover.
They no longer prove authorship. A weekend take-home is now a 20-minute prompt. If the submitted artefact is your gate, you are measuring prompt fluency and nothing else.
Banning AI assistance is unenforceable and beside the point. Change what you evaluate instead: cap unpaid work at 3 hours, pay for anything longer, and move the assessment live — same problem, 60 minutes, own tools, AI allowed and expected. The question stopped being whether a candidate can produce a function. It is whether they can judge the output, spot the plausible-looking wrong answer, and explain why they kept one version and threw the other away.
Why keyword matching fails for developer roles
Start with the case every engineering pipeline drops. A candidate has never been paid to write Rust, so every line asking for commercial Rust experience removes them — and they have maintained a crate for 3 years, with releases, an issue tracker and contributors who are not them. That is more inspectable evidence than the word Rust under a job title, and it is exactly the record a keyword filter cannot see. The same logic runs across languages: 8 years of production Java covers most of what a Go role needs, because concurrency under load, service decomposition and the instinct for which failure will page you at 3am do not reset with the syntax. The syntax costs about 2 weeks of ramp-up; nothing else on that list does. The general version of that argument is in why keywords are dead.
What makes technical roles different is that the evidence is unusually available and almost nobody reads it. Commit history shows whether someone works in reviewable increments or drops 4,000 lines on a Friday. Review comments on other people's pull requests show how they disagree. Upstream bug reports and design documents show how they explain a system to someone who cannot see their screen. None of that is on the CV, all of it is public or askable, and none of it survives a filter matching nouns. Meanwhile the filter punishes honesty: the candidate who lists what they genuinely used loses to the one who lists everything they have stood near.
Pickr is the AI-native recruiting platform I built for this: it scores every candidate on evidence of skills rather than keyword matches, including adjacent and transferable skills, which is what keeps the crate maintainer and the Java engineer on the list instead of in the rejection pile. Evidence-based candidate matching changes who reaches your shortlist; the patterns specific to engineering pipelines are collected in hiring for technology roles. Two more things matter for technical loops. Interviews are transcribed and scorecards arrive pre-filled with evidence mapped to each criterion, so a staff engineer edits a draft instead of confronting an empty form on Thursday about a conversation from Monday. And what happened to the people you actually hired feeds back into how the next candidates are evaluated, so the working definition of a strong backend engineer comes from your own hires rather than a template. Interviewer and hiring-manager seats are free, which is a design decision rather than a discount: the moment feedback costs a licence, feedback stops arriving in the system.
What evidence-based technical hiring costs you
It is more expensive at the top of the funnel, and anyone telling you otherwise is selling. A live problem session spends 60 minutes of a senior engineer's week per candidate, and a panel that agrees its criteria in advance has to hold a meeting most panels currently skip. Structured interviews are also mildly unpopular with interviewers, who feel a script removes their judgement, when what it removes is the fourth interviewer's ability to overwrite the first three.
The harder limit is that evidence-based scoring can only read what exists. An engineer whose best 5 years sit behind an NDA at a defence contractor, or who never writes anything down, looks thinner on paper than they are, and no scoring approach fixes that — the interview has to. Public work also skews towards people with the free time to produce it, which is the take-home's bias in a nicer jacket. Treat absent evidence as absent, not as a negative, and keep a live conversation that can still rescue someone whose record is invisible.
How to redesign an engineering hiring loop
| Stage | The proxy version | The evidence version |
|---|---|---|
| Screening | Keyword match on framework and years | Evidence of shipped work, adjacent skills counted |
| Technical assessment | 10 to 15 hour unpaid take-home | 60-minute live problem from the real backlog, own tools |
| Panel | 4 people improvising, verdict by discussion | Same questions, same criteria, written before the loop |
| Decision record | A rating and the word fit | Specific observations tied to what was said |
| After the hire | Nothing | Outcome at 30, 60 and 90 days feeds the next brief |
In the loops I have run and watched, engineering hiring commonly takes 30 to 45 days from first contact to offer, and most of that is scheduling and waiting on written feedback, not evaluating anyone. Fixing the assessment usually fixes the timeline as a side effect, because a structured conversation produces a decision the same day and an unstructured one produces a meeting 3 days later.
Where to start without new software
Take one open engineering role. Replace the take-home with a 60-minute live problem from your own backlog. Write the criteria down before the panel meets anyone, and have every interviewer score against the same ones. Record the outcome at 30, 60 and 90 days so the next brief is written from what happened rather than from the last job ad. None of that requires buying anything. It does require deciding that a shortlist is something you build from evidence, not something you filter out of a keyword search.
Frequently Asked Questions
What actually predicts engineering performance in a hiring process?
Three things carry most of the signal. Evidence of work the candidate actually shipped, described with enough specificity that you can tell what constraint they were under. Reasoning under a realistic constraint, watched live rather than graded afterwards. And a structured technical conversation where every candidate gets the same core questions and is scored against the same criteria. Years with a named framework, brand-name employers and whiteboard puzzles are proxies for these, and they are weak ones.
What is wrong with take-home tests for engineering roles?
Four things. They tax candidates who have spare time rather than candidates who have ability, so a 10 to 15 hour exercise quietly filters out parents and people already working full time. They are graded inconsistently, because most teams have no rubric and no calibration between reviewers. They measure greenfield code in a clean repository, which is the easiest programming there is, while real work is reading code you did not write. And a current language model produces a passable submission for most of them in minutes, so the artefact on its own no longer proves who wrote it.
Why does keyword matching fail for developer roles?
Because the keyword lives on the CV, not on the person. An engineer who has never been paid to write Rust but has maintained a widely used crate for 3 years will be dropped by every keyword filter, despite leaving behind more verifiable evidence than someone with the word Rust under a job title. The failure runs the other way too: 8 years of production Java carries most of what a Go or Kotlin role needs, and no keyword search will surface it.
How long should an engineering interview loop take?
In my experience engineering loops commonly run 30 to 45 days from first contact to offer, and the majority of that time is scheduling and waiting for written feedback rather than evaluation. 2 weeks is achievable for a four-stage loop if interviewers submit their scorecard the same day, which mostly means removing the friction of writing one from a blank page days after the conversation happened.
How does Pickr evaluate engineers differently?
Pickr scores every candidate on evidence of skills rather than keyword matches, and it counts adjacent and transferable skills instead of requiring the exact noun from the job description. Interviews are transcribed and scorecards arrive pre-filled with evidence mapped to each criterion, so an interviewer edits a draft rather than facing an empty form days later. What happened to the people a company actually hired then feeds back into how the next candidates are evaluated.
Free recruiting audit · 2 minutes
Find out what your hiring process is actually costing you.
Answer eight questions, or connect your current system read-only, and get a report on where your funnel loses candidates and which changes are worth making. No signup, no API key stored, data stays in the EU.
Written by Andreas Amann
Founder of Pickr. Former operator at startups in Berlin and Silicon Valley, where he helped scale companies from 40 to 200+ people. Built Pickr after years of using every major ATS as a recruitment agency owner at ScalingPPL.