Everywhere around medium, blogs, and LinkedIn, you’ll read people talking about how the technical interviews process is broken, it’s unfair, time-consuming, and ultimately not useful.

What does that actually mean, and why most companies are not interested in doing something about it? And, come to ask, is there anything you can do if you work for such companies?

The process is broken

We can all agree that the system is broken? But, can we? Many people could argue that the process is fine as it is, that it produces good results and while it may not be perfect we just need to accept it. Courses are held to teach how to ace technical interviews, books have been written on how to crack it and people even boast about how many leetcode exercises they have solved as if that meant some sort of superpower. Is the process broken for them? Hard to tell.

I will argue that the process is broken. Completely. Not only because it doesn’t evaluate what I think really matters but, more importantly, because it can be gamed! Technical interviews are, in many cases, no longer about knowledge, expertise, or even problem-solving skills, they’ve become a number game where if you can answer the questions in a certain way or solve a coding challenge from a pool of challenges then you’ve aced it. It doesn’t really matter whether that will apply to your job or not or whether you have real experience to back it up.

What do most companies asses vs what should be assessed?

When I am interviewing for someone to join my team, I care about only these things:

• Ability to learn from experience and past mistakes
• Ability to abstract away what’s the core of a problem from what’s it’s just unimportant context
• Ability to understand the trade-offs between getting more clarity on requirements vs making decisions and executing
• Basic craft skills (e.g. a developer should be able to write code)
• Ability to talk about and explain all of the above

In contrast, this is what most companies assess with their processes:

• Ability to memorize ideal answers without regard for what’s actually needed
• Ability to create speeches to say the right things
• Ability to write clever code
• Ability to remember theoretical concepts

Most technical interview processes don’t assess what I am arguing are the key areas. With the current process, I don’t think it’s possible to extrapolate how a person will be able to understand the company’s business and problems and solve them from the type of questions and challenges they do.

If the role’s description is solving coding challenges in the most efficient way without any regard for code cleanliness, testing, or performance vs delivery trade-offs, the current technical interview process will be well suited for it. But that’s not the reality for most companies.

Most companies’ development jobs involve solving roughly the same technical problems within the roughly same context of unclear requirements, lack of documentation, and quality vs speed trade-offs. Although no interview process will tell you exactly how a person will act in a real job, a process that focuses on assessing things like when the person asks questions about requirements and when they make a decision, or how the person receives a lot of information about a problem and they decide what’s the core of it and what’s just noise and how they explain that, things that they will actually do daily, will definitely provide more trustworthy and accurate results.

Why don’t all companies change then?

Two main reasons: interviewers don’t always care and simplicity for managers. Let me explain.

Interviewers don’t always care

I have been interviewing people for the last 8 years and I’ve done it in 3 different companies. Two of those companies were big, multinational companies with hundreds of developers, and the last one, the current one, it’s a startup with 6 developers where I interviewed all of them.

The interview processes were massively different in the big companies from what it was in the startup.

In my current company, I was able to create the interview process that I thought was best, from the topics I’d cover, the number of rounds, the type of technical questions I made, and how I made them. I was privileged to have the space to do that and I was lucky enough to have experienced too many bad interviews (both as interviewer and interviewee) to want a change and have the tools for it.

In contrast, in the big companies, I had zero agency over how the interviews were done, the topics I’d cover, and the questions I’d make. Before an interview I would pick a coding challenge from a list of approved challenges, I would read the expected answer and the questions I had to cover.

In my current company, I interviewed the people who would work closely with me, the people I would delegate things and rely on to help me solve problems at a fast pace with limited resources.

In contrast, in the big companies, I would interview people that would not join my team and instead would join a pool of new hires that would end up in a team that I couldn’t predict. This made the interview be even more generic because you didn’t know the needs of the team that would receive this person.

This goes all to say that, in the big companies, the stakes were lower. I was less incentivized to really assess what I thought was important because the chances of being impacted by making a bad hire were lower and, on top of that, even if I decided to change how we interviewed, I would have to go through a bureaucratic process where I would suggest some changes by writing a very lengthy document that would get reviewed and people would comment with agreement or disagreement and then would get reviewed again and then maybe if I was lucky the changes would slowly be rolled out. I was never that lucky. I tried a few times to no avail and I stopped trying.

That’s why I meant when I said that, oftentimes, companies won’t change the process because most interviewers don’t care. Being part of a process that has low stakes for the interviewer that has no or little agency to change is not the best scenario for trying out new ways of interviewing and challenging what needs to be assessed.

Simplicity for managers

Another difference between the startup I work for now and the big companies I worked for before is what would happen after the interview: meeting to decide whether that person was a yes or a no.

In my current company, after an interview, we gather with the developers and we talk about our impressions, what things we found interesting, what possible red or amber flags we noticed, what surprised us for good or bad, and whether we see ourselves working with this person in the feature. We use expressions such as I think or I feel, we don’t have scorecards or a fixed process. We don’t expect black or white answers and we welcome guesses. All we’re trying to do is share all of our points of view and find common ground.

In the big companies, we had scorecards and we had to score dozens of items. We couldn’t use our impressions without a concrete data point behind them. This seems a good approach at first but taken to an extreme, it can become a problem.

The skills that are important to assess in an interview are subjective, nuanced, and qualitative. Focusing on scores and data-backed answers instead of feelings, impressions, and open-ended answers inevitably will end up focusing on the subjective aspects of the interview process, such as whether the candidate answered a set of questions in the expected way. or the code challenge was solved in an approved way. The nuance is lost.

That’s why I think the current, broken interview process is simpler for managers. It’s easy to measure, it doesn’t require actual involvement to work through the nuance. The fact that what is being measured is not valuable is irrelevant because in many cases they are not hiring for themselves but for the company.

Can we even do something to change technical interviews then?

I think we can but it won’t be easy. If you’re lucky as me to be in a position where you can shape the entire interview process, please take that opportunity to implement a fair process that doesn’t fall into the same mistakes. Try something new, evolve it, share what works, and always be very clear about what you’re assessing and why.

If, I guess, as most people, you are not in a position to change the process you’re company is using completely, I think something that you can do and is achievable is to try to use the existing process and questions in a more open-ended way. Focusing on turning the interview into a conversation and using the follow-up technique can be a massive improvement in the right direction.

I have written about the value of using follow-up questions to asses expertise and why they are a powerful tool.

Conclusion

The technical interview process needs a complete overhaul and luckily we’re already seeing some changes in the right direction, but unfortunately, until big companies start shifting in that direction too I’m afraid we will only see localized improvements in small companies.

If you are involved in interviews, I’d urge you to focus on the candidate’s ability to analyze a situation and extract the key problem from it, understand the trade-off in understanding a problem in depth vs the need for execution and then use past experiences to work out a solution to a problem they’ve never seen before.

Let’s change it!