Debrief meetings can be short and productive.
Wrapping the Code Interview
Whether you work on a big company or a start-up, after some time you will be involved in interview loops.
In some companies you will be trained and taught how to prepare for an interview, how to ask behavioral questions, and which values you need to look for. In some cases, you will also be taught how to ask and evaluate technical questions. After your official training, you will be invited to shadow some interviewer(s) and continue learning by example. In smaller companies you will start at the shadowing phase.
What most training sessions skip over is what happens after we introduce the candidate to the next interviewer.
We need to look at those notes (you took notes, right?) and make a decision: is this candidate good enough to give them an offer? Luckily, you do not decide alone; there will be a debrief session… which most probably was also not covered in the training.
Here we will discuss the steps after the actual interview, how to provide feedback and get the most from your debrief session. Keep in mind that your company can have their own process and you have to follow their approach if it conflicts with our discussion here, but these guidelines may still prove useful.
Interview Notes and Feedback
We will start by talking about your notes and feedback. Your feedback will be based on the notes you take during the interview, so the better these are, the easier it will be giving your feedback.
“Good notes are the first step for giving valuable feedback”
You can use any method to take your notes, if you prefer typing on your favorite editor, take your laptop to the interview. If you prefer pen and paper, grab a notebook and pen on your way to the interview room. In any case, always tell the candidate that you will be taking notes, so they do not feel that you are writing an email, slacking with somebody on your team or writing a poem while they try to do their best to answer your coding question.
Once selected the medium to record your notes, the next relevant point is what needs to be written in your notes. The details and structure of your notes will depend mainly on your own style, but I recommend to start the interview with some notes already written. You already know which questions you are going to ask, so you can have those already in your notes. If using paper and pen, put different questions in separate pages. Have any alternate or follow up questions there as well.
Your notes do not need to be perfect, they are meant to help you remember what happened during the interview when giving feedback and eventually during the debrief during the final discussion. For this reason, your notes should be taken as reminders of relevant moments during the interview.
Take notes if the candidate asks good clarifying questions, or makes a set of assumptions before solving a problem. Add also general notes about their process. For instance, write down whether the candidate verbalizes their thoughts or stays quiet while thinking about the problem, or whether you have to point out a bug, or they discover the bug by themselves. Also note any time you have to give hints. Include some few details about their abilities to solve and explain problems, use of the whiteboard and other tools and resources.
There is no need to write down the code as they work on it, just have your phone or a camera close and snap a couple pictures at the end, or before they erase any partial code they have. You can transcribe from these pictures and also have them at hand during the debrief.
As I previously said, good notes are the first step for giving valuable feedback.
“You can experience what is called confirmation bias”
Thorough notes will help you remember what happened at the beginning and middle of the interview. This is relevant because it is easy to get biased by whatever happened by the end of the interview. If the candidate had a moment of inspiration by the end of the last question and ended with the most optimized solution you have seen for your problem, you may forget you had to give them 15 hints to help them get there. Or a candidate panicking in the last 10 minutes because their code is not complete can make you forget that they mastered the previous two questions.
On top of that, several of the recruiting tools in which you have to enter your feedback expect you to select your vote (usually 2 or 4 options) and your feedback. Also, recruiters want you to start your textual feedback by stating your vote.
Do not state your vote before writing the rest; doing so can bias your whole feedback. If you decide now your view may be skewed by what happened by the end of the interview (as I already mentioned) and you can experience what is called confirmation bias, that is, you will read your notes and unconsciously focus on the facts that confirm what you have already decided.
A better way is to skip the vote and read your notes trying to reconstruct the whole interview. Write this reconstruction and look at it as if you were an observer, then evaluate the candidate based on all the information. Try to objectively weight the good signals and the bad ones. Add a conclusion at the end and then state your vote.
Writing this reconstruction from your notes can get very detailed and long, therefore it can be good to summarize it for the recruiter (and others reading your feedback). An interesting practice is, after you have finished writing the whole reconstruction and the conclusion at the end, you can go back to the beginning of your feedback and add:
- A first line stating your vote (I am inclined/not inclined/other options…)
- A TL;DR paragraph in which you summarize the whole feedback in two or three lines.
Debrief
You managed to give your feedback. The next step is to prepare for the debrief.
What is expected during debrief and how can you give your best?
During the debrief session all the feedback is contrasted and the team should come to the final conclusion whether to extend an offer to this candidate or not. To do this, all participants would discuss the different feedbacks and try to complement their initial impression on the candidate with this extra information. The idea is to interchange points of view, clarify contradicting pieces of information, and finally all participants would vote again, with a more complete view of the candidate.
“The debrief meeting is to talk about the candidate — not the interview!”
Ideally, all participants would have access to all the feedback before the meeting. To accomplish this, it is relevant that each interviewer enter their feedback timely. If everybody has read all the feedback, the debrief meeting can start directly with a previous recount of votes and addressing the points that have conflicting data from the different interviewers.
To prepare for the debrief then, you should read all the feedback and again, take some notes. Anything reported by other interviewer that seems to contradict what you saw is a point that you should try to clarify during the meeting. For instance, if somebody indicates that the candidate was too quiet, but in your interview, the candidate explained all they were doing. Also take note of any piece of information you were not aware of that could either explain something you observed or change your point of view on the candidate.
During the debrief participate actively on the discussion. If somebody mentions any of the points you have in your notes, read them aloud and explain why you agree / disagree. In contradicting pieces of information, try to find some logical explanation. Maybe the candidate was a little shy at the beginning of the day, but get more relaxed as the interviews progressed. Or they felt they did not do well in our of the interviews, they could have acted less confidently in the next one.
Once these discrepancies had been discussed, you should have a clear idea on what your final vote should be. If you still have some doubt, present it and try to get the opinion of the rest on that point. When everybody has their final vote, the recruiter and the hiring manager will collect it and work on next steps. Your work is done — congrats
Reducing Bias
We talked a little bit about confirmation bias and how to use your notes to prevent it, but there are several other considerations in the process in order to avoid other possible interferences that can bias your vote.
First, and this depends on the process in your company, so it is not always easy to accomplish, is to avoid talking with other interviewers about the candidate before entering feedback. We already said you should reconstruct your experience based on your notes and your memory. If you talk with somebody else, you cannot unhear their point of view.
Even if you are shadowing an interview, if you are going to enter feedback and participate on the debrief, then you should discuss with the main interviewer after both have entered your feedback.
Once you have entered your feedback, you can discuss with other interviewers that have entered theirs, but be very careful not to be overheard by other interviewers who have not. Discussing your feedback with these interviewers that entered theirs can actually lead to a shorter debrief.
That’s it! Hope this helps and that you enjoy recruiting the best candidates for your organization — Good Luck!
