Keeping Score: Getting the Best Out of Hiring Panels

Posted on April 19, 2021

Hiring panels can be a mixed bag, and don’t always deliver. Multiple interviewers should bring more viewpoints and help dig deeper and choose the best candidates from the pool.

But not always. Sometimes interviewers just choose candidates they like. Different panelists may use different criteria to make their decisions or might not really have clear criteria in mind at all. Group think, distraction or laziness can all have an impact. Good candidates can be lost and friendly candidates with strong presence  but lacking the right skills can sneak through. The results can be disappointing, ultimately leading to bad hires, more recruitment… and more interview panels.

Faced with this challenge, many firms try harder, accepting that recruitment is a hit-and-miss affair. They may look at even more resumes, hoping for better luck. Other approaches are to work from a list of interview questions then meet immediately to discuss while conversations are still fresh. But scripted questions don’t help panelists decide how to assess the answers. And if several interviews are happening in a day, everyone’s brain turn to mush anyway.

Interviewing with Scorecards

A creative solution is to produce a detailed interview scorecard for each position. It should have prompts to ask technical, interpersonal and values-based questions. It should provide a standard grading scale developed with specific reference to the job requirements, as well as company culture and ethics. And panelists should be coached on how to use it. A good interview scorecard will capture detail, force interviewers to document their thoughts, and provide a fair comparative platform if several candidates are interviewed over a span of time (not to mention, legal protection in the rare case of a challenge). Above all, the investment in effort to develop effective interview scorecards will be repaid in improved, more sustainable hires.

“We wanted to bring more eyes into the hiring process to get better decisions, but the question was how to do it effectively?” says President and CEO Kevin Peterson of the Long Beach, California-based engineering and design group P2S. “So we invested in a review of our recruitment process, which resulted in hiring panels, scorecards, interview training – and significantly improved recruitment.”

Making Panels Work

1. As soon as the job description and advertisement are written, create the interview scorecard. This should be done together by HR and the hiring manager, with the former ensuring the structure and detail of the scorecard and the latter focusing on essential requirements and experience for the post.
2. Craft the core section of the scorecard to prompt interviewers to ask about job history – specifically what candidates actually did in prior jobs. There should be a half dozen or more tasks or duties that are essential to the job and for which demonstrable experience should be confirmed.
3. Develop one or two case studies relevant to the position to give to all candidates. This should be scripted out, so that panelists can ask each candidate how they would respond to the exact same situation.
4. Create a standard section for gathering impressions of the candidates in key areas, including corporate culture, values, career goals and ambitions. Again, draft questions that will draw out specific stories and experiences to move beyond generalities.
5. In each section, leave room for notes by each interviewer and a score. HR and the hiring manager should talk over weightings and give direction to the panelists about priorities and levels of importance for different qualities and experience.
6. Once the scorecard is completed and the panelists are selected, coach the panelists on how to use the scorecard effectively. This may include a refresher of basic interview techniques (the importance of holding your tongue and listening!), a review of the specific questions and an agreement for dividing up who asks what.
7. Schedule interviews so that there is an adequate interval between each to capture impressions in writing. Immediately after each interview, panelists must stay in the room (or offline for distance interviews) and, in silence, rate the candidate. Once all panelists have completed their forms, then they should have a conversation among themselves – not before.
8. Following interviews, collect the scorecards with completed notes for the records. Any panelist who fails to write up his or her impressions after an interview should be excluded from participating in future interview panels.

Three years ago, P2S adopted a panel-based approach backed by interview scorecards. (See here for a sample.) Where previously only the hiring manager would conduct interviews and make the decision, now at least three managers are required to participate in all interviews.

Since adopting the new process, P2S’s rate of bad hires has decreased dramatically. “Every year, we used to make 4-5 unsuccessful hires,” CEO Peterson attests. “Since adopting the new process, we haven’t had a single hiring failure.”

Better Panels, Better People

Hiring panels take time and managers are already busy enough. They may resent having to participate in recruitment processes for positions they do not oversee and for which they may not feel best-positioned to judge. Making the process more time-consuming by working up detailed scorecards, coaching panelists, spreading out the interview schedule to incorporate assessment and discussion time – all of these steps only add to the perceived burden. Is this process just overcomplicating things?

Your company is your people, and excellent hiring is the foundation of your success, regardless of department. Increasing the quality of new hires will benefit everyone. Hiring panels spread the decision-making burden and improve the results. But if you are going to invest the significant time and effort required, it is critical to do it properly. Scorecards structure interviews, align criteria, enhance efficiency and result in better decisions exactly where it matters most: choosing your people.