[su_note]You can listen to Eric’s interview on the podcast right here.[/su_note]
Many know it can be difficult to measure the quality of a hiring decision. Just as important is to do so rather quickly, so output from the process can be fed back into the system, improving both the quality and accuracy of subsequent interviews (and their resulting hiring decisions). A major obstacle to measuring quality-of-hire (QoH) is that job candidates are exceptionally complex (we’re human, after all!), and performance reviews typically do not occur for some time (up to a year, in many cases) after hire. Additionally, elevated individual performance is only one (and a rather narrow) potential outcome of a successful hire; what about team/company integration or job fit?
One way to solve this problem is to construct a multi-faceted framework for reflecting on the impact and efficacy of the interview process, and to build that framework in a manner that does not simply provide an assessment of decision quality, but also points to precise points of failure. Here I’ll detail Namely’s internal approach to developing, reporting, and validating our QoH framework.
Pre-Hire
When we set out to measure QoH, Namely had just transitioned to a scorecard-based applicant tracking system, allowing us to collect detailed interviewer ratings of candidates against sets of competencies specific to every role at the company. These scorecards served as key data touch points, in which all candidates were evaluated on multiple competencies by multiple interviewers, providing a rich picture of their interview performance and their demonstration of competencies during their on-site visit.
The QoH Index
In partnership with top-performing managers and executives at the company, we developed a composite QoH index, consisting of four elements: the new hire’s (1) ramp to productivity, (2) job-skill fit, (3) values fit, and (4) “bar-raising” behavior. These components aligned with the hiring goals of the company, and were selected to provide a well-rounded profile of each new hire’s overall early impact.
Data Collection
Due to data availability constraints, the index was designed and launched as a manager-report survey, distributed to managers eight weeks after a new hire’s start date. The survey asked for (1) ratings on each of the four index components, and (2) ratings on the key competencies that the new hire was assessed for in their interview (remember, these are specific to each role!). The survey was fewer than 10 items long, all items were five-point Likert scales, and the whole survey took managers on average 2-3 minutes to complete for each new hire.
Reporting and Continuous Improvement
The four index components were averaged to yield an individual’s QoH score. These individual scores were never used to make decisions about individuals, but were instead reported as the percentage of scores above a “high-quality” threshold (e.g., above 4.0). This “QoH Index” was reported directly to executives and the board of directors on a quarterly basis. The post-hire competency ratings on the survey were tied to the pre-hire ratings of the same competencies from the interviews (because we now had two snapshots of the same person), and through a blend of correlation and regression we were able to identify precise competencies that we were ineffective (or effective) at evaluating as a company, and leverage those learnings when building and improving our interviewer training program.
Validation & Outcomes
Finally, as a result of measuring QoH and feeding these reflections back into the interview and hiring process, Namely saw its overall QoH score increase by 18% over three quarters. Further, the four-item index was found to be both a statistically and practically significant predictor of turnover; employees scoring in the lower half of the QoH scale exhibited a 50% probability of leaving before their one-year mark, while those scoring in the upper half had just a 5% probability of leaving. Interestingly, the most predictive item of both turnover and early job performance was the item on values fit.
This was a brief summary of our journey building a QoH framework. I emphasize the idea of QoH as a framework, not just a metric, because adopting a framework mindset enables you to build and adapt a series of tools or measures (not just one) that empower you to reap all of the benefits of these reflections. Particularly, an understanding of how well we hired, and a hyper-targeted view of where we went wrong (or right).
