Tutorial: Ratings

Companies often have trouble maintaining an objective, consistent, and constructive interview process. Not only do you have to contend with a broad range of jobs and related skills to be assessed, but also with a variety of different personalities and styles, on both the hiring teams and amongst the candidates. All of these factors can lead to disparity across interviews, which often negatively impacts how hiring decisions are made and ultimately how candidates perform when hired. But here at JobScore, we’ve got a solution for this problem -- ratings.

What are Ratings?

Ratings are a simple and effective way to gather feedback from your hiring teams as they work through the interview process. When entering feedback about a candidate, interviewers are not only presented with a text template to complete, but also with a series of ratings that are relevant to the specific job and stage of the hiring process. Interviewers simply click on icons or enter numeric figures that best correspond with their assessment of the candidate’s skills -- that’s it!

Why Should We Use Ratings?

In short, ratings provide an objective, consistent, and constructive way of assessing candidates’ skills. Here’s how:

Rather than relying solely on written feedback from your hiring teams -- which often varies widely from one person to the next -- ratings provide a simple and quantitative method of candidate assessment. In other words, it’s not “tell us about this candidate’s aptitude for skill X” -- it’s simply “rate this candidate on a scale of 1 to 4 for skill X.” It’s not only more clear and concise, but a lot less work for your team!

In cases of disagreement or controversy amongst the hiring team about who to hire, the quantitative nature of ratings makes the selection process much more objective. Teams can more easily understand a candidate’s strengths and weaknesses by looking at ratings rather than just relying on written feedback.

Once you’ve got ratings up and running, they provide a much greater level of consistency within the interview process -- not only across candidates, but also across interviewers, jobs, and your entire company. And, as in most cases, a consistent and repeatable process equals efficiency and effectiveness -- hiring the right people for the right jobs, more of the time.

After a short time, you can also start to take advantage of JobScore’s reporting features for ratings. The ratings reports can help identify individual interviewer “biases” amongst your hiring teams -- whether a person generally rates high or low, and how those ratings compare against the other team members -- as well as whether the ratings for certain skills are helping or hindering the hiring process. Reporting can also help management key in on which interviewers and which ratings are indicators of future success by comparing candidates’ initial assessments with their subsequent job performance.

Ratings can even help with the onboarding process for new hires. Understanding a candidate’s strengths and weaknesses more accurately -- as well as how the various members of the hiring team feel about the candidate (i.e. who is “in their corner” and who is not) -- will help the hiring manager make sure the candidate is successful in their new job!

How Do I Get Started with Ratings?

In order to make your use of ratings as successful as possible, here’s a few recommendations on how to get started…

Company Culture & Global Ratings

It’s probably a good idea to gather some feedback across your organization about what candidate attributes are important for ALL jobs. These will likely relate in some way to your company culture and, since they will be assessed for all job openings, should be appropriate for “global” use. For instance, your list of global ratings might include something like “handling adversity,” but will NOT likely include something like “C++ programming.” It’s also a good idea to keep this list short and sweet -- find out what key attributes really matter to everyone in the company, and try to build consensus if it’s not already there.

Start Small, Start Simple

Don’t try to “boil the ocean” by setting up ALL ratings for ALL jobs across ALL hiring teams. Start small! Add a simple “overall” rating to your Interview Templates, just to get folks familiar and comfortable with using ratings. Then pick a high-volume position that you can get more granular with, and work with the hiring manager to determine what specific ratings are relevant to that job, including any global ratings you’ve already identified. And as with global ratings, be careful not to make the list too long -- you don’t want this to become a “checklist” of job skills, but rather a concise and accurate assessment of how a candidate measures up to the essential requirements of the position. Once you’ve got ratings set up for one job, wait a while and see how things go. Most likely you’ll want to adjust your strategy at least once or twice until it’s dialed in. Then you can start thinking about rolling it out across other positions...

Set Up Your Ratings & Templates

In order to get ratings up and running across your company, you’ll need to (1) set up the various types of ratings you’ll be using, (2) create the necessary Interview Templates for each job and/or stage of the interview process, and then (3) link the appropriate ratings to each template. Click here to learn how this is done. While this is likely going to require significant upfront effort, don’t worry -- the time you invest will be rewarded! As ratings begin to roll out, you should very quickly start to see that your interview feedback is more objective and consistent, leading to easier and more constructive hiring decisions.

Here’s a few other things to consider when setting up ratings and templates:

Individual rating types should be able to be summarized succinctly in one or two words and then further described in a sentence or two that’s understandable to all users. That way everyone on the hiring team is on the same page, using the same assessment criteria.

You can use ratings as simply or as complex as you like, but consider setting up unique templates not only for each specific job but also for each stage of the interview process. You may be scripting each stage of the process already, so adding ratings to each template is just an extension of that.

Give your team (and yourself) permission to be wrong! Don’t think of your ratings and templates setup as an end state, but rather an ongoing and improving process. Once you build up enough data to review how you’re doing (via the ratings reports), you may see that the ratings and templates you set up aren’t quite doing the job as well you’d like. That’s OK! Just adjust along the way until it’s up to snuff.

Reporting

We recommend that you start off looking at the ratings reports once a month to determine how things are going, and then maybe switch to once a quarter when you’ve got it dialed in. Here’s a few questions to ask when reviewing the reports:

Are you using the right ratings for each job and for each stage of the interview process? Are any of the ratings significantly helping or hindering the hiring process?

Are you seeing any significant biases from user to user, i.e. are individuals rating very high or very low, when compared to others on the same hiring team for the same jobs?

How do your ratings compare to the actual performance of candidates once hired? Are any of the individual interviewers and/or specific ratings key indicators of a candidate’s future success?

Bake it In to the Process

Maybe the most important thing you can do to ensure success with ratings is to “bake it in” to the process, not only when opening a new job requisition, but also when interviewers enter feedback. Whenever anyone adds a new job in JobScore, make sure they know to also create the appropriate templates and ratings. This should become standard operating procedure for adding a job, again resulting in consistency across your company. And once you start to roll out ratings to your hiring teams, make sure everyone is fully trained on how to use ratings and how they work. It’s important that they not only understand the mechanics of how to enter a rating, but that they’re also familiar with what the individual rating types and the ratings themselves mean. Considering the example mentioned above, everyone in your organization should be on the same page regarding what the “handling adversity” rating is meant to assess, as well as what each rating in the scale represents (each icon or each numeric rating from 1 to 4).