When companies select an assessment for prehire screening of job applicants, they typically have some knowledgeable person download a performance model from the test publisher’s library of patterns or they create a concurrent performance model using assessment results from a representative group of incumbents.
The decision about whether a single performance model will do the trick –so to speak—or more than one model is necessary can become complicated quite quickly. A thorough job analysis of the role in question should uncover the following:
1. Is there a single job description?
2. Are there different levels of experience or different compensation plans?
3. Do incumbents work in vastly distinct geographical regions?
4. Do they speak different languages?
5. Is the role normally compartmentalized by:
a. Market type
b. Product or Service
c. Purchase size
d. Contract type
e. Length of relationship with the client?
Appropriate statistical methods assist clearing the air when answers to these questions are clouded in doubt. One type of analysis is easily applied to situations when the outcome is having just one or two performance models. Another type of analysis should be applied when the result is likely to be several performance models.
Uncertainty abounds. Take for instance if a company seeking to build a performance model for five different regions wants to know how many models will be necessary. The correct reply could be from one (1) model up to five (5) models. If the optimal answer is two (2), three (3), or four (4) models, the question becomes how many regions will map to each performance model and which ones.
The following table demonstrates how ten (10) comparisons are necessary to explore the question completely regarding five (5) different regions.
Table 1
1) Can the Region #1 Model be used for Region #2?
2) Can the Region #1 Model be used for Region #3?
3) Can the Region #1 Model be used for Region #4?
4) Can the Region #1 Model be used for Region #5?
5) Can the Region #2 Model be used for Region #3?
6) Can the Region #2 Model be used for Region #4?
7) Can the Region #2 Model be used for Region #5?
8) Can the Region #3 Model be used for Region #4?
9) Can the Region #3 Model be used for Region #5?
10) Can the Region #4 Model be used for Region #5?
Regular maintenance of Performance Models has an expense to it. Time, effort, and money are allocated to evidence-based validation of each model. The payoff has the potential of increasing hiring quality in each region by 4, 5, 6, 7, or 8 percent, maybe more. This means hiring one more Top Performer out of each 25 hires to one more Top Performer out of each twelve (12) hires. The added value of hiring one more Top Performer is such that it pays for the entire assessment several times over. Therefore, using appropriate statistics to answer model quantity questions is not just a nice to have or under-appreciated add-on.
In situations where only a few candidates will be hired per year, answering these questions as correctly as possible can become a determining factor of the small enterprise’s profitability. Poor, inaccurate, roll the dice hiring harms smaller companies more quickly than larger ones.
Your statistician should know when to apply ANOVA, CHI-SQUARED analysis with a procedure for post hoc comparisons and dummy coding of variables with bi-variate OLS Regression. These are only the most common methods for analyzing psychometric data and specifically, performance models. Other more sophisticated approaches normally require larger sample sizes than are atypical for performance model validations.
Finally, it might be good to remember some examples of clients that began using a single performance model and then created more than one to leverage assessment data more powerfully. A US Circuit Court ended up creating a performance model for young Public Defenders and another for Experienced ones. A National Retailer went from a single Store Manager model to several based upon geographic zones and store size. A public-school district went from a Teacher performance model to different ones for elementary, middle, and high school Teacher.
Don’t just roll the dice. Ask for statistical insight from the test publisher’s number-savvy support personnel, from a colleague, a consultant, or from Kunze Analytics, LLC.
