In episode 32 of the Hire Up Podcast, John interviews guest, Chris Kunze. Chris is a multi-lingual assessment specialist with over 15 years of experience analyzing psychometric and workforce data. Assisting hundreds of companies all over the globe implement HR software, Kunze has used statistical trend line analysis to translate assessment results into the language of employee job performance.
As a Phi Beta Kappa graduate of Marquette University with a liberal Arts degree, Kunze also completed studies in Philosophy at the Gregorian University in Rome, Italy. He was a recruiter throughout Europe and worked three years as a student counselor at the Albertus Magnus University in Cologne and the Heinrich Heine University in Duesseldorf, Germany.
President of Kunze Analytics, LLC, Chris recently worked four years in R&D at John Wiley & Sons’ Workplace Learning Solutions Division. While there, he leveraged governmental workforce and client data to publish a library of 139 job-related performance models for PXT Select™ Certified Partners and their clients. He is known for calculating ROI using the venerable BCG Utility Formula. Finally, Kunze enjoys the challenge of optimizing performance model validity by writing software code in SPSS Syntax and Excel Visual Basic languages.
KUNZE ANALYTICS INTRODUCTION
Hello. This is Chris at Kunze Analytics. Thanks for watching our video.
We’d like to tell you who we are as a company and what we do. Kunze Analytics is based upon over 15 years in the testing industry and what we’re great at is looking at companies’ key performance indicators and seeing if the assessment data, psychometric, survey data they’ve captured over the years for workshops, training sessions, and even benchmarking their employees to try to find candidates that fit well for open positions… We marry those data those data and correlate them with your key performance indicators. So, you’d like to know if any of those data are very beneficial and can be used for forecasting impact. That’s what we do. We’ll tell you not only what factors are [impactful] but we’ll translate the data into tables that are in your metrics of interest. For example, these could be sales revenue, profit margin, average days in tenure of employment (financial and non-financial metrics). Our website is KunzeAnalytics.com. We’re in the Madison, Wisconsin area and service southern Wisconsin and northern Illinois, but now with technology you could be anywhere in the world. Give us a call and thanks for watching this video.
EXECUTIVE SUMMARY ELEMENTS
Hello. This is Chris with Kunze Analytics. Thanks for watching our video.
Today we’d like to talk about our Executive Summary report and the elements that are involved in that document. We know that you as a busy Executive, President, CEO, CFO of your company look at of information every day. So right off the bat, at the beginning of our report, we’ll tell you which were the key performance metrics or indicators that you asked us to analyze.
Next, we will tell you statistically what would be a rational range of weight that you could put on the assessment, psychometric, survey data that we’ve analyzed. Sometimes companies or consultants will say you should put so much weight on the information for screening in candidates to give them employment, promote them or train them in some group, but we will tell you statistically what would be the recommended range.
Next, we will interpret the data using the language of the US Department of Labor. That also helps to reduce some subjectivity. As a result of our analysis, we’ll be able to tell you that for every percent increase in fit score of the assessment or psychometric tool or survey, you should expect “x” amount of dollars which would be our best, good faith estimate or forecast for impact on your bottom line.
And then finally what we’ll look at will be which of the competencies that you had us analyze are truly critical success factors and [on the other hand] those that really less informative than the rest. So, thanks for looking at our video. Give us a call. We’d love to work with you.
Hello. This is Chris with Kunze Analytics. Thanks for watching this video.
Today, we’d like to talk to you about the deliverables that we give you after analyzing your data. There are six different objects, so I’d like to show them to you.
First of all, we build a correlation table that summarized the evidence for relationships between your key performance indicators, metrics and any psychometric, survey, assessment data.
Next, if a performance model was used in scoring the assessment data, we’ll look at the critical success factors, which scales are important, and if a candidate or a group candidates are in a risky area, we call that a red flag zone. And so, we will mark that out.
We’ll also give you a detailed output of the regression analysis so that you can see the prediction equation.
Next, we’ll build a contingency table (a what-if table) leveraging the formula using the slope and intercept.
As a fifth element, we’ll go back to the different levels of performance in your organization: the above average, average, and below average performance and create a table that looks at the mean key performance indicators you’ve asked us to analyze.
And then finally, we use a venerable equation that has over 80 years of use in different derivations, the Brodgen-Cronbach-Gleser Utility Equation to estimate return on investment.
We’d love to do business with you. Give us a call.
RELIABILITY & VALIDITY
Hello. This is Chris with Kunze Analytics. Today, I’d like to talk about a topic that’s found in almost every technical manual of assessments. The topic is reliability and validity. I’d like to explain how they’re related and how they link to out Executive Summary.
I’ve got my glasses on because we are going to be reading some numbers. So first, let’s talk about reliability. As you know, the US Department of Labor tells us that a good and adequate reliability for assessments starts at .70. You’ll find that only a few scales would be in the excellent range above .9. Some of them might be in the .8 range and above, but let’s think of an assessment that has an average of .76. So, if we are to square that number, that reliability coefficient, we would come up with .58. That’s reliability squared.
Now if we look at validity, .58 which is the reliability [coefficient] squared becomes the validity ceiling, meaning it’s the high-bound value. We’ll never see a correlation coefficient above .58 for an assessment whose reliability is .76 on average. Then you’ll remember that the government says for validity, usage of an assessment, we’re looking for assessments that have a validity coefficient of about .35. So, the upper bound [value] would be .58 [the range from .35 to .58]. Now, if in our first equation we look at reliability and we get the upper ceiling for validity, in our second equation we actually square the validity coefficient, .35, and that’s where we get the one third. That’s the personnel decision weight, the highest that we could achieve. So, in summary, we put that in our executive report and it’s the second bullet point. Thanks so much for watching.
SOME EVIDENCE OF IMPACT
Hello. This is Chris with Kunze Analytics. Today I’d like to talk about a topic: it’s evidence of impact.
There are two main strategies: First of all, we’d like to consider hiring predictive accuracy; and the second strategy is noteworthy performance.
So, let’s begin by focusing on hiring predictive accuracy. There’s a statistic, an equation called tau, which allows us to understand before we undergo any improvement initiative or implementing a performance model to increase hiring accuracy in an organization, we could calculate the tau statistic. How does this work? Let’s say we have five different roles within an organization (this is an educational institution): Financial Aid Officer, Admissions Representative, Student Placement Officer, Administrative Assistants, and Campus Presidents. You can see that we have assessed a number of people over time and have hired a group also. So, the question becomes, if we hadn’t used the assessment at all, how many people could have been successful in that position just due to random chance. So, the tau statistic gives us that number. In those five positions by random chance, we would have probably had eleven successful hires and after an audit of performance in this case we had 30. So, the improvement in hiring is 18%. That’s the first strategy to measuring hiring accuracy. I found this in a book called Discriminant Analysis by William Klecka.
Now let’s talk about noteworthy performance. I propose that as a second strategy we look at what are called behavioral control charts. There are two types of charts. The first one measures the individual key performance indicator or performance statistic. We’ll look at either the average, the mean, or the median over a period of time. We can also then focus on the variability and when significant changes occur, we’re able to note those using certain rules for distinguishing between statistical noise and our signals. Books that are really helpful to explain statistical control charts, behavioral control charts, are Donald J. Wheeler’s book, Understanding Variation, and then also Stacey Barr in her Practical Performance Measurement book. Thanks for watching.