2017
2016
2015
2014

Category Archives: PSI Blog

A Better Situational Judgment Test

Situational Judgment Test picWhen you’re developing new testing products, sometimes it helps to think about the history of tennis racquets.  Any time before the 1970s, tennis racquet heads had about half the surface area of today’s racquets.  Not surprisingly, this made playing tennis a lot harder.  In 1976, the appropriately named sports entrepreneur Howard Head introduced a racquet with a larger head.  This, obviously, made playing tennis a lot easier, so almost immediately every manufacturer followed.

Why then, for decade after decade, did all manufacturers make the hitting surface of their racquets so small?  It turns out there wasn’t a reason, not a good one anyway.  It was simply that, in their minds, this is how things were done.

There are many examples of this kind of locked mindset in almost every industry.  Whenever PSI develops a new product, we try to think like Howard Head, and PSI’s new True to Life™ situational judgment tests (SJTs) are a good example.  SJTs present job applicants with work-relevant situations, followed by a number of ways to respond.  Candidates indicate, in some way, how effective or ineffective those responses would be.   But it is innovations in the way these responses are translated into hiring recommendations that set PSI’s SJTs apart.

Let’s look at an example.  Imagine that two people, we’ll call them Jack and Jill, are being considered for a job that will require, among other things, that they go up a hill.   So to decide who to hire, we present these two young applicants with a list of different ways climb the hill, and ask them to rate how effective each of these methods would be.

When Jack or Jill are hired, we’d like them to perform like people who really know what they’re doing, so before we score their responses, we ask a group of job experts how effective each of the hill-climbing methods would be.   Now we have a standard:  The closer Jack or Jill’s responses are to the experts, the higher their score.

Situational Judgment Test pic 2So now let’s look at the figure showing the expert ratings and the answers provided by Jack and Jill.  For the first response, “Drive the Jeep”, Jack’s rating is one away from the experts, but Jill’s is two away, so Jack scores somewhat better on that one.  Adding up the differences like this across all three hill climbing options, Jack is a total of four spots away from the experts, and Jill is six spots away.  Conclusion, we hire Jack, because overall he’s closer to the experts.  This all seems very reasonable.   And, after all, for situational judgment tests, it’s how things are done.

But now let’s step back and think like Howard Head.  What are we actually trying to accomplish here?  The answer, for any employment test, is that we want to have some idea of what a job applicant will actually do on the job after he or she is hired.  So now let’s imagine that Jack, Jill, and the experts are all on the job together facing a range of hill-climbing situations.  In the first situation, they have all of the three options mentioned available to them.  Jill and the experts board the ski lift together, because they each rated that option as better than the other two, but Jack drives up alone in the Jeep, his highest-rated option.  If the ski lift is broken, everyone rides up together in the Jeep, but if the Jeep is broken, Jack again breaks from the group and walks up the hill, because he rated that option as better than taking the ski lift, the option everyone else uses.

You’ll notice something here — Jill is acting like the experts, but Jack often isn’t.  In fact, for these particular climbing approaches, no matter what happens in terms of the availability of the different methods, Jill and the experts will always do exactly the same thing on the job.  This is because the pattern or shape of their ratings, represented by the blue and green lines in the figure, is identical.  Conclusion, definitely hire Jill.

PSI’s True-to-Life™ SJTs are perhaps unique among currently published SJTs in that we will recommend hiring Jill, even as our competitors will try to sell you on Jack.  Methods that look at similarity in the shape of different patterns of responses have been around for a very long time, and are widely used in other settings.  Historically, however, not in scoring SJTs.  This locked mindset has created an opening for PSI to, like Howard Head, offer a better way.


Victor Jockin Victor Jockin, Ph.D., is Senior Manager of Assessment Solutions at PSI.

Click here to receive more information about PSI’s assessment solutions.

The Tipping Point For Automated Leadership Assessment

Tipping Point blog photoIdentifying and developing leadership talent remains an important challenge for all organizations.  Large organizations face challenges associated with making leadership assessment “scalable,” while smaller organizations face challenges in deploying high-quality leadership assessments with limited available resources.

Over the past decade or so, advances in technology and research have led to the development of accessible and affordable automated assessment tools and reports that provide valuable insight into individual and organizational leadership talent strengths and gaps.  However, many organizations have not adopted these solutions and some might argue that, with apologies to Malcolm Gladwell, the tipping point has not yet been reached in automated leadership assessment.

Why assess? Effective assessment of an organization’s “bench strength” and identification of high potential employees is central to succession planning and leadership development programs. Assessment instruments provide an efficient and effective means of identifying employees who possess important qualities underlying the potential to develop into future leaders in the organization. They can also provide valuable feedback to prospective candidates for leadership development programs with respect to areas for growth and development. To optimize the effectiveness of the assessment, it should be aligned with competencies that are crucial for leader success in the organization.

An Affordable and Scalable Solution.  Technology-based leadership assessments have demonstrated value by automating features to increase accessibility and cost-efficiency, such as:

  • Internet-based delivery for a range of remote testing and proctoring models;
  • Automated competency-based scoring approaches that capture expert interpretation;
  • Dynamic computer-generated narrative feedback based on algorithms that simulate a psychologist’s hand-written report;
  • Automated reporting and report distribution; and
  • Data management and integration with enterprise systems for talent and learning management.

With the availability of these technology-enhanced capabilities, one could argue that we have indeed passed the tipping point for accessible leadership assessments. The power of automated leadership talent insight is available to organizations now more than ever.


 John Weiner is Executive Vice President & Chief Science Officer at PSI.  Contact him at jweiner@psionline.com or by phone at 818.847.6180.

Click here for more information about PSI’s comprehensive leadership solutions.

 

Was That Really Good Advice?

adviceHuman Resources leaders are often faced with the challenge of having to quickly acquire extensive knowledge and expertise across a wide array of disciplines.  Given the significant impact of their decisions, HR executives may at times seek information from external consultants in order to get the expertise they need to consistently make the best decisions.  As topics become more technical in nature and the impact of decisions more extensive, reliance on good advice becomes ever more critical.

Talent Assessment and Selection is one such area that requires deep technical expertise, and whose impact can range from enormously favorable to catastrophic.  A good selection process can help your organization hire salespeople who close more business, or customer service representatives who delight your clients.  Whereas, an ineffective selection process can just as easily fill your company with poor performers, wreak havoc on your culture, and reduce your company earnings via the high cost of turnover.

In a very real sense, the quality of the advice you get in designing your assessment and selection processes can mean the difference between growing your business, and irrevocably harming it.  So, how do you know the advice and tools that you’re getting are any good?

Two words – Validity & Reliability.

In their 84-page guide, Testing and Assessment: An Employer’s Guide to Good Practices, the United States Department of Labor outlines the qualities of a good assessment.  At the top of their list?  You guessed it – A VALID & RELIABLE ASSESSMENT.

When a test is valid and reliable, it accurately and consistently measures what it claims to measure.  The greater the validity, the more accurate the assessment.  By using assessments with proven validity, you become better equipped to identify new hires who’ll excel in your organization – new hires who get up to speed faster, perform better, and stay longer – increasing the productivity and operational efficiency needed to drive both top and bottom line growth.

Human Resources leaders face many challenges – designing a good selection process doesn’t need to be one of them.  The next time someone recommends an assessment, just ask them one simple question: What’s the validity and reliability evidence?  Then see how their evidence stacks up to the DOL recommendations for what constitutes adequate, good and excellent levels of reliability and validity.  Now that’s advice you can use.


Brad Schneider, Ph.D.
Brad Schneider, Ph.D., is Senior Vice President of Talent Assessment at PSI. Contact him at bschneider@psionline.com or by phone at 818.847.6180

Smaller Than You Think

Blog-1-pic
Recently, on a flight to Buenos Aires, I was reminded just how small the world really is.  Traveling by night, you close your eyes in Chicago, and when you awake, you’re there.  Even the time zone remains constant.  There are of course a few subtle differences with regard to language, food and culture.  However, the similarities far outweigh them.

Effective customer service, for example, may look a bit different from one country to the next, yet the desire to provide good service is universal.  Leadership approaches may vary from culture to culture, yet the ability to coach and develop others and raise the bar on talent is ubiquitous, and critical.

In a global economy, a core challenge faced by many organizations is the ability to ensure consistent quality across regions.  As companies compete for greater market share by extending their respective brands into less charted waters, maintaining one’s caliber of service becomes ever more challenging.  How can you ensure the quality of your offerings remains true throughout the world, and that the reputation you’ve built in the East unwaveringly transcends to the North, South and West?  The answer is, by ensuring global consistency in your quality of hire.

The approach to getting there is more familiar than you might think.  Identifying the characteristics needed for success in role, applying proven assessments to tap those characteristics, and using those assessment results to identify and train new hires on relevant performance gaps, is a formula for success.  Online testing, in particular, presents a proven and efficient means for ensuring consistent quality of hire throughout the globe.  Time zones even work to your advantage, empowering you to assess candidates continuously, as daylight hits a new country every hour of the day.

Ensuring the same exceptional quality and consistency of your brand throughout the world is a challenge inherent to the global economy.  Yet, by capitalizing on a globally consistent hiring process, you can ensure the same caliber of excellence, whether you’re in The Windy City or the City of Fair Winds.


Brad Schneider, Ph.D.
Brad Schneider, Ph.D., is Senior Vice President of Talent Assessment at PSI. Contact him at bschneider@psionline.com or by phone at 818.847.6180

Click here for more information about PSI’s assessment solutions.