FAQs

Frequently Asked Questions about using Employment Tests

Q. How does testing compare to other selection methods?

A. Tests are one of the most effective ways to measure the skills, abilities and attitudes that are important for job success. Research has shown that ability testing is more effective than nearly any other method, including the interview, training and experience ratings, or academic achievement measures in predicting job success. Work attitude measures can also be powerful predictors of job performance, employee fit and retention. Compared to other approaches, tests represent one of the most valid, cost effective and efficient forms of assessment.

Validity of Alternative Selection Procedures

Validity of Alternative Selection Procedures

Q. How should tests be used?

A. Tests are an important part of an effective personnel selection system: they are one of the most effective ways we can measure certain abilities and attitudes that are critical to success on jobs. Information from tests should be combined with other sources of information about candidates, such as work history, interview data and references. By combining information from these various sources, employers equip themselves well to make selection, placement and employee development decisions.

Q. Can I use both ability and attitude measures?

A. Yes. Combining ability and attitude measures can maximize the overall validity of your selection process. Research indicates that the highest overall validity is achieved by using both. Adding an attitude measure to a battery of cognitive measures can mitigate adverse impact.

Q. How do I select the appropriate tests?

A. Typically, an appropriate test battery is selected following a review of job information. Tests are selected which measure abilities and characteristics identified as important for the job and which have been shown to predict job performance or training success.

In selecting tests for your battery, be sure that the chosen tests measure skills, abilities or attitudes that are important in the job. You can select tests based on considerations such as similarity to the content of the job, testing time, coverage of a wide range of jobs for program efficiency, or tailoring test batteries to differentiate among job families. A test battery may consist of any number of tests; however, the use of more than four ability tests, along with an attitude measure, rarely increases predictive validity in any practical sense. In many situations it is suggested that a battery of four abilities tests and an attitude measure be considered.

PSI has developed several suggested test batteries that you can use, or you may customize your own battery. A PSI representative is available to discuss these options with you.

Q. How did PSI select the suggested test batteries?

A. On the basis of job analysis and validation work, PSI developed test batteries that are predictive of job performance and measure required abilities and attitudes in the context of tasks performed on the job.

Q. How should I interpret the test scores?

A. PSI most often suggests that test scores be interpreted by using appropriate norm tables to identify candidates’ performance levels. Test information may be used as an initial screen or integrated with other information gained during the selection process and the hiring decision made on the basis of all information available.

Q. How do I select the appropriate norms?

A. The validation and norming studies for PSI tests include a broad range of jobs or job classes. These jobs are grouped into occupational families based upon careful analysis of the jobs in question. To determine the appropriate norm, select the table that best matches your target job.

Q. Can you give me some information that will help me to interpret the norm tables?

A. Norm tables enable you to compare your candidate’s test performance to that of a sample of individuals who perform or have applied for similar work. A percentile indicates the individual’s relative position in a group. If your candidate’s test score is at the 95th percentile, their score exceeds 95% of the norm group.

One technique for using this information that has proven to be extremely effective for many of our clients is “banding.” Percentiles are grouped into three or four “action bands” which help to guide the hiring decision maker. Individuals whose scores fall in the top bands are predicted to have the highest potential for job success and are given top hiring priority. . Individuals in the middle bands are given next hiring priority, being more carefully scrutinized and having greater attention spent exploring offsetting skills and abilities that may indicate success in job performance. Individuals in the lower bands would not ordinarily be considered for employment unless assessments of work history, background or experience reveal outstanding qualities that warrant more careful review.

Q. Should I use cut-off scores?

A. PSI does not recommend the use of rigid cut scores without conducting an in-depth analysis of our client’s hiring process. Setting a cut score is a management decision that must reflect job requirements, the company’s ability to attract qualified applicants, and the relationship between test scores and job requirements. Among other factors it may be important to consider the availability of qualified applicants, the company’s willingness to mount a substantial recruiting effort, the compensation and benefits programs available, and the organization’s EEO posture.

Q. How should we make the final selection decision?

A. The final decision to hire a candidate is best made on the basis of a variety of sources of information. In all cases, hiring decisions should be based on the best information available. This should include test scores and other types of job-related information about the candidates. Test scores alone may be used to exclude applicants from consideration; for example, in a successive hurdles selection model where candidates must successfully pass a test battery in order to proceed to an interview assessment. When used in this way, employers should be prepared to demonstrate that the tests are job-related for the position in question. PSI tests are professionally developed and supported by validity evidence that generalizes to a wide range of jobs. Contact a PSI representative about options for local validation of tests for specific positions in your organization.

Q. What is adverse impact?

A. It is not unusual to observe some differences in average ability test scores for various ethnic, gender or age groups. Differences in “average scores” are also often observed for measures other than ability tests, such as structured interviews, reference checks, experience requirements, educational backgrounds, etc. Sometimes these differences may be attributed to actual differences in the developed abilities and job preparedness of the applicant groups under consideration. Generally there is a substantial overlap in ability test score distributions for various groups. Noncognitive attitude tests are typically neutral with respect to racial/ethnic and gender group performance; i.e., average scores tend to be comparable among groups.

Q. Do tests contribute to adverse impact?

A. It is not unusual to observe some differences in average ability test scores for various ethnic, gender or age groups. Differences in “average scores” are also often observed for measures other than ability tests, such as structured interviews, reference checks, experience requirements, educational backgrounds, etc. Sometimes these differences may be attributed to actual differences in the developed abilities and job preparedness of the applicant groups under consideration. Generally there is a substantial overlap in ability test score distributions for various groups. Noncognitive attitude tests are typically neutral with respect to racial/ethnic and gender group performance; i.e., average scores tend to be comparable among groups.

Q. How can tests be an effective part of the selection system and avoid adverse impact?

A. PSI recommends that employers maintain a highly qualified and diverse workforce. Properly utilized, cognitive ability tests can materially assist employers in achieving these objectives. If there is no adverse impact in the selection system, employers are generally not called upon to defend the job-relatedness of their system. When test information is integrated with other candidate data prior to making a selection decision, there is no adverse impact associated with the tests because the tests are not the sole criterion for selection. Rather, tests are one aspect of the selection process, all of which are integrated when making the hiring choice. If an employer’s overall selection system results in a diverse and representative workforce, with no adverse impact in the overall selection decision, there will generally be no requirement to determine the adverse impact of each selection component.

Q. If a test or other selection procedure results in adverse impact, what must be done?

A. The presence of adverse impact triggers the requirement that employers demonstrate that their selection procedures are job-related and valid. Eliminating tests will not eliminate adverse impact if groups truly vary in ability, and ability is reliably considered in selection. If adverse impact exists, regardless of the procedure used, the employer may be required to demonstrate that the system is valid, reliable and fair. This is often a time consuming and expensive process. Tests have an advantage over other selection techniques in that you can more easily transport validity from setting to setting. For many of PSI’s test series, transportability studies are available which allow employers to link-up to previous validation research.

Q. How can adverse impact be reduced?

A. Depending on applicant pool characteristics, it may be possible for employers to use tests without incurring adverse impact, even when substantial differences in test score distributions among various groups exist. In some cases this may require changing the “passing standard” associated with the test. In other cases, there may be no “passing standard” set. Instead, additional sources of information (e.g., reference checks and interview results) may be considered with test scores and hiring decisions (hire vs. don’t hire or reject vs. keep for further consideration) may be based on a composite of information that may show less adverse impact or no adverse impact at all. By establishing broad groups of qualified candidates, the pool of eligible minorities is enlarged, and other job relevant factors can be taken into account in determining who is screened.

The use of attitude measures which typically have little or no adverse impact can also be used to help mitigate adverse impact. Other measures that may help employers to achieve a balanced workforce include targeted recruiting, outreach recruiting, test preparation and remediation programs.

Q. Is it necessary for each candidate to go through the entire selection process in the same way?

A. No. Selection is an information gathering process. As you gather information about candidates, some will clearly be better qualified than others. Those better qualified should receive the bulk of the attention in the selection process as the hiring authority continues to probe and research candidates’ backgrounds and experience to determine if there is a proper fit with the organization. Individuals who do not have sufficient experience, expertise, or job skills can be less actively pursued, although you may find that there is a fit with other jobs within the organization.

Q. Do I need to interview every job candidate?

A. Although not required, PSI suggests that, when feasible, clients conduct a personal interview with each candidate. For some candidates, that interview will be an in-depth discussion to explore and refine the information available so that a hiring choice can be made. For other candidates, the interview can be more cursory, during which the candidates are given the opportunity to present other qualifications, gather information about the organization and, perhaps, explore other job opportunities.

Q. Are there guidelines about how to administer tests?

A. It is very important that tests be administered in a standardized fashion for reliability and validity. It is not appropriate to modify the test administration and scoring procedures. Test administration policies and procedures must be standardized and consistent to ensure that the resulting scores are fair, reliable and accurate.

PSI tests are designed to be easy to administer and score. Tests may be administered on an individual basis or to groups. It is critical that test time limits and instructions be followed as described in the User’s Manual.

Certain ADA accommodations may be made so long as they do not undermine the purpose of the test.

Q. What accommodations in testing must be made to comply with ADA requirements?

A. According to the Americans with Disabilities Act (ADA), any employer conducting pre-employment testing should offer the tests in a place and manner accessible to persons with disabilities. This means making all “reasonable accommodations,” including providing appropriate test forms, e.g., audio taped, Braille, or large print examinations. Tests for persons with disabilities must also be offered in equally convenient locations and as often as other tests. It is best that reasonable accommodations for testing be decided upon by the employer and the applicant on a case-by-case basis. PSI offers a free booklet entitled, “Accommodating Employment Testing to the Needs of Individuals with Disabilities” (1994) that more completely describes procedures that may be used to accommodate testing to the needs of individuals with disabilities.

Q. Is disclosure of test scores required?

A. PSI does not recommend disclosing test scores to examinees, unless required by state law.

Q. Who should have access to test scores?

A. Test scores are confidential and should be kept secured and separate from personnel records. We do not recommend releasing test scores to individuals outside of the Human Resources department. If HR representatives are referring candidates to operating department heads, they should discuss the interpretation of the test score — “this candidate has strong computational skills,” rather than release the test score.

Q. What type of retesting policy should I implement?

A. Because the general cognitive abilities and noncognitive attitudes measured by PSI’s tests are generally stable, there is typically no advantage to retesting candidates within a short time period. Clients who adopt a retesting policy often require a waiting period of 3 to 6 months before allowing the candidate to take the same tests. There may be circumstances that require more immediate retesting, for example, if the test administration procedures were not followed. If it is necessary to retest prior to a lengthy waiting period, it is preferable that an alternate test form be used, if available. Generally, test policies should limit the number of retests that are allowed. At some point, after many administrations of the same test, the test will lose its effectiveness as a measure of ability.

Q. Have PSI tests been approved by the EEOC?

A. The Equal Employment Opportunity Commission (EEOC) does not approve tests. Instead, it has established guidelines with respect to validity and fairness that all employment tests are expected to meet. PSI’s tests have been developed and validated in accordance with professional standards and principles, as well as legal Guidelines. Because the development process has focused on the job-relatedness of the tests, the process of demonstrating compliance with the Guidelines for these tests is relatively straightforward

Q. What validation procedures are recognized by the courts?

A. Validity is the benchmark of any test’s usefulness. Several validation strategies are outlined in the Uniform Guidelines on Employee Selection Procedures, including content validity, criterion-related validity, construct validity, and transportability.

Content validity refers to the relationship of test content to job content. Content validity is achieved by a combination of job analysis, evaluation of test content for job relevance, and evaluation of statistical properties of the test. Criterion-related validity is established by showing how strongly a particular hiring method (or predictor) relates to job performance. The resulting coefficient indicates the strength of that relationship. Validity coefficients can range between 0.0 and 1.0. Industrial psychologists have determined that, in most cases, a validity coefficient above .20 is acceptable and a coefficient above .30 is good depending on sample size. Construct validity involves showing that tests are an acceptable measure of an attribute, such as mechanical aptitude, that is also found to be important to job performance. This demonstration usually requires criterion-related validation. Transportability involves using the results of a criterion-related validation study which has been conducted elsewhere to support the validity of a test in a new setting. The procedure involves careful documentation to show that jobs in the new setting are substantially similar to the jobs studied in the original validation study


If you would like more information, or have additional questions, call PSI’s Test Publication Client Services at: 1-800-367-1565. PSI offers consulting services to assist you with all phases of your employee selection process.

References

American Educational Research Association, American Psychological Association, National Council on Measurement in Education, Joint Committee (1999). Standards for educational and psychological testing. Washington, D.C.: American Psychological Association, Inc.

McDaniel, M.A, Whetzel, D.L., Schmidt, F.L., & Maurer, S.D (1994). The validity of employment interviews: A comprehensive review and meta-analysis. Journal of Applied Psychology, 79(4), 599-616

Ones, D.S, Viswesvaran, C., & Schmidt, F.L. (1993). Comprehensive meta-analysis of integrity test validities findings and implications for personnel selection and theories of job performance. Journal of Applied Psychology, 78(4), 679-703.

Schmidt, F. L., & Hunter, J. E. (1998). The validity and utility of selection methods in personnel psychology: Practical and theoretical implications of 85 years of research findings. Psychological Bulletin, 124, 262-274.

Society for Industrial and Organizational Psychology, Inc. (2003). Principles for the validation and use of personnel selection procedures. (Fourth edition) Bowling Green, OH: Author

U.S. Equal Employment Opportunity Commission, U.S. Civil Service Commission, U.S. Department of Labor, & U.S. Department of Justice (1978). Uniform guidelines on employee selection procedures. Federal Register, 43, (166), 38295-38309.