Mobile Edition
Print
Get the Mag
Weekly UpdatesThe use of personality and biodata measures in safety force personnel selection is increasing due to demands to reduce adverse hiring, placement, and promotion decisions. The U.S. Department of Justice advocates the use of personality tests for this very reason. The selection of safety forces such as police is often contentious and is therefore one of the most likely instances in which the use of personality tests would be contemplated.
Personality testing for safety force personnel decision making raises at least two concerns, however. These are the coachability of personality tests, and the low predictive value personality measures have regarding police performance. (1) In a 2003 Public Personnel Management article, Barrett and colleagues highlighted practical issues that arise in the use of personality tests in police selection by exploring in depth exploration the meta-analytic finding that conscientiousness predicts job performance, focusing only on studies that had relevant samples, job performance criteria, and tests that correlated performance with conscientiousness. (2) They found it would be incorrect to assume that conscientiousness would be a useful benchmark for all police selection situations because the personality trait was not a significant predictor of performance. In other words, while research results showing that conscientiousness predicts police performance could be useful to guide theory, those results have limited practical application.
Although this conclusion might seem surprising, many researchers have suggested that meta-analytic results should not be taken at face value. Further, other researchers have specifically corroborated Barrett et al.'s finding. A meta-analysis of meta-analyses by Barrick and colleagues revealed that conscientiousness was not a significant predictor of police performance, as the lower credibility bound included zero. (3) Thus, although on average conscientiousness predicts police performance, it has not been strongly shown to do so in any one particular study. Also, the results of the studies are highly variable, and there are likely to be moderator variables that influence whether the relationship between conscientiousness and performance is significant in a particular sample. This latter observation would be true of any correlation study because the chance of finding a significant result for any given variables is less than one in three. In fact, there is almost a one in four chance of finding a negative correlation.
Moving beyond simple correlation, the study described in this article has performed to explore what might occur if some applicants received coaching on personality tests. The findings make the use of personality tests even less appealing.
Test developers who specialize in safety force testing have cautioned that safety force applicants expect to be able to prepare for important tests. (4) Even researchers who have suggested that faking is a minor or negligible problem have called for research investigating the effects of coaching. (5)
Training programs for civil service selection tests are common. (6) Workshops on "test strategy" that cost hundreds of dollars are routinely offered to applicants. In fact, developing training sessions for a test can be more lucrative than developing the test itself. In at least once instance, a consulting firms offered training seminars for a tests it had unsuccessfully bid to create and then later provided expert witnesses for a plaintiff who claimed the test was unfair. (7) There is such a demand for safety force test training that even the municipalities are offering programs such as situational interview preparation for police and fire promotion. (8) In the cited example, employees of the municipality conducted the interview prep, which consisted mainly of offering tips such as get plenty of rest, think before you speak (a period of silence is acceptable), make eye contact, and look your best. Role-play demonstrations were done in a training room that was set up exactly as the actual interview room would be, including a video camera that would be used to record candidates' interviews. Participants in the program did score higher on the subsequent situational interview than individuals who did not participate.
One might wonder what benefits a municipality might gain from such a program. Goodwill among employees is one. Safety force members want to study and prepare for exams, and the testing program was viewed positively by safety force members, the unions, and the community. (9) A second benefit would be meeting employees' needs cost-effectively. Safety force unions commonly encourage their members to participate in test preparation programs. Thus, if the municipality did not offer a training program, someone else would satisfy the demand. An analogous case would be that of the Educational Testing Service (ETS, which writes and administers the GRE and SAT). ETS provides test-taking software, past tests, and study guides for the GRE and SAT on its Web site. In addition, the company provides many of these materials with registration confirmations.
A report of a situation in Suffolk County, NY, describes an actual coaching situation that would likely be disconcerting to HR professionals. (10) In an attempt to increase the rate of minorities selected and fend off pressure from the U.S. Justice Department, the Suffolk County Police Department stopped using cognitive ability tests and instead used only biodata tests to select its 1988, 1992, and 1996 recruit classes. The biodata tests did not change much, which would make them susceptible to coaching effects. The top scores did in fact rise from year to year. The top scores in 1988 ranged from 88.2 to 95.5, and in 1992 the top scores ranged from 93.2 to 99.
When the police department could still not meet minority hiring goals, it began an unusual recruitment program. Forty-three minority applicants were offered part-time unpaid positions as cadets and given police clerk jobs. These cadets were promised officer positions at the end of 30 months of volunteering if they could score 70 or higher on the applicant test. In 1992 ordinary candidates had to score above 90 just to be considered. As the salary of a Suffolk County patrol officer is $70,000 per year, some of the candidates turned down college or law school admissions to enroll in the cadet program.
When it came near time to take the applicant test, most cadets protected their time investment by taking a preparation course given by a sergeant on the force at the time who had been involved in the validation of the instrument. Sgt. Brian Bugge had given courses to hundreds of past applicants for several hundred dollars, but e made an exception for the minority cadets and only charged $20. Bugge's training consisted of "hypothetical" questions and "preferred" answers such as
Q: How many of your relatives work in law enforcement?
A: Three.
Q: Which hobbies do you engage in at least once a year?
A: Hunting.
Again, the exam given in 1996 hadn't changed much from the versions given in 1988 and 1992, and many of the questions were versions of the "hypothetical" questions given in training. All of the cadets passed; several scored above 90, and some even earned scores of 100. The attitude of the cadets, that there is a right answer and providing it is not cheating, is not uncommon among applicants. The test used in Suffolk County has been used by hundreds of police forces across the nation, so it likely that other jurisdictions are facing situations similar to that in Suffolk County.
Despite suggestive research findings and at least one well-document case study that shows coaching could be a problem when personality-based tests are used for selecting police officers because it produces skewed outcomes, no published research addresses how coaching on personality tests effects hiring can be found.
This gap in the literature led the authors of this article to formulate and test the following hypotheses:
* Hypothesis 1: Participants who receive coaching on personality theory and personality test construction will achieve higher scores on the personality measures than a sample of participants who did not receive the training.
* Hypothesis 2: Participants who received personality test training will be more likely to be selected for the job than participants who did not receive the training.
Method
To test the hypotheses, the authors recruited a total of 246 participants took three tests designed to assess conscientiousness (NEO-FFI, Neo Five Factor Inventory; Hogan Personality Inventory, and the CBDQ, Conscientiousness Biodata Questionnaire) and a Social Desirability measure, commonly known as a lie scale (Marlowe-Crowne) either with or without coaching. All of the study participants were introductory psychology students at a large Midwestern university, and they received extra credit for their participation.
As freshmen who were beginning college, the participants had educational experiences similar to most police officers, as most police officers have some college experience. (11) It is also potentially significant that the study participants were enrolled in an introductory psychology courses since those courses typically include material on personality and personality measures. Sixty-six percent of the participants were female, and 13.5% of the participants reported themselves to be of African American descent. Two percent were Asian, 79.2% were Caucasian, 2% were Hispanic, 0.3% were Native American, and 2.6% classified themselves as Other. The mean age of the participants was 20.65 years, while the mean years of education completed was 12.30. Ninety-six and nine-tenths percent of the sample had work experience, and the mean work experience for the entire sample was 5.08 years.
FEED