Mobile Edition
Print
Get the Mag
Weekly UpdatesA wealth of empirical literature provides compelling support for collecting background information from candidates when making personnel selection decisions. For example, many studies indicate that biodata can be used to predict successful job performance, turnover, and success in completing training with criterion validity coefficients ranging between .20 and .60. (1) However, many researchers have also found that a selection tool should have positive qualities beyond test reliability and validity and that applicants' reactions may be equally important to consider when developing and evaluating a selection device. (2) For example, Smither and his colleagues (3) explained that applicants' negative reactions to a selection tool might damage the reputation of an organization, which may lead potential employees to search for jobs elsewhere. Furthermore, a selection tool that applicants perceive negatively is likely to give rise to complaints and possibly costly and time consuming appeals of personnel decisions. Last, unsatisfied candidates may intentionally perform below their ability during the assessment exercise when they are required to use a selection tool they view negatively, and this can diminish the validity and utility of the tool.
Gilliland (4) has presented such arguments against using standard HR selection tools. He has also highlighted the need to consider the ethical implications of the practice. He wrote that rejected candidates may experience problems with efficacy, esteem, and overall psychological well being as a result of taking a test that that they perceive negatively. He presented nine distributive and procedural attributes of an HR selection tool that should be considered (see Table 1). He clustered those rules into three categories--formal characteristics of the selection tool, explanation of process/tool, and interpersonal treatment--that characterize the components of candidates' judgment of the fairness of a given selection tool.
Perhaps the most important attribute of an HR selection tool is perceived job-relatedness. Gilliland cited several independent studies that indicated that perceptions of overall fairness were more positive when a selection tool was perceived to be job related. The cited studies also indicated that concrete items were perceived as being more job related than were abstract items. Thus, how job candidates perceive and react to a selection tool should not be underestimated when an organization creates and evaluates any selection instrument.
Although basing personnel selection decisions on candidates' background information has been found to be effective, there is unfortunately little information about how collecting such information is perceived by job candidates. (5) To complicate the picture further, existing research has produced inconclusive results. Kluger and Rothstein (6) concluded that collecting biodata outperformed general mental ability (GMA) assessments in being perceived as fair by job applicants. Biodata was seen as more fair than GMA because it incorporated job-relevant characteristics other than intellectual ability and, thus, provided a more holistic candidate profile. Kluger and Rothstein also reported a significant correlation between perceived job-relatedness and perception of fairness.
In contrast, Smither and colleagues (7) found that entry level managers perceived collecting biodata and other methods involving abstract items (e.g., personality measures) as having lower predictive validity than did methods that employed concrete measures (e.g., math problems, structured interviews, in basket activities). In fact, biodata was perceived as having the lowest predictive validity out of the 14 measures, with less than 45% of the entry-level managers in the Smither et al. survey indicating that collecting biodata would be a valid, job related method for selecting job candidates. Thus, to enhance candidates' perceptions of the usefulness of collecting background information, it appears that employers would do well to acknowledge the importance of how applicant perceive the procedural and distributive attributes of HR selection tools and apply recommendations from the justice literature (8) to their personnel selection process and ensure that the tools they use have the attributes identified by Gilliland.
A Novel Approach to Collecting Candidates' Background Information: The EVE Background Questionnaire
A new HR selection tool--the Education, Vocational Training, and Experience (EVE) background questionnaire--was recently introduced at a large public employer in the western United States. The EVE was developed in an attempt to make the gathering of background information more user friendly by relying on a systematic approach with clearly labeled parameters related to assessment criteria, item development, and item keying.
The questionnaire is a combined evaluation of a candidate's education, training, and experience that is hypothesized to provide a measure of general job competence. Following recommendations from the literature, (9) the EVE asks job candidates to provide only objective and verifiable data. Only looking at hard information was believed to yield highly accurate responses, increase job candidates' perception of procedural justice and the face validity of the selection tool, and produce fewer negative reactions from applicants and, therefore, fewer appeals of personnel decisions. Last, it is also believed that gathering candidates' background information in a generalizable, common format is likely to enhance the overall efficiency of the selection process.
Purpose of the Present Study and Hypotheses
The primary purpose of the study described here was to clarify whether using the EVE background questionnaire identified the same quality job candidates as a traditional written job knowledge test. We were also interested in assessing candidates' perceptions of the procedural justice and face validity of the EVE and comparing those perceptions to the candidates' perceptions of the procedural justice and face validity of the written job knowledge test. Last, we explored whether applicants for different types of jobs perceived the procedural justice and face validity of the two selection tools differently.
It was hypothesized that the EVE would be positively correlated with the written exam because a better prepared candidate (i.e., one with a higher level of education, more vocational training, and more experience in the field) would be likely to perform better when answering job related questions on the written exam. It was also hypothesized that candidates' overall perception of the face validity of the EVE would be higher than the overall perception of the face validity of the written exam. This finding was projected because the EVE only requests information in a few highly job-relevant areas, while the written exam includes numerous construct related items that a candidate could perceive as being unrelated to their actual ability to perform the job.
Another hypothesized advantage of the EVE instrument is that it would allow candidates to provide a self assessment in an uncomplicated and less-intimidating way than the written exam, which would translate into increased perceptions of the EVE's procedural justice. However, due to the differences in the two jobs that were studied, we also explored whether differences existed in the perceptions of the face validity and procedural justice of the EVE and the written exam between applicants for an engineering aide position and applicants for a plumber position.
Method
Participants
Data from two employment exams that were administrated by a large public organization in the western United States were used for this study. Participants were job applicants for engineering aide (N = 52) and plumber (N = 72) positions that were open to both the general public and current employees. Since the participants were competing for regular employment, only job-related information was collected to avoid legal problems. Therefore, demographic data were not collected for the study.
Measures
Three measures were used in this study--the EVE background questionnaire, a perceptions of procedural justice and face validity questionnaire, and the job knowledge employment tests designed to assess core competencies.
The EVE Background Questionnaire
The EVE generates a score for each applicant based on how the applicant matches up with predetermined assessment criteria. The score is hypothesized to indicate a candidate's combined quantity and quality of qualifications for the job. The EVE follows a common format that quantifies candidates' educational credentials, vocational training, and previous job related experience. Although some items are identical for each version of the questionnaire (e.g., "What type of degree have you earned from an accredited U.S. school or university?"), most items are customized to collect the unique information that is relevant to a specific job. Thus, the two EVE instruments used in this study were very similar in structure, but also somewhat dissimilar in content.
Questions for each of the EVE questionnaires were based on multiple sources of job relevant information, including archival data (e.g., job analyses, current job description) and statistical information (i.e., subject matter experts' ratings of critical tasks and the frequency of task performance). Items were only included in the questionnaire if they were deemed objective and verifiable, appeared face valid, and judged to be critically necessary to the function of a given position.
The EVE questionnaire uses a rational keying approach, as described by Mael and Hirsch. (10) Each item is multiple-choice, with rational weights assigned to each answer based on criticalness and conceptual relevance (e.g., A = 1, B = 0.75, C = 0.5, D = 0.25, E = 0). For EVE items assessing previous experience, the maximum score is set at five years because job proficiency as a function of job experience reaches its peak at this point and then tends to plateau. (11)
FEED