Mobile Edition
Print
Get the Mag
Weekly UpdatesFor the past 30 years researchers and practitioners have relied on the same job analysis methods for their work. Most often, the projects begin with interviews of subject matter expert (SME) groups and conclude with task- or competency-based surveys. Regularly, the resulting product is a report consisting of 20 pages of text and 200 pages of tables. With the exception of two noteworthy advances, little has changed in job analysis since the 1970s.
The first advance has been competency-based job analysis. (1) There is some controversy, however, over how competencies and the traditionally used knowledge, skills, abilities and other characteristics differ. (2) The second important advance has been conducting Web-based surveys.
Even though there has been little change in job analysis, the tried-and-true methods have proven effective for characterizing jobs. Some would say, "If it's not broke, don't fix it." However, advances in data analysis and visualization techniques offer opportunities to incorporate new tools into job analysis and to build upon the success of past practices. Data analysis techniques that incorporate visualization offer an opportunity to better communicate job analysis results, thus making study results more influential (and probably improving the efficiency with which study results are communicated). It is time for job analysts to look outside their insular world and adopt technological advances and embrace information visualization and knowledge visualization techniques.
Information visualization and knowledge visualization are young interdisciplinary fields that draw heavily from cognitive science, visual perception, and computer science. Information visualization is the representation of selected features or elements of abstract and complex data. Whereas information visualization requires the use of computer-supported tools to analyze large amounts of data, knowledge visualization involves the transfer of knowledge among persons. (3) Both methods allow researchers to present data or information in nontraditional forms, using, for example, 2-D or 3-D color graphics or animation to show the structure of information. Information visualization and knowledge visualization also allow data users to navigate through the collected data and modify its presentation to explore, discover, and learn. Although the disciplines are in their relative infancy, information visualization and knowledge visualization each offer tools and methodologies that may be well suited to job analysis research.
Network analysis is another tool that can be useful for job analysis. This is a burgeoning field of study, and its methods have been applied to such diverse topics as the analysis of the U.S. power grid, the relationships among movie actors, and neurobiology. (4) Network analysis is important because job analysts often want to examine the relationships among a number of jobs (or a network of jobs) and network techniques emphasize the relational aspects of data.
Network analysis, when used in conjunction with visualization techniques, gives a job analyst a visually efficient way to present complex data to end users. However, behind the simplicity lie highly sophisticated tools that facilitate the elegant presentation of data. Thus, the complexity of job analysis data is made understandable to the end users.
A picture is worth a thousand words" may be a cliche, but it is absolutely true. The human eye and mind are particularly well suited to interpret images, forms, and patterns. Simply put, presenting data visually allows viewers to grasp the interrelationships of the data points without performing or understanding complex mathematics. (5) In other words, it allows data users to easily sort through and understand large amounts of data quickly. While the physical and engineering sciences have dealt with increasing data complexity by using visualization techniques, the behavioral sciences have been slower to adopt such tools. (6) In this article, we describe how human resources and training data collected as part of a job analysis project were presented using information visualization and knowledge visualization techniques.
Method
Job Analysis Data
The data described in this article were collected as part of an agencywide job analysis for a large federal government organization. To present a concise and understandable visualization example, we focus here on human resources (HR) and training and development (TD) jobs.
About a year before the job analysis, the agency's HR and TD departments, which had been two separate organizational units, merged into a single business unit named Human Development (HD). Our analysis focuses on 10 jobs that were associated with the separate HR and TD organizational units that were placed under one leadership structure.
To begin the job analysis, a group of SMEs identified skill and knowledge requirements for each job, using prior data and expert opinion. The requirements served as the basis for a survey that was completed by a representative sample of job incumbents. Only knowledge and skills that were supported by the survey results were retained as components of the official knowledge and skill sets for each job.
Next, the similarity of the 10 jobs was determined. To do this, we used each job's skill and knowledge sets and performed a Jaccard analysis. (7) A Jaccard analysis determines the degree of similarity between two jobs, using the formula SJ = a/(a + b + c) * 100, where SJ = Jaccard similarity coefficient, a = number of elements shared by all groups, b = number of elements unique to the first group, and c = number of elements unique to the second group. A Jaccard analysis, performed with binary variables, excludes joint absences from both the denominator and the numerator and equally weights matches and nonmatches.
The similarity scores among jobs from the two groups ranged from 0.00 to 1.00, with smaller numbers indicating that compared jobs have less similar skill and knowledge requirements. The result of the analysis was a 10 * 10 similarity matrix, in which the lower half was an identical reflection of the upper half, so only half of the matrix indicating 45 similarities was important. Because each job was perfectly related to itself, the diagonal values were 1.00 and could be ignored.
We used two additional pieces of information to produce our visualizations (i.e., graphical visualizations). First, we obtained information on the number of incumbents in each job. Second, we determined each job's main organizational designation within the HD organization. Technically, a job could be located in any HD organization, but the jobs tended to be associated with specific organization units.
Visualization of Job Analysis Results
The similarity data, incumbent data, and organizational designation data were imported into the network analysis and graphing program Pajek. (8) The program was designed to allow researchers to analyze large data sets (up to a million vertices). Pajek has the added advantages of including sophisticated network graphing tools and powerful mathematical tools for performing statistical analyses. Pajek can be downloaded and information about the program can be found at http://pajek.imfm.si/doku.php.
Once HR and TD job analysis data were imported into Pajek, we could explore the data relationships visually. Even though we used an automatic graphing algorithm, graphing a network is often an iterative process that requires several passes until a meaningful and visually appealing representation emerges. (9)
The first graph we generated showed all similarity data. That meant 45 lines or links were connecting the 10 jobs. The result was a dense graph with clutter hiding the important links and the underlying structural relationships of the jobs were not readily apparent. Therefore, we had to systematically reduce the number of lines, or links, in the graph.
Link Reduction
There are two basic ways to prune the number of links in a graph and make a graph more understandable--the threshold approach and topology-based approaches. (10) The threshold approach is the simplest. Using this approach, all links that fail to reach a specific level of similarity are removed from the graph. Thus, only the strongest links remain in the graph, highlighting the most important relationships.
We wanted to have the smallest number of links that still resulted in a meaningful graph showing the interrelationships of all HR and TD jobs. Given that all the jobs we were analyzing were within a single organization, we felt it was important to have a completely connected graph. Our goal was that an agency manager or leader who looked at our graph could identify the similarities between an HR job and a TD job by tracing the lines on the graph; no job or groups of jobs would stand-alone. To produce such a "user-friendly" graph, we began by removing the weakest links in the data matrix. After several passes, we found that 25 links resulted in a completely connected graph. Any fewer links, and the graph had stand alone jobs. Any more links added clutter, obfuscating the most important job similarities.
In comparison to threshold approaches, topological approaches to culling network links rely on the identification of deeper intrinsic properties of data. The topological approach we used was Pathfinder network scaling, which was originally developed by cognitive psychologists. Specifically, we used similarity data (i.e., the Jaccard similarity scores) to identify the most efficient connections between HR and TD jobs. (11)
The Pathfinder algorithm is fairly complex, requiring the user specify two values, q and r. When q equals the number of nodes minus 1 and r is equal to infinity the result is a completely connected graph with a small number of links. For our graph we used q = 9 and r = [infinity].
FEED