The next generation of leadership assessments: some case studies.

Great leaders. We know them when we see them, but how can we identify those high-potential employees who will one day run their organizations? Workforce planners have been warning for years of the impending wave of baby boomer retirements. Many organizations are finally heeding those warnings and taking steps to identify and nurture leadership talent.

While professionals in psychology have been in the forefront of leadership assessment for many decades, some fascinating new approaches have recently emerged that allow organizations to maximize the use of technology, and these are gaining broad acceptance in corporate America and in local, state, and federal government organizations.

In this article, we provide a brief overview of leadership theory and research, some background on the traditional assessment center process, and detailed descriptions of three of the latest approaches to leadership assessment, along with case studies drawn from public and private organizations. We conclude with some suggestions for new directions in leadership simulation design.

The first of the new approaches we shall describe is the Telephone Assessment Program (TAP), which was created by one of the authors of this review, Seymour Adler, PhD. TAP was one of the earliest attempts to take the complex and costly assessment center process and make it more efficient.

A later innovation that will be described here is LEADER, which is a mini assessment center on computer. Aon Consulting was the first company to market the fully automated leadership simulation developed by Kirk Rogg, PhD. Sadly, Dr. Rogg passed away in 2007, ending a brilliant career much too soon. This article is dedicated to Dr. Rogg, as well as to Dr. Mark Lifter, the former head of the Aon Consulting Talent Solutions practice, who supported the LEADER research.

The third approach we describe, Video-Based Assessment Centers, is another attempt to enhance the efficiency and effectiveness of an assessment center process by using videotape in creative ways. David M. Morris, PhD, JD, president of Morris & McDaniel and the designer of this innovative, streamlined assessment system, is also one of the authors of this review.

What Does the Leadership Research Tell Us?

We are fortunate to have an excellent summary of the history of leadership theory and research in the form of a 2007 special issue of the American Psychologist. (1) This issue begins with a compelling quote by Robert Sternberg: "The United States became a great nation because of the leadership skills of the Founding Fathers. Whether it will remain a great nation will depend, in large part, on the leadership skills of those in power today." (2) So, with the success of our organizations, our government, and our future depending upon our leaders, what should those responsible for developing tomorrow's leaders be assessing?

Assessing the Person

Zaccaro has summarized the evolution of trait-based leadership theories. (3) He began with the earliest leadership studies and their emphasis on the unique attributes of leaders that were inherited and part of leaders' genetic makeup. (4) Zaccaro indicated that this perspective, suggesting that leadership qualities were traits that were largely immutable and not amenable to development, guided the preponderance of the leadership research into the 20th century until Stogdill and others suggested that trait-based leadership was insufficient to explain leaders' effectiveness. (5) Zaccaro added, "This rejection [of trait-based leadership] was widespread and long lasting, and it echoed in most of the major social and industrial and organizational psychology textbooks for the next 30 to 40 years." (6)

Zaccaro noted that in the 1980s, charismatic and transformational leadership models again surfaced, emphasizing extraordinary qualities of individuals as determinants of those people's effectiveness. Zaccaro concluded that recent studies, including his own research, have linked personality variables and other stable personal attributes to leaders' effectiveness, providing a substantial empirical foundation for the argument that traits do matter in the prediction of individuals' effectiveness as leaders. (7)

Many theorists have gone beyond traits, emphasizing extensive lists of abilities, competencies, and skills that are essential for leadership effectiveness. (8)

Assessing the Situation

While many theorists have been concentrating on the identification of individual attributes that contribute to leaders' effectiveness--whether they be stable personality traits, competencies, knowledge, values, or something else--others have focused on the environment in which leaders operate. Vroom and Jago (9) credited Stogdill (10) with being among the first to conclude that an adequate analysis of leadership involves not only a study of leader attributes, but also an investigation of the situations in which a leader operates.

The Ohio State studies in the 1950s and 1960s, with Shartle, Stogdill, Fleishman, and others, focused on leadership behavior. Vroom and Jago suggested, "Leader behavior research was a step in the direction of acknowledging the role of situation or context in leadership. Unlike traits, behavior is potentially influenced not only by the leaders' dispositions, but also by the situations that leaders confront.... Leader behavior can therefore be an effect of subordinate behavior as well as a cause of it." (11)

There were also a number of pure situational theorists who found that very little of the organizational outcome variance could be explained by changes in leadership. (12)

Assessing Both

Vroom and Jago have summarized the current state of leadership theory best, writing, "Most social scientists interested in leadership have now abandoned the debate between person or situation in favor of a search for a set of concepts that are capable of dealing both with differences in situations and with differences in leaders." (13) These approaches have yielded contingency theories that try to describe what type of people and behaviors are effective in different situations. (14) Vroom and Jago concluded that leadership, which involves the process of influence, is represented in all aspects of that process, including the traits of the source of influence, the cognitive process of the source, the nature of the interaction that makes influence toward a goal possible, and the situational context of that process. With so many variables influencing the effectiveness of a leader, the leadership assessment had best be complex. What better way to capture and manipulate these various aspects of leadership than in simulations?

The Traditional Assessment Center Process

For more than 25 years, assessment centers have been seen as a valid--by some, the most valid--method for assessing leadership potential. (15) A core element of the assessment center method is deploying a team of trained assessors who evaluate each candidate's job-relevant leadership competencies by observing the candidate in multiple simulation exercises. In addition, assessors often also draw on other assessment tools such as tests and interviews. Traditionally, final candidate evaluations are arrived at through a judgment process that involves discussions between the assessors of their individual evaluations and the formation of a consensus opinion regarding a candidate.

It is important to note that a half century after Douglas Bray first introduced assessment centers in a nonmilitary context for leadership assessment at AT&T, (16) it is still not entirely clear why assessment centers are valid, as C.E. Lance has discussed. (17) What is clear is that assessment centers, when properly designed and administered, provide valid information for making hiring, promotion, and development planning decisions for leadership positions.

For most of the past 50 years, leadership assessment centers have been physical locations, with multiple candidates and multiple assessors present for periods ranging from one to four days. (18) In the most typical setup, 10 to 12 candidates, six assessors, and an administrator come together at a central facility that has one or two conference rooms and six or more smaller rooms for breakout sessions. Candidates come to the location for a day or two of assessment exercises, interviews, and tests. The assessors stay a day or two longer to discuss each candidate and to develop a composite candidate profile and final overall evaluation for each candidate. This traditional structure is still widely used, especially in the public sector. (19)

This traditional bricks-and-mortar design for assessment centers is costly and cumbersome. The notion that this traditional design is the only acceptable way to do assessments has, over the past 20 years, led many organizations to move away from using assessment centers despite the centers' proven effectiveness and toward alternative and often inferior ways of assessing employees' and possible hires' leadership potential. Other organizations have increased assessment center efficiency by videotaping participants' performances on exercises and scoring the participants at a later time. (20) Lievens and Thornton (21) discussed the increased use of computerized in baskets or video simulations in which participants provide responses to multiple-choice questions. They also pointed out that, while such uses of technology can increase the efficiency and flexibility of assessment centers, those efficiencies and flexibilities come at a significant cost in terms of lowered fidelity of the simulation exercises and, thus, the credibility and validity of the exercise results.

One of the key benefits of using assessment centers in which candidates and assessors interact face to face is the rich behavioral information the interactions provide about a candidate. That information can be critical in a developmental assessment center when participants receive diagnostic feedback and work with a manager or coach to create a specific and actionable individual development plan. It can also be valuable in a selection/promotion assessment center for promoting perceptions of procedural fairness. (22)

Print Get the Mag Weekly Updates