Accountability: Measuring Mentoring and Its Bottom Line Impact.

Mentoring is a valuable tool for developing leadership talent, and it can have a bottom-line impact. To evaluate a mentoring program, a five-step process is recommended: 1) establish baseline numbers, 2) monitor the program, 3) measure mentoring (using the Alleman Mentoring Activities Questionnaire), 4) evaluate results and 5) calculate "return on investment. "Planned mentoring leverages a firm's succession planning efforts.

Introduction

Mentors have developed proteges since ancient times. The original Mentor in Homer's Odyssey trained his protege for national leadership and then helped him outmaneuver the ancient Greek equivalent of corporate raiders in a hostile takeover attempt.

Managing kingdoms or companies in turbulent times is a challenge. Using mentors, while not a panacea for resolving problems in the organization, can help companies deal with a variety of problematic situations. Managing effects on productivity of massive job shifts during reorganizations, developing a cohesive corporate culture in a merger, developing the potential of all employees including minorities, and reducing turnover are a few examples. While the benefits of mentoring have been recognized for years, this potent force operates entirely unmanaged in most companies.

Mentoring can be managed. Research on mentoring has shown us what a mentoring relationship is like [1,6,7,8,9,11] and has made it possible to understand, measure, manage and evaluate the relationship.

Mentors teach, guide, help, counsel, and inspire their proteges. They have a big impact not only on their proteges but also on their organizations. Many benefits (and a few potential risks) of mentoring relationships were clearly established by the early research on these relationships, and these benefits have been spelled out in a number of publications [5,8].

Early research has also looked at the nature of the relationship as described by the mentors and proteges involved. Mentoring was found to be active. Specific mentoring practices were identified which happened more often in mentoring relationships than in non-mentoring relationships [1]. Mentoring is also dynamic. It changes and evolves over time, moving through a series of stages or phases as it develops [9,11].

Alleman [4] identified specific behaviors that mentors used and variations in the patterns of mentoring practices as relationships developed. Managing behavior is a common organizational practice. Mentor behaviors can be taught, learned, managed, and evaluated, so this powerful force can be harnessed to accomplish organizational goals. Benefits can be maximized and potential problems managed. Ignoring these relationships allows them to operate uncontrolled within the organization, producing unpredictable and unmeasured results.

Benefits of a Mentoring Program

Mentors have an important impact on organizations and individuals. Research provides evidence of the benefits including higher productivity, better performance ratings, development of leaders, advancement of minorities and reduced turnover. In addition, participants acquire greater knowledge of the business, its politics, policies, products and customers. [2,3,5,12,13].

The first step in developing a mentoring program is to answer the following three questions: 1) What business issues are you trying to address (e.g., turnover, recruitment cost, productivity or some other problem)? 2) Why is addressing this issue important? Companies address issues that have a financial impact or affect the quality of the products and/or services. 3) How will the organization be different as a result of the program? For example, a more stable work force or more internal promotions may occur.

When business issues involving the effectiveness of employees have been identified, the next two questions arise: 4) Who do you want to change or develop? (i.e. who will be the proteges?), and 5) How will those people be different? Answering these questions establishes program objectives. These objectives must be specific, measurable, and realistic, and they must have an appropriate time frame.

To illustrate the process of evaluating a mentoring program, the experience of a Fortune 500 manufacturing company with unsatisfactory turnover in one division will be described. The division typically hired 18 to 20 new engineers a year; they experienced a 50% turnover rate for those new hires. Thus success of the program was defined by a turnover decrease from 50% to 20%. The objective was to increase the retention of newly hired engineers during the first three years of work. Their plan answers specifically who will be affected, the change desired, and how it will be measured. The results of the measurement (discussed later in the article) demonstrate that the program exceeded its goals.

Components of a Successful Mentoring Program

Sports analogies are often used in business. What it takes to win the game is similar to what it takes for a successful mentoring program. The coach's goal is a winning score. Success is easy to measure. Does the team have enough points to win? The team needs adequate resources, which include trainers, protective gear, practice facilities, and the true commitment of those in charge to provide what is needed for the full season despite occasional losses. They must have enough pre-season time to develop the players, teach them the plays and game plans, and develop their individual skills at the sport.

The coaches must have full knowledge of the game, the latest techniques and equipment, what has and hasn't worked in the past, and how to get out of tight spots and recover from setbacks. Most of all, the coaches must be able to teach what they know to the players, because it is the players, not the coaches, who score the points.

The same components are required for a mentoring program. The business objective of the program must be spelled out in clear, specific, measurable terms. What is the company trying to do? How will the company managers know if you "win?" What is the specific business objective of the program? How will success be measured?

Adequate budget, time, facilities, and true commitment of business leaders and participants (including the proteges bosses) are essential. Lip service is not enough. Appropriate length of a program varies with the program objective, but it must belong enough for mentoring to occur, for the lessons to sink in, and for the proteges to demonstrate the results. Mentoring practices are many and complex, and opportunities to apply them do not necessarily arise daily.

One investigation of spontaneous mentoring was conducted because people promoted out of a particular department were so successful later in the company. The managers in the department reported on their usual mentoring practices toward employees. Their reported behavior showed that all were acting as typical mentors. Both former and current employees reported on their experiences in the department as proteges. The longer the proteges had been in the department the more closely their reports matched those of their mentors. Those with three or more years reported the full array of mentoring experience. Those with less time in the department reported varying degrees of mentoring behavior depending on their time in the role. This finding conforms to research that shows that spontaneous mentoring relationships last three to five years and that the relationship and mentoring practices change over time as the protege develops [9, 11].

Those designing a mentoring program must have two sets of knowledge. First, they need to have a thorough background in mentoring. Reading a few articles or a book is not enough. This expertise involves the mentoring practices, their variations, and how to adapt them to different situations, but also what things can go wrong and how to avoid or minimize them. Second, they need to be familiar with the business setting in which the program will exist so that the program can be tailored to its particular needs. Since most businesses do not have in-house expertise in mentoring, a program design team usually consists of in-house staff familiar with the business and its people and a consultant with expertise in mentoring.

Finally, just as with sports teams, training-training-training and supervised practice-practice-practice are keys to developing the mentors and proteges and those (such as bosses) who will be working with them. Exhibit 1 summarizes the essential elements of a successful mentoring program.

Why Some Programs Fail

Designing an appropriate planned mentoring program to meet organization objectives is not easy and some programs fail. Research has documented the reasons for failure (see Exhibit 2) [3,5]. A failed program can have a severe impact on the company in addition to wasted resources. For example, a consultant called in to "fix" a badly floundering program was able to diagnose what went wrong (failure to train participants and unrealistic expectations) and was able to assess the damage that had been done (extreme frustration and anger), but was unable to save the program. The recommendation was to abandon the program and ... "never mention the word 'mentor' again." The department (Human Resources) and the managers promoting the program had lost so much credibility and had generated so much hostility as a result of the failed program that not only was the mentoring program lost, but other sound programs were also undermined.

Problems in a mentoring program pose not only the threat of cost to a company, but the risk of personal harm as well. A mentoring relationship, whether spontaneous or part of a planned program has a profound impact on both the mentor and the protege, on their professional development, their lives, and their careers. For example, if a boss feels excluded from the relationship of a subordinate and that subordinate's mentor (perhaps thinking normal lines of authority are being violated or that the mentor is interfering in areas of the boss' responsibility) the boss may become resentful and subtly sabotage the relationship and the careers of both protege and mentor. Another example of personal cost is the possible resentment of a spouse over non-work time devoted to the relationship by the mentor or protege. In the instance of a cross gender pair, there is the possibility of spouses and co-workers thinking the relationship is personal rather than professional.

These and other problems listed in Exhibit 2 can be avoided, minimized or dealt with, but the program designer must understand where the booby traps lie and have a good map to maneuver through the minefields. Do not set up a program that is likely to fail. In view of the harm it can do, a failed program is worse than no program at all. The four key elements absolutely essential to avoid failure are appropriate training, adequate resources, inclusion of bosses, and the true commitment of key people.

Importance of Evaluation

Planned mentoring programs are established to accomplish specific business outcomes. Investment in the program can therefore be justified by demonstrating a return on that investment. Program evaluation must answer three questions: 1) Have the program objectives been met? For example, has turnover been reduced? Have the pool of qualified job applicants and the pool of employees qualified for promotion increased? Or has performance improved? Has productivity increased? 2) What is the dollar impact? That is, is the investment in the program justified? Have turnover costs decreased? Is better performance reflected in reduced costs or increased sales? Have recruitment costs declined because the company has become an employer of choice? 3)Was it mentoring that made the difference? That is, are employees with a mentor more successful, more likely to be promoted, and more likely to stay than those who do not have a mentor?

Evaluation is a critical part of a planned mentoring program. A program is established to accomplish a goal. Investment in the program can be justified if a return on that investment can be demonstrated by meeting program goals and realizing the dollar gains associated with those goals.

In addition to measuring program outcomes to demonstrate that the objectives were met and to justify program investment, there are other reasons to measure mentoring itself. Assessing the pre-program levels of mentoring activity can be used as a guide to planning the training of participants or to align expectations. Mentoring activity is multi-faceted. The data in Figure 1 was gathered using the AMAQ which contains items assessing ten activities of typical mentors. The scales are: Teach the Job, Counseling, Endorse Activities, Sponsor, Protect, Teach Politics, Career Help, Challenging Tasks, Friendship and Demonstrate Trust.

Each item is rated on a scale of one to five with five indicating a high level of mentoring activity. The chart can reflect data from individual mentors, proteges or group participants. When examining individual data, the goal is to understand expectations before and after an event (such as prior to training on what a good mentor does or after several months have passed). It may be used to compare mentor and proteges expectations as a program begins, or at specific intervals throughout the mentoring relationship.

Figure 1 is an example of data which was gathered from participants before and after a mentoring program. Figure 1 contains data obtained from mentors describing their plans for activities with proteges after a training program (noted as "Plans After" on the chart). The "Actions Before" line on the chart represents the ratings done by the mentors describing their behaviors prior to the training. Before the mentoring training the mentors described themselves as not providing guidance in organizational politics and counseling on how to handle situations. Endorsement, sponsorship and protection were also rated less frequent an activity. After training, the change in beliefs about how time should be spent during a mentoring relationship is evident. After training, the participants have a better understanding of the activities which may occur between mentor and protege The figure shows that after the program the participants report planning to spend more time on the mentoring areas of teaching politics, counselin g, providing protection and friendship. The charts on questionnaire feedback could be compiled for each mentor-protege pair to align expectations or other combinations. The measurement of actual behaviors and/or expectations of behaviors can assist program developers in designing, implementing, and revising to assure a successful program.

This diagnostic use of the assessment tool can show relative areas of strength and weakness so that training can focus on mentoring activities that are weakest. Comparing that same data with a measure of the mentor's post-training mentoring plan can evaluate the training process. Finally, comparing postprogram measures of mentoring activity with the preprogram baseline data will indicate whether mentoring activity increased.

The evaluation of the mentoring process occurs in three phases: 1) preassessment of existing relevant baseline data; 2) monitoring and using feedback for program management; and 3) measurement of program outcomes. All three are closely tied to measurable, specific program objectives.

What to Measure

Measurement includes gathering data on both the quality and quantity of behavior that is or is not occurring between the mentor and protege. The pre- and post-program levels of the objectives are measured as well as the cost of the program. To accomplish this, several kinds of information must be gathered and calculations must be made.

The amount and quality of mentoring activity must be measured, not mere frequency of contact, but what actually happened during that contact and how much mentoring took place. Mentoring includes behavior focused on teaching, coaching, managing politics, influencing and showcasing. The Alleman Mentoring Activities Questionnaire measures the frequency and quality of specific instances of these mentoring practices from both the mentor and proteges perspectives. This measurement is detailed in steps 1 to 3 below. The level of success in meeting program objectives must be evaluated, as explained in step 4. Finally, the return on investment is calculated by assessing the dollar benefits of the program, the cost of the program, and the difference between them as explained in step 5 below.

Step 1: Establish baseline numbers

Before the program is put in place, gather two types of data: the preprogram level of mentoring activity (How much spontaneous mentoring activity is occurring?) and the preprogram level of the specified program objectives. Information gained can be used not only as baseline data for before and after comparisons but also can serve as a guideline for training sessions in the program.

Step 2: Monitor the program

Mentors teach proteges throughout the relationship. Which things they teach and the methods they use to teach them are different in the early days of the relationship, as the relationship develops, and as the mentoring process nears completion.

One source of potential problems in a mentoring relationship is failure to move appropriately through the mentoring stages. Mentoring activity must be analyzed during the program to ensure the program is progressing as planned and to give feedback to participants. Analyzing patterns of specific kinds of mentoring activity and changes in those patterns shows movement of mentors and proteges through the sequential stages or phases of the relationship.

Step 3: Measure mentoring activity

Specific program objectives allow evaluators to measure the effects of the program on desired outcome(s) by comparing preprogram and post-program data. Program success can be shown by demonstrating what increases in productivity were gained and how much turnover was decreased. To credit the mentoring program with this success, however, it is necessary to demonstrate that mentoring activity also increased. Figure 1 contains an example of before and after ratings of specific mentoring behaviors. This data can be gathered on individual pairs using the AMAQ to assess the quality of the mentoring relationship or on all program participants to determine how well a program is working.

Step 4: Evaluate program results

When the program ends, evaluate outcomes. Compare the end-of-program data for the mentoring questionnaire and the desired outcomes with the pre-program data. This comparison answers two critical questions: 1) Did mentoring activity increase? and 2) Were the program objectives met?

Step 5: Calculate return on investment

Justify Investment in the program by comparing the dollar gains associated with the targeted outcome changes and the cost of the program. Consider all of the costs mentioned in the next section, both internal and external. Lapointe and Verdin [10] give detailed instructions for making this kind of calculation.

Demonstrating Results in Terms of Dollars

Returning to the experience of the manufacturing company discussed earlier, the next step is to measure results and apply dollar figures [See Exhibit 3].

While turnover cost varies by occupation and from one company to another, it is generally estimated to be 140% of annual total compensation for the position. Avoiding the loss of a single $50,000 per year engineer (with a $70,000 total compensation package) would save $98,000.

The company hired eighteen engineers and lost only one by the end of three years. They realized a significant savings in turnover costs for eight engineers. Total savings were $784,000 for the engineers hired in the single year of the pilot project. Since the program is repeated with each year's new hires, additional savings are realized with each group.

Dollars invested in the program must be subtracted from the gains. These will vary by company according to their costs and structure. The elements shown in Exhibit 4 must be measured (usually in terms of time and compensation for that person's time) and dollar figures must be assigned.

Program investments costs consist of two major categories. Internal or "soft" costs are the costs of staff time in planning, evaluating, implementing, and promoting. The time dedicated to the project is a major expense in "soft dollars." The program developers and sponsors can spend many hours selling the program and managing the internal hurdles as they occur. Since program champions in influential positions are critical to success, this time is a valuable investment in the program. Program participants (mentors, proteges and bosses) will also invest their time to achieve the desired results.

The external costs (shown in Exhibit 4) are the "hard" costs often associated with a new venture. The mentoring program will require a budget to develop the program, provide the training, hire a consultant and evaluate the effectiveness. The training and development costs are frequently under-budgeted, particularly when the planners are unfamiliar with the importance of training, the extent of training needed and the key it provides to successful experiences for participants. Both internal and external costs need to be factored in to assess the program's impact.

The first two questions raised at the beginning of the article have been answered. The program objective of reducing turnover to 20% was not only met but also exceeded in a real world situation. The dollar impact was calculated at $784,000 per year less program costs. The program costs were estimated at $50,000. Thus, for a $50,000 investment a gain of $734,000 was realized. One of the original questions remains.

Did Mentoring Make the Difference?

While research has shown the impact of mentoring on individuals and organizations, many factors can moderate the outcome of any specific mentoring program. Changes in the outside job market can influence turnover rates. Installation of new equipment or systems can change productivity levels. The state of the economy can affect sales objectives. All of these and other factors can be controlled statistically, but the simplest way to demonstrate that mentoring was a part of the solution is to show a relationship between the level of mentoring (scores from the mentoring questionnaire [4]) and whatever measures have been developed to quantify the success criteria for the business objective (s) of the program. For example, in companies with a planned mentoring program with well-designed training, the typical finding is that high levels of mentoring are associated with high levels of success in meeting the program's business objectives. Similarly, moderate levels of mentoring are associated with moderate levels of o utcomes and low levels of mentoring are associated with little or even negative effects [31.

Summary

Research has shown the effects of mentoring on organizations and individuals. Spontaneous mentoring has long been a means of development available to the select few. In the last twenty years the tools have been developed to manage the mentoring process and measure the outcomes, including the economic impact. Today companies can reap the benefits of mentoring by designing and monitoring planned mentoring programs. The Alleman Mentoring Activities Questionnaire is used by many companies to measure behavior and align expectations. Careful planning combined with training of all involved can positively impact a mentoring program to assist in meeting business objectives.

Mentoring programs can make a difference and will positively affect the bottom line of an organization if it is designed, implemented and managed well. Mentoring can be a valuable tool in achieving company goals, and the benefits can be quantified, including financial gains.

Biographies

Elizabeth Alleman, Ph.D.

Dr. Elizabeth Alleman is an internationally recognized expert on mentoring relationships with clients on four continents. She authored the Alleman Mentoring Activities Questionnaire. A consultant to business and industry and a licensed psychologist, she specializes in the design of leadership and staff development systems for organizations.

Diana L. Clarke, Ph.D.

Dr. Diana L. Clarke specializes in building effective organizations by aligning the people systems with the business objectives. She consults with organizations on assessment, development, selection, retention, succession planning and alignment of talent with the core business competencies. Clients vary from Fortune 500 companies to small privately held corporations. Mentoring programs are designed to support the succession planning and leadership development programs in the organization. She is a licensed psychologist and member of the board of trustees of the Ohio Psychological Association.

References

(1.) Alleman, E., J. Cochran, J. Doverspike and I. Newman. "Enriching Mentoring Relationships," Personnel and Guidance Journal, 62(6), 1984, 329-332.

(2.) Alleman, E., I. Newman, H. Huggins and L. Carr. "The Impact of Race on Mentoring Relationships," International Journal of Mentoring, 1(2), 1987, 20-23.

(3.) Alleman, E. "Two Planned Mentoring Programs that Worked," Mentoring International, 4(1), 1989, 6-12.

(4.) Alleman, E. Alleman Mentoring Activities Questionnaire. 1999. Available from Silver-wood Enterprises, LLC. PO Box 363, Sharon Center, Ohio 44274 (330) 239-1646.

(5.) ASTD. "Design Productive Mentoring Programs," Infoline #609. 1986. Available from American Society for Training and Development. 1630 Duke Street, PO Box 1443, Alexandria, VA 22313.

(6.) Bowen, D.D. "On Considering Aspects of the Mentoring Process," Behavior Today, 13(15), 1982, 4-5.

(7.) Caruso, R.E. An Examination of Organizational Mentoring: The Case of Motorola. Unpublished Doctoral Dissertation, London School of Economics, University of London, 1990.

(8.) Gray, WA and M.M. Gray (eds.) Mentoring: Aid to Excellence (2 vols.). Available from International Centre for Mentoring, 1986, 11316 Ravenscroft Place, Sidney, Canada V8L 5R4.

(9.) Kram, K.E. "Phases of the Mentor Relationship," Academy of Management Journal, 26(4), 1983, 608-625.

(10.) Lapointe, J. and J.A. Verdin. "How to Calculate the Cost of Human Resources," Personnel Journal, 67(1), 1988, 34-45.

(11.) Phillips-Jones, L. Mentors & Proteges. New York: Arbor House, 1982.

(12.) Roche, O.R "Much Ado about Mentors," Harvard Business Review, 57, 1979, 14-28.

(13.) Wilson, J.A. and N.S. Elman. "Organizational Benefits of Mentoring," Academy of Management Executive, 1990, 88-94.

A GOOD MENTORING PROGRAM MUST HAVE:

* Clear, specific, measurable objectives

* Adequate resources

* Enough time

* Real commitment of company leaders

* Expertise in the mentoring process

* Full knowledge of the business and its history

* Training for mentors, proteges, the proteges bosses, and others affected by the program

"Training is a crucial component for an effective planned mentoring program."

WHY PLANNED MENTORING PROGRAMS CAN FAIL

* Unrealistic objectives

* Not training participants and others on mentoring

* Program design flaws

* Geography constraints -- mentor and proteges in different offices

* Lip service -- no real commitment by key people

* Time -- expect changes too quickly (1 year minimum)

* Inadequate resources

* Organizational structure or barriers

* Failure to anticipate and plan for problems

* Others (spouses, co-workers) misunderstanding the relationships

* Pairing problems -- misaligned expectations, non-complimentary styles

* Real world events (takeover, downsizing)

* Bosses not included

MENTORING PROGRAM RESULTS Objective Success Criteria Measured By Engineers retained Turnover decrease Personnel records three years from 50% to 20% 17 of 18 new engineers Turnover decreased No need to replace retained for three years from 50% to 10% eight engineers Objective Dollar Gain Engineers retained $98,000/per person three years 17 of 18 new engineers $784,000 -- Minus retained for three years program investment

PROGRAM INVESTMENT ELEMENTS

Internal Costs "Soft Costs"

* Design team time

* Participant training

* Program monitoring

* Problems:

-Managing them

-Cost of their impact

External Costs "Hard Costs"

* Mentoring expertise:

-Use a consultant?

-Develop expertise Internally?

* Training (place, materials)

* Evaluation expertise

* Evaluation materials