Surrogate Expertise Indicators of Professional Financial Analysts.

As in Hunton and McEwen (1997), the case firm was AutoZone, Inc., a company listed on the New York Stock Exchange. We chose AutoZone, Inc. because the brokerage firm did not follow this company or the automotive parts industry; thus, none of the participants would be at an advantage during the analysis. The case included the complete annual report, a summary of general economic and industry information, and other company information from the 10-K. At the exit stage of the experiment, subjects were informed that the dates attributed to the case firm were incorrect. They were also asked to maintain secrecy concerning the case until the experiment was completed on Friday afternoon. All analysts received the same information at the beginning of the experiment. After analyzing the information provided, subjects activated an icon indicating that they were ready to provide a first-quarter earnings forecast, and a buy/sell/hold recommendation.

We examined the relations between various experiential and cognitive variables with expertise and All-America Team membership for the subjects. We assessed the relation between these surrogates for expertise with management's assessment of expertise. We then examined the relation between forecast error and experimental determinants of forecast error as described in Hunton and McEwen (1997), in order to validate the usefulness of All-America Team membership as a surrogate for expertise.

RESULTS

The experiment conducted by Hunton and McEwen (1997) resulted in the collection of various experiential and cognitive variables. The experiential variables are as follows: age, years as a financial analyst, years with the firm, and compensation rank (where firm management ranked the analysts from "1" as highest to "60" as lowest in terms of annual compensation.) The cognitive information processing variables include the following: cognitive search strategy, time spent initially searching for information, time spent reviewing information before issuing an earnings forecast, and the number of times the analyst accessed a built-in calculator. We collected one additional demographic variable: membership on the All-America Research Team. This variable was coded such that a "one" represents membership and "zero" reflects nonmembership.

Cognitive search strategy was determined as follows. First, the computer assigned a sequential log number for each informational item the subject accessed. Next, the computer identified each item by a position number, indicating the relative position of the item on the computer screen (it is important to note that all items were randomized per individual). Finally, the cognitive search strategy variable was defined as the correlation between the sequential log numbers and the computer-screen position numbers.

Hunton and McEwen (1997) analyze differences in experimental performance as a function of this cognitive strategy. Sequential searchers analyze information by looking at the next item on the screen, and will exhibit a relatively high degree of correlation between the log and position numbers. Directive searchers, who look for specific information, will exhibit a relatively low correlation between the log and position numbers. Hunton and McEwen confirm that more accurate analysts tend to be directive searchers.

Since the use of an IRIS system is not available to the public, our objective is to validate a surrogate for expertise that appropriately reflects accuracy and that is publicly available. We begin our search using grouped t-tests (our results are not significantly different from those reported using a Mann Whitney test) where subjects are grouped as more accurate if they fall in the top 50% of management's rankings, less accurate otherwise (the binary indicator variable designates subjects "1" through "30" as more accurate and subjects "31" through "60" as less accurate) The results are presented on Table 1, panel A.

Panel A provides evidence that experiential and cognitive variables differentiate the binary classification of expertise. Although there is no significant difference in the age of the relatively low and high expertise analysts (p-value [less than] .1150), low expertise subjects tend to have worked fewer years as a financial analyst (p-value [less than] .0401), fewer years with the firm (p-value [less than] .0301), and have a higher compensation rank (lower compensation level) (p-value [less than] .0263). The subjects are also differentiated by the cognitive information processing variables. High expertise subjects tend to have lower cognitive search strategy scores (p-value [less than] .0001), spend less time in initial screening task (p-value [less than] .0001), but more time in the review phase (p-value [less than] .0001), and use the calculator more often (p-value [less than] .0001). Panel B (Table 1) provides evidence that all of the experiential and cognitive variables are significantly intercorrelated, with the exception of time reviewing prior screens and compensation rank. We exclude team membership from this analysis since it is a binary variable and thus not appropriate for correlation.

Our next step was to factor analyze the experiential variables (Table 2, panel A) to obtain a composite experiential score. We then separately factor analyze the cognitive variables (Table 2, panel B) to yield a composite cognitive score. Our final factor analysis includes both the experiential and cognitive information processing variables to gain some insight into the relative independence of these factors as surrogate indicators of expertise (Table 2, panel C).

Using Varimax rotation, the factor analyses of the composite experiential (panel A) and composite cognitive variables (panel B) both yield single factors where the eigenvalues exceed one. When combined, factor analysis of the joint experiential/cognitive variables yields two significant factors, where the eigenvalue of each factor exceeds one (panel C). All of the experiential variables load high on the first factor, while the second factor consists of cognitive information processing variables.

We next perform logistic regressions using the rotated factor scores as independent variables and the binary classification of accuracy, provided by firm management, as the dependent variable. We do so in order to assess the degree to which the experiential variables alone and the cognitive variables alone provide a meaningful surrogate for our performance-based measure of expertise. We then combine these variable sets to determine their efficacy as joint surrogates. Finally, we repeat the analysis using All-America Team membership as the surrogate for expertise. Table 3 provides the classification output from the logistic regressions.

The use of experiential variables alone (Table 3, panel A) in the factor/logistic regression yields a correct classification rate of 60%. The experiential variables are exceptionally poor classifiers of low accuracy subjects, correctly classifying these subjects only 50% of the time. The variables are somewhat better at characterizing the higher accuracy subjects (70% correct classification). Use of the cognitive factor scores alone to classify accuracy greatly improves the correct classification rate to 88.33% (panel B). Low accuracy subjects are classified correctly 83.33% of the time and high accuracy performers are correctly classified 93.33% of the time. Combining the experiential and the cognitive factor scores in the logistic regression (panel C) provides the highest correct classification rate (91.67 %) with 86.67% of the low accuracy subjects correctly classified, and 96.67% of the high accuracy subjects correctly classified. These results improve the classification rate noted in Hunton and McEwen ( 1997) of 88.33%.

For the subjects of this study, the combined model (including both cognitive search strategy and experiential variables (panel C)) seems to classify performance-based expertise appreciably better than experience alone (panel A) and cognitive scores alone (panel C). Unfortunately, cognitive variables are neither publicly available as surrogates for expertise nor are they conveniently and reliably obtainable via psycho-metric measurement. Table 3, panel D provides the results of a measure that is publicly available. A logistic regression of the binary accuracy classification upon All-America Research Team membership provides a correct classification rate of 90.00%. These results are not substantially different from the classification results using cognitive factor scores (panel C). This finding suggests that membership on the All-America Research Team may provide a reliable surrogate for sell-side professional financial analyst's expertise level.

Validating the Surrogate

In order to validate the All-America Research Team surrogate for expertise, we use regression models as defined in Hunton and McEwen (1997). The dependent variable is experimental forecast error, defined as quarterly (annual) forecast less the quarterly (annual) actual earnings. Unlike the earlier study, we use membership on the All-America Research Team as the surrogate for expertise, replacing years as a financial analyst and cognitive search strategy.

The validation results include the effects of motivation since these effects have been shown to substantively contribute to forecast bias. Each subject was randomly assigned to one of the three motivation scenarios: the subjects had to assume that his (her) firm would 1) underwrite a share offering for AutoZone, Inc., 2) begin to follow the firm or the industry, or 3) have no future contact with the firm or the industry. Two dummy variables assess the contributions of incentives on experimental forecast accuracy. The underwrite indicator variable is coded as "1" for an underwrite position, "0" otherwise. The follow-only indicator variable is coded as "1" for a follow-only indicator, "0" otherwise. Our regression results are shown on Table 4.

Print Get the Mag Weekly Updates