Mobile Edition
Print
Get the Mag
Weekly UpdatesVariable Measurement
The standards used in assessing effectiveness in our model include both goals and program characteristics. Perceived goal effectiveness was assessed along two dimensions--goal importance and primary goal achievement. An examination of auditing literature resulted in the identification of the following implied goals: reduce audit failure (Public Oversight Board, 1993), improve audit quality (SEC Practice Section, 1989), enhance public knowledge and image of the profession (SEC Practice Section, 1992), reduce congressional criticism of the accounting profession (Public Oversight Board, 1983), and detect/reduce fraud in financial statements (National Commission on Fraudulent Financial Reporting, 1987; Opinion Research Corporation, 1974). An "other" category was provided for respondents to mention a goal not specified. Respondents were asked to rank order these goals (1 = most important).
Perceived primary goal achievement was assessed by using the following question: "To what extent has the peer review program of the accounting profession been able to achieve the objective you ranked No. 1 above?" Responses were recorded on a 5-point, Likert-type scale (1 = completely, 5 = not at all).
Perceived program effectiveness was assessed with an eleven item scale developed for this study based on the peer review literature (American Institute of Certified Accountants, 1986a, 1986b; Public Oversight Board, 1990). The items describe the purpose, methods and effects of the program. These items include: (1) the peer review program assures CPAs have adequate quality control systems, (2) the peer review program assures that quality control standards are applied in audits, (3) firms in the peer review program are better able to detect fraud in audits, (4) the probability of audit failure is reduced for firms in the peer review program, (5) peer review is ineffective due to its lack of sanctions, (6) a measure of the effectiveness of the peer review program is the number of proceedings filed against CPA firms, (7) clients should insist that their CPAs undergo peer review, (8) peer review is an educational versus a control process, (9) the investment community understands the role of the peer review progra m, (10) client companies understand the role of the peer review program, and (11) the public understands the benefits of the peer review program. The selection of the number of effectiveness items was constrained by differences in constituents' exposure to the details of the peer review program, as well as the desire to encourage responses by the use of a shorter scale. The length of the instrument, however, was deemed to be sufficient to explore constituent perceptions of the peer review program. A 5-point, Likert-type scale was used to record responses (1 = strongly disagree, 5 = strongly agree).
Constituent groups serve as the primary independent variable in the study. This variable has four components: CPA firms, audit clients, bankers, and financial analysts.
The control variable, peer review program familiarity, was assessed by a single item. For CPA firms, familiarity was assessed using a somewhat objective question regarding the firms last peer review with a response scale ranging from 1 = never reviewed to 5 reviewed less than a year ago. The other constituent groups were asked to indicate their degree of familiarity with the program using a response scale ranging from 1 = not at all familiar to 5 = extremely familiar. Our data show that 95% of the CPA firms had undergone a peer review in the past three years and 60% of these CPA firms had also conducted peer reviews of other CPA firms. Based on these statistics, it appears that the CPA firms are familiar with the peer review program.
Data Analysis
Because some responses were based on ordinal scales and others on interval measures, both parametric and non-parametric tests were used in analyzing the data. Hypothesis H1, concerning perceived goal effectiveness, was analyzed by first calculating Kendall's coefficient of concordance (W) to assess the agreement of the rankings of goal importance among the constituents. A Kruskal-Wallis (KW) test was used to determine the goals on which there was disagreement. Significant K-W tests were followed by multiple comparison tests to explore specific differences (Conover, 1980). Hypothesis H2, concerning the extent of perceived agreement on goal achievement, was examined using analysis of covariance (ANCOVA) controlling for program familiarity as the covariate. Significant F-tests were followed by multiple comparison tests (Tukey, HSD) to explore significant differences among constituent groups. Agreement on one or more goals or processes provides some evidence in support of the effectiveness of the peer review pro gram.
Hypothesis H3, concerning perceptions of the effectiveness of peer review programs, was analyzed using multivariate analysis of covariance (MANCOVA). The eleven process variables were the dependent variables, and the constituent groups served as the independent variable. Program familiarity served as the covariate to control for different levels of familiarity with the peer review program among the constituents. Significant MANCOVA tests were followed by univariate ANCOVAs and multiple comparison tests (Tukey, HSD) to identify differences among the specific constituent groups on each variable.
Analyses and Interpretation of Findings
Before analyzing the data, we examined the extent to which the four constituent groups had prior familiarity with the peer review program which might influence their perceptions of its effectiveness. An ANOVA revealed that the four constituent groups differed significantly regarding their familiarity with the peer review program (F=11.05, p [less than or equal to] .001). CPA firms were the most familiar (x = 3.80), followed by the audit clients (x = 2.88), bankers (x = 2.38) and financial analysts (x = 1.78), who were the least familiar on average. Given these differences, subsequent tests for differences among constituents were conducted, controlling for program familiarity.
Goal Effectiveness
Hypothesis H1, concerning constituent perceived differences on the relative importance of the peer review program goals, was partially supported. There appeared to be some agreement among constituents concerning goal importance when rankings of all goals plus the "other" category were analyzed. The significant coefficient of concordance of 0.49 (p [less than or equal to] 0.0001) indicates an association among the separate goal rankings of the constituents (see Table 3, Panel A). The extent of agreement was weaker when only the six specific goals were analyzed, as indicated by a decrease in the coefficient of concordance to 0.30, p [less than or equal to] 0.0001 (not reported in Table 3). Most constituents believed the important goals were among the six specified; three of the four constituent groups ranked the "other" category as number 7 (least important).
An examination of Table 3, Panel B for the differences among the constituent rankings for each goal revealed that, contrary to hypothesis H1, all four groups agreed on the two most important goals of the peer review program. All four constituent groups ranked "improve auditing practices of member firms" as the most important goal of the peer review program. Furthermore, they agreed that the second most important goal of the peer review program was "self-regulation."
There were significant differences among the constituent groups on the rankings of the other four goals, however, partially supporting hypothesis H1. All groups except accountants ranked "reduce audit failure" as the third most important goal. CPA firms, on the other hand, felt that "enhancing the public image of the profession" was the third most important goal. They also ranked "reduce government criticism of the profession" as the fourth most important goal, higher than any other group. Interestingly, the two groups which represent users (banks and financial analysts) ranked "detect audit fraud in financial statements" as the fourth most important goal, while accountants ranked this goal as last in importance and audit clients ranked this goal sixth in importance. The three non-accountant constituent groups ranked the "Other" category last in importance, while accountants ranked it as the sixth most important goal. Responses of the four constituent groups with respect to "Other Goals" revealed the followi ng additional goals: public relations, and improve compliance with quality control standards, education, and membership requirement.
In summary, with respect to hypothesis H1, all constituents view the primary goal of the peer review program to be the "improvement of audit practices." These results also demonstrate that accounting firms and other constituents do not agree on the importance of peer review in helping auditors better detect fraud in financial statements. When compared to the other three groups, the accounting firms view peer review as being least important for "improving fraud detection." Thus, hypothesis H1 is partially supported.
Once it was determined that all four constituent groups agreed that improving auditing practices" was the most important goal of the peer review program, we performed an ANGOVA (while controlling for the level of program familiarity as a covariate) in order to test hypothesis H2 that there were differences among the four groups regarding the degree to which the program's primary goal had been achieved (see Table 3). Although an inspection of the means [2] suggests that CPA firms (X =2.15, n=48) believed that this goal had been achieved to a greater extent than the other three (audit client firms (X =2.48, n=23), bankers (X =2.60, n=15), and financial analysts (X =3.00, n=13)), the ANCOVA results are not significant and, therefore, do not support hypothesis H2. In general, the constituents agree that the primary goal of "improving audit practice" has been achieved to some degree.
FEED