Analytical Tools to Facilitate Transitions Into New Writing Contexts: A Communicative Perspective.

Through a field study using essays from the Analytical Writing Assessment (AWA) of the Graduate Management Admission Test we developed a set of analytical tools to diagnose students' potential problems in MBA writing assignments. The four tools--task, coherence, reasoning units, and error interference--can be used to evaluate writing effectiveness based on success of the communicative interaction between reader and writer. We tested the tools via blind and open scoring methods and in direct writing consultations with students at two contrasting business schools. Correlations between the AWA holistic scores reported by the Educational Testing Service and our analytical tool scores suggest that the tools conceptualize distinct traits in the AWA essays and can in turn be used to diagnose potential deficiencies in MBA writing. In post-consultations surveys, students reported learning about problems in their writing and about ways to address these needs as well as improved confidence as they transitioned into MBA writing. By accommodating reader and contextual differences, these analytical tools may bridge discourses where conventional checklists have failed.

Keywords: writing assessment, MBA writing, GMAT. Analytical Writing Assessment (AWA), writing across disciplines

If writing like an expert depends in large part on "being there" for some period of time and learning the habits of a particular community, how can we help individuals move into new writing contexts? Beyond basic textual skills like writing grammatically correct sentences (Anson & Forsberg, 1990; Kent, 1993), there remains much to be learned regarding what kinds of writing skills might carry over from context to context--particularly since we know that writing goals, practices, processes, expectations, and standards all differ across disciplines (Bazerman, 1988; Swales, 1998), genres and types of writing (Northey, 1990; Swales, 1990; Yates & Orlikowaki, 1992), and professional/organizational contexts (Driskill, 1989; Rogers, 2000). If one learns discourse practices only after actively participating in a community (Berkenkotter & Huckin, 1995; Suchan, 1998; Winsor, 1996), then what, if anything, can be done to assist writers transitioning into a new writing environment?

Questions regarding how we, as business communication faculty, might help new students move into the writing context of the MBA program confronted us in a very practical way when, in October 1994, the Analytical Writing Assessment (AWA) was added to the Graduate Management Admission Test (GMAT), the entrance examination for graduate business schools worldwide. For the AWA, individuals compose two short essays (30 minutes for each), one analyzing an issue, the other critiquing an argument. Under the administration of the Educational Testing Service (ETS), these essays are assessed holistically to produce a single overall AWA score (ranging from a low of 1 to a high of 6) for each test taker. Subsequently, the AWA scores and the actual essays are made available "to help business schools select the best applicants for admission" and to serve as "a diagnostic aid for determining whether prospective and accepted students need specific work to develop communication skills" (GMAC, n.d., p. 1). It was this latter ap plication of the test that concerned us as we considered whether the writing performance for the AWA could be used to help students enter the MBA writing environment.

As a diagnostic tool for MBA programs, the AWA has significant short-comings, problems which we discussed at length when the test was instituted (Rogers & Rymer, 1995a & b; 1996a & b). The holistic scores offer no indication of a writer's needs, and the essays themselves represent a decidedly different type of discourse than business students face in their MBA writing assignments.

This discrepancy in discourse type poses a real barrier for diagnostic use of the AWA. The essays represent a generic kind of analytical writing associated with academe, especially papers assigned in introductory humanities courses (Rogers and Rymer, 1995a & b). This kind of classroom writing--which some scholars call "essayist literacy"--is rational and decontextualized, with both writers and readers largely invisible players (Farr, 1993; Farr & Nardini, 1996). In contrast, MBA writing is highly contextualized and interactive. Business school training focuses on professional practices and practical matters involving employees, co-workers, customers, investors, and the wider community. Replicating business situations, for example, case assignments challenge students with the complexities of the managerial roles and tasks they will assume, confronting them with the possible impact of their decisions on the firm's profitability and on stakeholders in the case (Bridgeman & Carlson, 1984; Forman & Rymer, 1999a & b; Freedman, Adam, & Smart, 1994; Rogers & Rymer, 1995a & b; Russell, 1991). In addition to case analyses, a genre for which students must develop possible approaches to business situations, MBA students also role play case situations--for example, as an operations specialist proposing increased mechanization in baggage-handling procedures to increase customer satisfaction while decreasing costs. Also, many MBA students must complete internships and consulting projects, including all the oral and written deliverables expected for the specific company and its key constituencies.

So, all in all, despite some similarities as analytical writing tasks that can demonstrate critical thinking ability (Ackerman, 1991, 1993; Durst, 1987; Penrose, 1992; Rogers & Rymer, 1995a), the AWA essays and MBA writing represent quite different discourses. They inevitably tap some dissimilar skills, and writers' competence varies from one context to the next (Nystrand, 1989; Webster & Ammon, 1994; Witte & Cherry, 1994). Perhaps it is not surprising then that the diagnostic potential of the AWA remains largely untapped (Knight, 1999; Noll & Stowers, 1998).

By investigating the diagnostic potential of the AWA for MBA programs, we hoped to address this concern. Specifically, we hoped to explore the possibility of using the AWA essays to help new graduate students identify deficiencies that might interfere with their development into effective MBA writers. This research then led us to the more general question regarding transferable writing skills across genres, specifically, whether criteria for success on the decontextualized AWA essays might carry over to the MBA writing environment with its emphasis on various stakeholder audiences and communicative interaction.

To bridge these two contexts, we conducted a field study with MBA writing experts and students in two contrasting business schools. Our study involved identifying traits reflected in the AWA essays that the writing experts considered relevant to successful performance in MBA writing, describing these traits, and developing them into analytical tools for use in consultations with matriculating students. We subsequently tested these tools by applying two scoring methods and by conducting consultations with a sample of students using their AWA essays and the analytical tools. Students' post-assessment surveys and researchers' consultation notes suggest that the analytical tools help faculty and students identify and address deficiencies in the AWA essays that are relevant to MBA writing and, therefore, provide a way for MBA faculty to tap the enormous potential of the AWA, a writing test taken worldwide by thousands of students annually. In theoretical terms, the study's significance lies in the communicative a pproach it explores and the potential for developing analytical tools that may be relevant across different discourses, providing a way to use writers' performances in one context to facilitate their development in the practices of a quite different world of writing.

Literature Review and Theoretical Background

There was sufficient precedent to believe that tools based on an analytic scoring method could potentially be used to identify deficiencies in students' AWA essays that might inhibit their MBA writing. Rather than merely evaluating overall text quality like the holistic AWA score (GMAT Scoring Guides, 2001), analytic scoring methods evaluate specific factors or traits of writing (Purves, Gorman, & Takala, 1988; Wolcott with Legg, 1998). Based on a long tradition in both research and practice, analytic scoring was championed by ETS researcher Paul Diederich in his "Five Factors" for evaluating essay writing (Diederich, 1974; Diederich, French, & Carlton, 1961). In fact, a variety of analytic scoring schemes has been devised for many significant research projects, especially those determining what factors particularly affect overall quality in a holistic score (e.g., Connor, 1990; Connor & Lauer, 1985; Hoetker, 1982; Yeh, 1998). The foundation of some influential classroom instruction (Elbow, 1993), analytic s coring (though typically considered to be too expensive) has also been adapted for some large-scale assessments. The Vermont state assessment, for example, scores writing on purpose, organization, details, tone, and grammar (DeRemer, 1998). Closer to home, we were encouraged by the fact that analytical tools had been developed and successfully integrated into one of our business communication programs to score the undergraduate writing assessment, to assist MBA students in consultations, and to clarify concepts in business communication classes (Rogers, 1994; Rogers & Hildebrandt, 1993).