This paper describes an ongoing project to create an online version
of a reading programme, a custom-designed English language proficiency
course at a university in Japan. Following an interactionist view of
second language acquisition, it was hypothesised that comprehension of a
reading passage could be enhanced by online materials promoting
interaction between students as they completed a multiple-choice reading
comprehension exercise. Interaction was promoted: (a) through pair work
at a single computer and (b) by providing Elaborative feedback in the
form of hints about incorrect answers as a means of stimulating
discussion about corrections. Students were randomly selected from upper
and lower levels of English proficiency, as determined by the Kanda
English Proficiency Test (Bonk & Ockey, 2003), to receive either
Elaborative feedback or Knowledge of Correct Response feedback (which
supplies the correct answers). Within these groups, some students worked
in pairs and some alone. Quantitative results show that the interaction
between Type of feedback and Manner of study (individual or pair work)
was statistically significant; students performed best on a follow-up
comprehension exercise when in pairs and having been provided with
Elaborative feedback. Furthermore, qualitative analysis of transcribed
interactions also shows that Elaborative feedback was conducive to
quality interaction.
INTRODUCTION
Advancing the design and use of computer-assisted language learning
(CALL) activities is a key concern for researchers. As Chapelle (1997,
pp. 19-22) explains, critical questions need to be answered about how
CALL can be used to improve instructed second language acquisition
(SLA). Two such questions are:
1) What kind of language does the learner engage in during a CALL
activity?
2) How good is the language experience in CALL for L2 learning?
Chapelle (1997) describes how the answer to the second question is
dependent upon beliefs concerning
what types of language use are expected to be beneficial for second
language development. For those espousing an interactionist view of SLA
(Lantolf, 2000; Lightbown & Spada, 1999; Long, 1981; Pica, 1996; Van
Lier, 1996), there is an assumption that L2 acquisition is facilitated
by learners' interaction in the target language, thereby providing
opportunities to comprehend message meaning. Accordingly, to ensure that
L2 tasks meet such assumptions, and to facilitate SLA, researchers need
to specify ideal observable features of learner language, such as
signals that focus attention on language and features that may elicit a
repetition or an expansion of previously acquired language.
In line with Chapelle's recommendations, a key concern for
research is how these ideal features and appropriate tasks can be
incorporated into an experimental reading programme. This concern is
relevant due to the two goals of the current course, namely:
1) to provide students with the choice of an alternative and
principled mode of online study and
2) to promote learner autonomy (Benson, 2001).
Throughout this paper, focus is placed on the first exercise that
the students meet in the course, a reading comprehension exercise. It
was hypothesised that increased interaction could be facilitated by
requiring students to collaborate in pairs at a single computer (Beatty
& Nunan, 2004; Stevens, 1992), and by providing Elaborative feedback
in the form of hints to promote discussion as students self-correct
errors. This type of feedback was provided as an alternative to
Knowledge of Correct Response (KCR) feedback, which replicates
traditional paper-based answer sheets by providing correct answers. It
was also hypothesised that increased interaction through pair work with
Elaborative feedback would be an effective method for promoting
comprehension of a reading text. Results are analysed both
quantitatively and qualitatively.
Context of the Study
During the second term of the reading course (see Murphy &
Imrie, 2003 for a description), students are encouraged to choose from a
series of activities and create, with guidance, an individualized
syllabus. Students select, complete and then check their answers to the
exercises with answer papers provided by the teachers. However, this
procedure has proven to be problematic on the paper-based course for the
following reasons:
1) when correcting answers, it is uncertain whether students: a)
fully understand their errors and b) actively engage in the process of
self-correction and,
2) for those students who choose to work outside of lesson time,
there is potentially a wait of up to one week between lessons (and
longer during holidays) before they can check their answers.
In a bid to overcome these challenges, this research focuses on the
contribution that computer-mediated feedback can make. A key question
that arises is: how and what kind of feedback maximizes comprehension?
It is towards this issue that the following discussion is directed.
INTERACTION IN THE READING PROCESS
CALL researchers have turned to the work of interactionist SLA
researchers when evaluating the quality of learner language. As Chapelle
(1997) explains, the linguistic form of: "... a good interaction is
hypothesized to occur when the normal interactional structure has been
modified because the learner has requested, for example, a repetition,
clarification or restatement of the original input" (pp. 25-26).
This modified interaction is thought to be good because it can function
to promote both the negotiation of meaning of the input (Beatty, 2003;
Chapelle, 2001; Long, 1985; Nunan, 1993; Pica, 1994) and greatly
contribute to language acquisition (Ellis, 1998; Krashen, 1985; Van den
Branden, 2000). From a reading proficiency perspective, Larsen-Freeman
and Long (1991) note:
Modification of the interactional structure of conversation or
of written discourse during reading ... is a [good] candidate
for a necessary (not sufficient) condition for acquisition.
The role it plays in negotiation for meaning helps to make
input comprehensible while still containing unknown linguistic
elements, and, hence, potential intake for acquisition. (p.
144)
Following research that points to the importance of comprehensible
output to the acquisition of the target language (Chapelle, 1997; Swain,
1985), a conscious effort was made in this study to investigate the
effects of feedback designed to promote negotiation of meaning, form and
/ or content in situations similar to those described by Swain and
Lapkin (1995):
In producing the L2, a learner will on occasion
become aware of (i.e., notice) a linguistic
problem (brought to his / her attention either
by external feedback [e.g., clarification
requests] or internal feedback). Noticing a
problem 'pushes' the learner to modify his / her
output. (p. 373)
Although the importance of both negotiation of meaning and
comprehensible output is well documented, few studies have investigated
the effects on reading comprehension (Van den Branden, 2000);
nevertheless, the design of this study was informed by research that was
available and specifically by studies that point to the usefulness of
promoting reading proficiency through interaction (Grabe & Stoller,
2002; Shanker & Ekwall, 2003). Despite the fact that the studies
such as Eldredge and Butterfield (1986), Koskienen and Blum (1986) and
Nes (2003) were carried out in non-computer-mediated environments, they
provide positive implications for promoting interaction through paired
online reading activities.
Quality Student Interaction Around Computers
As with non-computer-mediated environments, it is important to
consider the interaction that is generated in computer-based tasks
(Beatty, 2003; Stevens, 1992), and the type of interaction that is
desirable for promoting comprehension, learning and language acquisition
around computers. Based on findings from Fisher's (1992) study,
students working on tutorial software exhibited the same IRF
(Initiation, Response, Follow-up / Feedback) discursive structure.
However, researchers have attempted to increase levels of interaction
between students in various ways. For example, Wegerif and Mercer (1996)
proposed a transformation to an IDRF (Initiation, Discussion, Response,
Follow-up / Feedback) structure by including a discussion stage.
Furthermore, software can also be developed to replicate techniques
which teachers use to stimulate interaction, notably: (a) eliciting
knowledge from students, (b) responding to what students say
(confirmations, repetitions, elaborations and reformations) and (c)
describing significant aspects of shared experiences ('we'
statements) (Mercer, 2004).
When considering the quality of interaction around computers, two
key features are particularly desirable: (1) learners need to be
actively involved (Van den Branden, 2000); and (2) learners need to
produce Exploratory talk in which partners engage critically and
constructively with each other's ideas (Mercer, 1995). Regarding
the former, Mercer (2004) explains how it is helpful for the analyst to
perceive the degree in which students in joint activities are: "(a)
behaving cooperatively or competitively and (b) engaging in the critical
reflection or in the mutual acceptance of ideas" (p. 146). As for
promoting Exploratory talk among learners, Wegerif, Mercer, & Dawes
(1998), having been influenced by findings of research into effective
collaborative learning (summarized in Wegerif & Mercer, 1996), note
the importance of: sharing relevant information, reaching agreement,
expecting reasons and challenges, discussing alternatives and
encouraging peers. A key concern for research, however, is how quality
interaction and reading comprehension can be promoted through
computer-based activities.
Reading, Computers and the Internet
Despite the fact that models and guidelines recommending
pedagogically sound practices for incorporating Internet-based materials
exist (Berry, 2000; Brandl, 2002; Chun & Plass, 2000), a major
concern is that the number of such examples remains limited. Likewise,
guidelines for offering a reading course via the Internet (Caverly &
McConald, 1998; Jones & Wolf, 2001; Mikulecky, 1998) are similarly
few. However, evidence exists to support the assumption that integrating
reading with computer-mediated support improves ESL students'
reading skills (Chun & Plass, 1996; Hong, 1997; Stakhnevich, 2002;
Williams & Williams, 2000). A common theme in studies such as these
is that learners benefit from facilities offering support and assistance
in web learning environments, for instance, from online dictionaries,
glosses, graphics, blogs, bulletin boards or chat rooms. As a further
example of a potentially advantageous facility that can also be offered
through computer-assisted language learning (CALL), the following
section comprises a discussion of the merits of computer-mediated
feedback.
COMPUTER-MEDIATED FEEDBACK
One area in which computers are playing an increasingly important
role in SLA concerns the identification of students' errors and the
subsequent provision of appropriate feedback (Brandl, 1995; Clark &
Dwyer, 1998; Tsutsui, 2004). However, many software products opt for a
generic form of feedback and rarely go above the level of indicating
whether an answer to a question is correct or incorrect (Sales, 1993).
Beatty (2003) explains:
A CALL program is likely implicitly to state
'I have the answers to your questions; just
click here.' A teacher is more likely to say,
'What do you think the answer might be?' or
'Why do you ask this question?' ... it is
difficult for computers to deal with ambiguous
learner input, but this is an area of research
that needs to be further investigated. (p. 138)
Clariana (2000), who has published extensively on the topics of
computer-mediated feedback, provides a succinct summary of the
traditionally investigated types of feedback in CALL:
Knowledge of response (KR) that states "right" or
"wrong" or otherwise tells learners whether their response is
correct or incorrect; Knowledge of correct response (KCR) that states or
indicates the correct response; and Elaborative feedback that includes
several more complex forms of feedback that explains, directs, or
monitors (Smith, 1988). Elaborative feedback includes the forms listed
below:
1) Explanatory feedback provides additional explanations, such as
why a learner's error response is incorrect or perhaps why a
correct response is correct and various types of additional remedial
screens that may amount to new instruction (Merrill, 1985, 1987; Spock,
1987).
2) Directive feedback may provide prompts, hints, or cues to assist
the learner in determining the correct response (Nielson, 1990). Answer
until correct (AUC, Pressey, 1926) is a common form of elaborative
feedback where the learner is directed to respond until correct.
3) Monitoring feedback, also referred to as advisement, lets the
learner know how they are doing overall. (p.1)
As can be seen, several forms of feedback exist. Accordingly,
numerous researchers have tried to identify the most effective forms in
different contexts. This issue is addressed in the next section.
How Effective is Feedback?
It is difficult to say which type of feedback is best for SLA as
results are mixed (Clariana, 2000; Mory 1994). For example, summarizing
findings by Bangert-Drowns, Kulik, Kulik, and Morgan (1991), AUC and
Elaborative feedback are considered to be the most effective:
KR < no feedback < KCR < AUC = Elaborative feedback
However, following a review of 30 studies, Clariana's (1993)
findings, which are consistent with both Schimmel's (1983)
meta-analysis of 15 studies and also Kulhavy and Wager's (1993)
research, show feedback has proven more effective than no feedback:
No feedback < KR = KCR = Directive feedback = Multiple try feedback
(AUC)
In addition to investigating the most effective type of feedback,
various other aspects of computermediated feedback have also been the
focus of research. For example:
1) Clariana and Koul (2006) investigate the effects of multiple-try
immediate feedback for questions differing in difficulty
2) Clariana and Koul (2005) compare multiple-try feedback to other
types of feedback also for questions of differing difficulty
3) Clariana (2003) and Clariana and Lee (2001) investigate the
effects of recognition (multiple-choice) and recall
(constructed-response) study tasks with feedback
4) Clariana, Wagner and Murphy (2000) consider the timing of
feedback and
5) Nagata (1996) and Mory (1994) investigate the effects of
Elaborative feedback.
A key question is whether computer-based feedback can offer
advantages over traditional paper-based answer papers. Of particular
interest, therefore, is the fact that Nagata (1996) found ongoing
intelligent computer feedback to be more effective than simple workbook
answer sheets for developing grammatical skill in producing Japanese
particles and sentences. However, Clariana (2000, p. 2) draws on
research to show how, in contrast to Nagata (1996) and Bangert-Drowns et
al. (1991), Elaborative forms of feedback often produce no significant
improvement over KR feedback despite requiring considerable development
and implementation cost (Merrill, 1987). Nevertheless, Ferris (2003)
explains how indirect feedback, or Elaborative feedback from a CALL
perspective, is generally thought to be conducive to long-term student
development; it forces students to think about their own errors and
self-correction, thereby leading to: " ... increased student
engagement and attention to forms and problems" (p. 52). Indeed, de
Bot (1996) explains how students need to be active when producing
language to discover what they can and cannot do. Noticing a problem,
possibly through feedback, may be the incentive learners need to
reengage with information in the input, thereby providing an opportunity
for learning. Therefore, by placing the onus on the students to identify
and correct their own errors, it would seem that the potential for
interaction and negotiation of meaning, form and / or content is
increased.
THE NEED FOR RESEARCH
When providing feedback, what messages should be supplied to the
students on the screen? If feedback is presented in the form of KR or
KCR feedback, it is simple to envisage what is displayed (a message such
as right or wrong, a highlighted answer or a mark indicating the correct
answer); however, with Elaborative feedback, what is it exactly that
should be displayed to promote interaction and comprehension? This is
the crucial question for anyone creating software to provide such
feedback. Among the extensive literature related to the theme of
feedback, the number of studies researching Elaborative feedback is
relatively small; however, notable examples exist (Brandl, 1995; Clark
and Dwyer, 1998; Mory, 1994; Nagata, 1996; Van der Linden, 1993). Even
more difficult to find are examples of guidelines for its presentation.
For example, Van der Linden (1993) notes: "Long feedback (exceeding
three lines) is not read and for that reason is not useful" (p.
56). Van der Linden concludes that: "... feedback, in order to be
consulted, has to be concise and precise." Mory (1994) recommends
that isolated feedback should be avoided as it may provide little
context for revision of an erroneous response. Consequently, Mory
advises designers to: "... include the learner's answer and
other alternative choices on the same screen as the feedback" (p.
287). As Chapelle (2001) explains, therefore, further research is vital:
"What is needed are theoretically and empirically based criteria
for choosing among the potential design options and methods for
evaluating their effectiveness for promoting learners'
communicative L2 ability" (p. 2). Accordingly, the following study
comprises an investigation into the effectiveness of Elaborative
feedback with the aim of identifying guidelines for researchers creating
software to provide such feedback.
Research hypotheses
The literature shows that feedback has the potential to promote
comprehension of a reading text; however, as to which type of feedback
is more effective, results have been varied. The literature also shows
that interaction between students can promote comprehension; however,
both (a) what kind of interaction is generated through pair work as a
result of Elaborative feedback, and (b) whether the interaction is
sufficient to promote comprehension need investigating. The effect of a
student's English proficiency level also needs to be determined.
Stemming from this discussion, the following hypotheses were formed:
Hypothesis 1: Elaborative feedback will be more effective for
promoting comprehension of the reading text than KCR feedback.
Hypothesis 2: Pair work will be more effective for promoting
comprehension of the reading text than individual work.
Hypothesis 3: Students with a higher level of English proficiency
will demonstrate higher levels of comprehension of the reading text than
those with a lower level.
Hypothesis 4: Students studying in pairs and receiving Elaborative
feedback will demonstrate higher levels of comprehension of the reading
text than other students.
Hypothesis 5: Students with higher proficiency receiving
Elaborative feedback will demonstrate higher levels of comprehension of
the reading text than other students.
Hypothesis 6: Students with higher proficiency studying in pairs
will demonstrate higher levels of comprehension of the reading text than
other students.
Hypothesis 7: Students with higher proficiency studying in pairs
and receiving Elaborative feedback will demonstrate higher levels of
comprehension of the reading text than other students.
RESEARCH METHODOLOGY
Participants
The participants are first-year English majors at Kanda University
of International Studies in Japan, who are streamed according to a test
of global proficiency (Bonk and Ockey, 2003). In the 2005-6 academic
year, 407 students (15 classes) were assigned to one of four bands:
advanced (three classes), upper intermediate (four classes),
intermediate (four classes) or lower intermediate (four classes). For
the purpose of this study, the top two bands were grouped together and
the bottom two bands were also grouped together.
Materials developed for this study were trialed with 162 of the 407
first-year students. 14 others were late or absent for lessons, and the
main study was conducted with the remaining 231 students. Six of these
231 students (two pairs from the Elaborative feedback condition and one
pair from the KCR feedback condition) scored 100% on the first
comprehension exercise on the first attempt. Due to the fact that these
students made no errors, and, therefore, received no feedback to promote
comprehension (the focus of this research), these records were omitted
from the study. Therefore, the statistical data analysis was performed
on the data from the remaining 225 students. Videos were recorded of 12
volunteer students (six pairs) as follows: (a) four pairs were recorded
during the trials and (b) two pairs were recorded during the main study.
From analysis of the transcripts of these recordings, qualitative data
is used to support / reject the various research hypotheses. All
students agreed to participate in the study.
Materials: Reading Materials
Materials comprised one reading text (see Appendix A for an excerpt
of the text) and two multiple-choice comprehension exercises, each with
15 questions (see Appendices B and C for example questions). Therefore,
the maximum score on each exercise was 15 points. While the questions in
the two comprehension exercises were different, the same content points
were covered by corresponding questions.
Materials: Feedback Treatment
The methodology for displaying feedback is based upon the results
of a study undertaken by Murphy (2005). Research was conducted into
identifying the types of errors that students made in response to
multiple-choice comprehension questions about a reading text and how
students changed their answers following Elaborative feedback with an
Answer-until-correct methodology. Students were allowed to change their
answers until they answered correctly, until they gave up or until they
ran out of time. Each question was associated with one piece of
Elaborative feedback. Whenever a question was answered incorrectly, the
corresponding Elaborative feedback was supplied irrespective of how many
times the answers had been checked. Based upon interviews with students
and an analysis of the way in which answers were changed as a result of
feedback, recommendations for displaying Elaborative feedback are
summarised as follows:
* Students should answer all questions before receiving any
feedback
* Students should be allowed to get the computer to check their
answers a maximum of four times by clicking on a 'Check
answers' button on the screen at the end of the exercise.
* After checking answers, Elaborative feedback should be provided
for each of the errors. If necessary, there should be up to three rounds
of different Elaborative feedback before the KCR feedback. For those who
make errors, the opportunity for random guessing of answers is minimised
by not identifying incorrectly answered questions by question number;
instead, students are encouraged to read the feedback, reengage with the
materials, locate any errors by themselves and then change any answers
they feel are incorrect. Furthermore, in order not to identify errors by
question number, students are not supplied the correct answers until
they either finish the exercise by answering every question correctly or
receive the KCR feedback after the three rounds of Elaborative feedback.
* Following the first check of answers, the first round of
Elaborative feedback should address the issue of silly errors such as
selecting an incorrect answer by mistake. In this situation, students
may not need a detailed explanation of the error if they are already
able to comprehend. Instead, they just need an indication that an error
has been made. Therefore, Elaborative feedback for each error should
only direct students back to a key area in the text, for example: Please
have