Succinct representation of source text gist is a critical component of good summary
writing performance. Given the importance of this task type in the academic domain
(Cumming, 2013; Leki & Carson, 1997; Rosenfeld et al., 2001) and the complexity of the
construct of summarization, it is imperative to provide L2 academic writers with
actionable feedback to facilitate learning (Boud & Molloy, 2013; Hattie & Timperlay,
2007; Lam, 2021). While the recent technological advancements offer a promising avenue
toward this direction, designing and developing feedback on source text content
representation in summarization is not a straightforward task. Previous L2 automated
writing evaluation research has focused primarily on corrective feedback, while available
theoretical and empirical works are scant concerning automated content feedback for
summary writing.
In this talk, the presenters will discuss a specific challenge faced in designing automated
content feedback for a web-based formative assessment module for summary writing in
introductory academic writing courses in Japan. Unlike written summaries elicited in
some existing automated content feedback programs, those in this module are shorter,
around 80 words. Since such summaries often comprise only a few sentences, the initial
step toward designing fine-grained content feedback is to identify an appropriate
within-sentence meaning unit that is easy to understand for learners. As candidates, the
team is currently comparing the idea unit (Kroll, 1977), often employed for qualitative
analyses of L2 recall and summary protocols, and the elementary discourse unit (EDU;
Carson & Marcu, 2001), originally developed in Rhetorical Structure Theory. The
comparison focuses on the reliability of automatic segmentation of summary protocols
and alignment of the obtained units to corresponding parts of the source text based on
semantic similarity for content feedback generation. Theoretical issues of consideration
and small-scale pilot results for comparing them between human coding and automated
algorithms will be discussed.
各種検索
サポート
ヘルプ