Empirical Investigations that Supported the Development of OpenEssayist: A Tool for Drafting Academic Essays Denise Whitelock 1st International Workshop on Technology-Enhanced Assessment, Analytics and Feedback (TEAAF2014)

1. EMPIRICAL INVESTIGATIONS THAT SUPPORTED THE DEVELOPMENT OF
OPENESSAYIST: A TOOL FOR DRAFTING ACADEMIC ESSAYS
Denise Whitelock, The Open University,
Abstract
Universities continuously strive to offer the best learning experience for their students, and to
support them toward achieving their academic goals. In an era where university students are
commonly enrolling onto courses with very large numbers, often studying, researching and
submitting at least parts of their courses online, the methods by which universities can offer support
to their students are constantly evolving. The ideal is to provide students with a personalised, just in
time service which will not be a content specific application. In many ways this is one of the
challenges for educators to provide feedback data in a form that can be readily understood by the
user themselves hence by-passing the tutor.
The SAFeSEA project (Supportive Automated Feedback for Short Essay Answers) set out to assist
students in writing draft essays. The project explored a number of feedback mechanisms to facilitate
this process. One such mechanism was investigating how to offer support regarding essay structure
based on the premise that a good essay is like a good story: it needs a beginning, middle and end.
The components are also well conducted and the middle section provides the evidence for the
argument proposed throughout the essay. We undertook a number of empirical investigations, one
of which provided hints to students before they wrote the essay and another where feedback was
given afterwards. Both feed-forward and feed-back had a statistically significant effect on students’
essays marks.
The essential features for writing a good essay which were investigated with the empirical studies
were incorporated into a computerised system (OpenEssayist) that provided students with
automated feedback on their draft essays. OpenEssayist operates through the combination of a
linguistic analysis engine which processes the text in the essay, and a web application that uses the
output of the linguistic analysis engine to generate the feedback.
This presentation will outline the system itself and present analysis of observed patterns of activity
while students engaged with and explored the system for their module assignments. There was a
significant positive correlation between the number of drafts submitted to the system and the
grades awarded for the assignments. The students also gained significantly higher grades for the
course overall than the students in the previous academic year. Automated formative assessment
tools are one way to provide students with just in-time feedback which can provide personalised
support.