1. Does Criterion help students improve their writing? Hank Delcore Jennifer L. Ivie

2. Automatically-Generated Data Students who made two submissions to ETS Criterion had higher average holistic scores than students who only made one submission. Students with lower initial holistic scores were able to raise their scores, with subsequent submissions, to the same level as those whose initial holistic scores were higher. Intervening variables?

3. Faculty Scoring Panel Overall, student writing improved in classes that used ETS Criterion. The greatest improvements from first to last submission were made in classes in which the instructors engaged students in substantial discussions of writing and how to use Criterion. Intervening variables?

4. Music 187 Control group study across three sections of Music 187 (Jazz and Rock), an IC course. Group 1: no Criterion Group 2: required 2-4 submissions Group 3: unlimited submissions Group 2 has significantly higher instructor-assigned grades than Group 1. Groups 1 and 3 had no significant difference in grades.

5. Collegiate Learning Assessment Asignificant difference was found between Criterion users and Criterion non-users on the CLA performance task. No significant differences were found between the two groups on any of the analytic writing task scores.

6. Faculty/Student Survey Highlights Students felt that Criterion …helps with writing mechanics. …did not help them improve their writing skills. But, ESL students were more likely than English as First Language students to respond that Criterion helped them improve their writing. And, they ESL students preferred Criterion feedback to instructor feedback. Faculty felt that Criterion …did not reduce their workload. …did not do as much as they had hoped. …made grading simpler. …encouraged students to spend more time improving their writing. … overall improved student writing.

7. Faculty Learning Community Faculty commented positively on …the ease and convenience of using ETS Criterion. …the added ability it gave them to focus on course content rather than grammar and mechanics. …the fact that students using Criterion were more engaged with the writing process. Drawbacks included …the inability of the program to recognize technical terms or subject-specific writing conventions. …perceived inaccuracies in holistic scoring. …lack of compatibility with TurnItIn and Blackboard.

8. Conclusion Some evidence that Criterion helps students improve writing, likely in combination with good instructor practices. Criterion seems to help most with mechanics. Convergence of survey and FLC data: faculty agree that students who use Criterion are more engaged with the writing process. Next steps…