Book: Scoring Second Language Spoken and Written Performance
Chapter: Understanding Raters and Ratings
Blurb:
In Chapter 1, we start by introducing the topic of raters and ratings. We provide a visual model that shows the complexity of the factors that influence rating quality in rater-mediated assessments, as well as some issues that are relevant to automated scoring. This model also provides an overview of many of the issues we introduce in the book. We then discuss the various rater effects that have been described in the literature and explore factors that may affect rating quality.