Efficacy of a web-based training andmonitoring procedure in scoring performance tasks
Document Type
Presentation
Journal/Book Title/Conference
Annual meeting of the American Educational Research Association
Publication Date
4-1-2007
Abstract
The study focuses on the effectiveness of the training procedure including web-based qualifying tests and the monitoring procedure using web-based refresher tests in rating the South Carolina Arts Assessment Program (SCAAP) Grade 4 music performances tasks. Five rater groups, comprised of two raters each, scored both performance tasks. Generalizability (G) theory was used to estimate inter-rater reliability. The researchers concluded that the training procedure and the monitoring procedure with four refresher tests were effective in establishing satisfactory inter-rater reliability estimates for each task. In summary, the researchers believe that a remote rating system is effective when sufficient training and stringent monitoring procedure are employed.
Recommended Citation
Yap, C., Lewis, A., & Feldon, D. F. (2007). Efficacy of a web-based training and monitoring procedure in scoring performance tasks. Presented at the Annual Meeting of the American Education Research Association. Chicago, Illinois: April, 2007.
