초등 영어 말하기·쓰기 평가에서 총체적 및 분석적 채점 방식이 채점 신뢰도와 채점자 엄격도에 미치는 영향

초록

This study investigated the reliability and rater severity differences between holistic and analytic scoring methods in elementary English speaking and writing assessments. Eighteen pre-service elementary teachers majoring in English education participated as raters. They evaluated authentic speaking and writing tasks performed by elementary students, using holistic and analytic rubrics. After undergoing rater training, participants scored students’ responses under both scoring conditions. To analyze inter-rater reliability and rater severity, Intraclass Correlation Coefficients (ICC) and Rating Scale Many-Facet Rasch Models (RS-MFRM) were employed. The results revealed that analytic scoring demonstrated higher reliability in both speaking (ICC = .91) and writing (ICC = .83) than holistic scoring, with the exception of speaking, where holistic scoring also showed a high reliability (ICC = .91). Analytic scoring tended to produce slightly higher average scores, indicating more lenient judgments. RS-MFRM analysis further showed that rater severity varied more widely in analytic scoring for writing, while holistic scoring showed greater differentiation among raters for speaking. Based on the findings, this study asserts that teacher training should move beyond one-off sessions in order to ensure reliable outcomes in classroom-based performance evaluations. The study also calls for more advanced assessment literacy education, incorporating data-driven monitoring of rater bias and continuous calibration practices.

키워드