A Study of CSAT English Tests as Criterion-referenced Tests: With Regard to Lexical Richness

초록

In this paper I noted that in terms of the number of test-takers assigned grades, 2017 CSAT and 2018 CSAT are significantly different. As an effort to explain this difference, I compared the two tests from the viewpoint of lexical sophistication, lexical density, and lexical diversity. Chi-square tests showed that the reading tests of the two CSATs are not different in lexical sophistication and density, while the listening tests are different only in lexical sophistication. Even though there is no way of guarantee that the two tests do not differ in lexical diversity, tiny differences between the measurements of lexical diversity in these tests strongly suggest that they do not differ. All these findings then suggest one thing: the difference between 2017 and 2018 CSATs is not lexical. Maybe, the test-takers of both tests are different in terms of English proficiency or the difference in test item difficulty is due to other variables like syntactic complexity, text complexity, the plausibility of incorrect options, etc.