Comparison of two test methods for VIS: paper-pencil test and CAT

Abstract

This paper examines listening comprehension skills of visually impaired students (VIS) using computerised adaptive testing (CAT) and reader-assisted paper-pencil testing (raPPT) and student views about them. Explanatory mixed method design was used in this study. Sample is comprised of 51 VIS, in 7th and 8th grades. 9 of these students were interviewed for determining student views about tests. Results indicated that scores obtained from CAT are significantly lower than scores obtained from raPPT. Additionally, a positive and high correlation was found between scores of CAT and raPPT. This result suggests that similar ability estimations were made by CAT and raPPT. Another finding is CAT made more reliable predictions, and was completed in shorter duration using fewer items. In qualitative part, student views were gathered through interviews and content analysis revealed three themes as technical features, test features, and psychological effects. In general, students reported positive views about CAT. VIS prefer CAT due to its listening/control options, shorter test durations, clarity of reading, and fairness of test, elimination of dependency to reader. Study provides implications for test developers and test-users to consider CAT as a multi-accommodation for VIS through its advantages.