Relative Efficiency of Test Scores Equating Methods in the Comparison of Students Continuous Assessment Measures

Abstract

The purpose of this study was to ascertain the relative efficiency of test score equating methods in the comparison of students’ continuous assessment measures. Three equating methods (linear equating, separate calibration and concurrent calibration) based on classical test theory (CTT) and item response theory (IRT) frameworks were studied. The design of the study was “Non-Equivalent Anchor Test (NEAT) group Design” and the area of the study was Cross River (state A) and Rivers states (state B). The population of the study comprised of all senior secondary three (SSIII) students of 2010/2011 academic session in both states, and a sample of 2,905 students was drawn through multi-stage sampling procedure from the population. The instrument used for data collection were two parallel forms of Mathematics Achievement Test (MAT) with reliability of 0.83 and 0.89 respectively. The instruments were designed in such a way that students’ scores from mathematics continuous assessment (MCA) were also obtained with it. Eleven research questions and six hypotheses guided the study. The research questions were answered using descriptive statistics, IRT differential item functioning likelihood ratio (IRTDIFLR) for three parameter logistic model and item characteristic curve (ICC). The six hypotheses were tested at 0.05 level of significance using independent t-test, chi-square statistic and Pearson product moment correlation coefficient. The data collected with MAT were analysed using BILOG-MG and SPSS. Major findings of the study showed that; (1) the average root mean square error (ARMSE) obtained were 0.09, 0.05 and 0.04 for separate calibration, concurrent calibration and linear equating respectively. This ARMSED values indicate that linear equating yielded the least error and therefore seems to be more efficient in this study,(2) there was no significant difference in the ability estimates of students in state A and B when their scores are scaled through separate calibration. (3) there was no significant difference in the ability estimates of students in state A and B when their scores are scaled through concurrent calibration. (4) there was a significant difference in the ability estimates of students in state A and B for MCA and MAT scores equated through linear equating. Other findings of the study include; (5) there was no significant difference in the item parameter estimates of the two forms of mathematics achievement test (MAT) used for test equating, (6) The items show negligible differential item functioning across states, sex and ability. (7) six items representing 15% did not fit the three parameter logistic model (3PLM) whereas 34 items representing 85% of the total test were not statistically significant and therefore fit the three parameter logistic model in both tests, (8) all the item characteristics curve (ICC) of the items in both tests were steep and vertically shifted towards the right corner of the curves except for item 1 and 2 in test A1 whose ICC were slightly flat, (9)there was a significant relationship between the performances of students in mathematics continuous assessment test and mathematics achievement test. Based on these findings, it was recommended among others that linear equating method should be used to standardize students’ continuous assessment (CA) scores. Score assigned to students’ responses for every cognitive based continuous assessment should be reported in person-by-item response pattern. This will permit better CTT or IRT analysis to be performed.