An Investigation on the Effect of the Difficulty of an Item in A Multiple Choice Examination When Item Choices are Rearrange

Main Article Content

Jonald Pimentel
Marah Luriely A. Villaruz

Abstract

A study was conducted to know if rearranging item choices in an item of a multiple choice test affects the behavior of the examinees the way they look at the item difficulty of the given items. Two sets of test instruments (pretest and modified posttest) containing fifteen items were made with the same items but item choices for the modified posttest instrument were rearranged. Among the 205 examinees who took the test during a two-week time interval, their responses were modeled using the Rasch model.  Results show that the estimates of the item difficulties for the majority of the fifteen items between the two tests were different. Majority of the items given in the exam showed an increase in the difficulty level as viewed by the examinees. The effect on the difficulty maybe due to the time interval the two sets of test were administered, that is first, students forget what they learned and see the items as difficult (time factor) and second, the rearrangement of the choices in each items in the post test affected the student’s way they see in dealing the items of the test which partly contributed to the increase of the level of difficulty of majority of the items in the test.

Article Details

How to Cite
Pimentel, J., & Villaruz, M. L. A. (2021). An Investigation on the Effect of the Difficulty of an Item in A Multiple Choice Examination When Item Choices are Rearrange. Advances in Social Sciences Research Journal, 8(6), 398–407. https://doi.org/10.14738/assrj.86.10417
Section
Articles

References

Van der Linden, W.J., & Hambleton, R.K. (Eds.), Handbook of modern item response theory ,1997. New York: Springer Verlag.

Pimentel, J.L., Item Response Theory Modeling with Nonignorable Missing Data, 2005. University of Twente, The Netherlands.

De Boeck, P., & Wilson, M. (Eds.), Explanatory item response models: A generalized linear and nonlinear approach, 2004. New

York: Springer.

Hardouin J.B., Mesbah M., “The SAS macro-program %AnaQol to estimate the parameters of item response theory models” C

Communications in Statistics – Simulation and Computation, 2007.

Skrondal, A. Rabe-Hesketh, S., Generalized Latent Variable Modeling: Multilevel, Longitudinal, and Structural Equation Models,

Rizopoulos, D., ltm: An R Package for latent variable modeling and item response analysis. Journal of Statistical Software, 2006,

(5).

Rasch, G., Probabilistic models for some intelligence and attainment tests. , 1960.Copenhagen: Danish Institute for Educational

Research.

Molenaar, I.W., Estimation of item parameters. In G.H. Fischer, & I.W. Molenaar (Eds.), Rasch models: foundations, recent

developments and applications, 1995. New York, NJ: Springer.

Bond, T.G., & Fox, C.M., Applying the Rasch Model: Fundamental measurement in the human sciences, 2001.2nd ed.

Birnbaum, A., Some latent trait models. In F.M. Lord & M.R. Novick (Eds.), Statistical theories of mental test scores,1968 (pp.395-

. Reading, MA: Addison-Wesley.

Bock, R.D., Estimating item parameters and latent ability when responses are scored in two or more nominal categories, 1972.

Psychometrika, 37.

Samejima, F., Estimation of latent ability using a response pattern of graded scores. Psychometrika Monograph

Supplement,1969.

Masters, G.N., A Rasch model for partial credit scoring,1982 Psychometrika, 47, No.2.

McDonald, R.P., Nonlinear factor analysis, 1967. Psychometric monographs, No. 15.

Lord, F.M., & Novick, M.R., Statistical theories of mental test scores,1968. Reading: Addison-Wesley.

Pimentel, J.L. & Villaruz, M.L.A., Comparison of Item Difficulty Estimates in a Basic Statistics Test using ltm and CTT Software

Packages in R, 2020. International Journal of Advanced Computer Science and Applications (IJACSA). Volume 11. No. 3 March

pp.367-372.

Zimowski, M.F., Muraki, E., Mislevy, R.J., & Bock, R.D., Bilog MG: Multiple-group IRT analysis and test maintenance for binary

items,1996. Chicago: Scientific Software International Inc.

Wilson, D.T., Wood, R., & Gibbons, R.D., TESTFACT: Test scoring, item statistics, and item factor analysis (computer

software),1991. Chicago: Scientific Software International, Inc.

Muthen, L. K., & Muthen, B. O., MPLUS: The comprehensive modeling program for applied researcher, users guide,1998. Los

Angeles, CA.

Verhelst, N.D., Glas, C.A.W., & Verstralen, H.H.F.M., OPLM: computer program and manual,1995 Arnhem: Cito, the National

Institute for Educational Measurement, the Netherlands.

Wu, M.L., Adams, R.J. & Wilson, M.R., ConQuest: Generalised item response modelling software, 1997. Draft release 2. Australian

Council for Educational Research.

Mair,P. and Hatzinger, R., Extended Rasch Modeling: The eRm Package for the Application of IRT Models in R, 2007. Journal of

Statistical Software, vol. 20 no.9.

Villaruz, M.L.A., Test Item Calibration for Multiple Choice Test Using IRT, 2015. Unpublished Undergraduate Thesis. MSU-IIT,

Iligan City, Philippines.

Fischer, G.H., Einf¸hrung in die theorie psychologischertests introduction to the theory of psychological tests,1974. Bern: Huber.

Fischer, G.H., Derivations of the Rasch model. In G.H. Fischer & I.W. Molenaar (Eds.), Rasch models: foundations, recent

developments and applications,1995. (pp.39-52). New York, NJ: Springer.

Embretson, S.E., & Reise, S.P., Item response theory for psychologists, 2000. Mahwah, NJ, Lawrence Erlbaum.

Fischer, G.H., & Molenaar I.W., Rasch models. Their foundation, recent developments and applications,1995. New York, NJ:

Springer.

Baker, F. B., Item response theory: Parameter estimation techniques,1992. New York, NJ: Dekker.

Hambleton, R.K., Swaminathan, H., and Rogers, H.J., Fundamentals of item response theory,1991. Newbury Park, CA: Sage.

Andersen, E.B., A goodness of fit for test for the Rasch model,1973. Psychometrika.

Pimentel, J.L. & Enock, O.U., The effect of Retention Interval Over a Three Year Period on the Elementary Algebra Performance of

High School Students, 2019. International Journal of Sciences: Basic and Applied Research (IJSBAR). Volume 43. No.2, pp 181-