Mokken Scale Analysis: Discussion and Application
DOI:
https://doi.org/10.14738/assrj.83.9949Keywords:
Mokken scale analysis, item response theory, statistics anxiety, R studioAbstract
This article presents a discussion and illustration of Mokken scale analysis (MSA), a nonparametric form of item response theory (IRT), in relation to common IRT models such as Rasch and Guttman scaling. The procedure can be used for dichotomous and ordinal polytomous data commonly used with questionnaires. The assumptions of MSA are discussed as well as characteristics that differentiate a Mokken scale from a Guttman scale. MSA is illustrated using the mokken package with R Studio and a data set that included over 3,340 responses to a modified version of the Statistical Anxiety Rating Scale. Issues addressed in the illustration include monotonicity, scalability, and invariant ordering. The R script for the illustration is included.
References
[1]. Embretson, S. E., & Reise, S. P. (2000). Item response theory for psychologists. Mahwah, NJ: Lawrence Erlbaum Associates.
[2]. De Champlain, A. F. (2010). A primer on classical test theory and item response theory for assessment in medical education. Medical Education, 44, p. 109-117. https://doi.org/10.1111/j.1365-2923.2009.03425.x
[3]. Gulliksen, H. (1950). Theory of mental tests. New York, NY: John Wiley and Sons.
[4]. Lord, F. M., & Novick, M. R. (1968). Statistical theories of mental test scores. Reading, MA: Addison-Wesley.
[5]. Mahmud, J. (2017). Item response theory: A basic concept. Educational Research and Reviews, 12, p. 258-266. https://doi.org/10.5897/ERR2017.3147
[6]. Jabrayilov, R., Emons, W. H. M., & Sijtsma, K. (2016). Comparison of classical test theory and item response theory in individual change assessment. Applied Psychological Measurement, 40, p. 559-572. https://doi.org/10.1177/0146621616664046
[7]. Finch, W. H., Immekus, J. C., & French, B. F. (2014). Applied Psychometrics Using SAS. Charlotte, NC: Information Age.
[8]. Lord, F. M. (1980). Applications of item response theory to practical problems. Hillsdale, NJ: Lawrence Erlbaum Associates.
[9]. Penfield, R. D. (2014). An NCME instructional module on polytomous item response theory models. Educational Measurement: Issues and Practice, 33, p. 36-48.
[10]. Ostini, R., & Nering, M. L. (2010). New perspective and applications. In M. L. Nering & R. Ostini (Eds.), Handbook of polytomous item response theory models. New York, NY: Routledge.
[11]. Sijtsma, K., & Molenaar, I. W. (2002). Introduction to nonparametric item response theory. Thousand Oaks, CA: Sage.
[12]. Avsar, A. S., & Tavsancil, E. (2017. Examination of polytomous items’ psychometric properties according to nonparametric item response theory models in different test conditions. Educational Sciences: Theory & Practice, 17, p. 493-514. https://doi.org/10.12738/estp2017.2.0246
[13]. van der Ark, L. A. (2012). New developments in Mokken scale analysis in R. Journal of Statistical Software, 48(5).
[14]. Wind, S. A. (2017). An instructional module on Mokken Scale Analysis. Educational Measurement: Issues and Practice, 36(2), p. 50-66.
[15]. Roskam, E. E., van den Wollenberg, A. L., & Jansen, P. G. W. (1986). The Mokken scale: A critical discussion. Applied Psychological Measurement, 10, p. 265-277.
[16]. van Schuur, W. H. (2011). Ordinal item response theory: Mokken scale analysis. Los Angeles, CA: Sage.
[17]. Hambleton, R. K., van der Linden, W. J., & Wells, C. S. (2010). IRT models for the analysis of polytomously scored data. In M. L. Nering & R. Ostini (Eds.), Handbook of polytomous item response theory models. New York, NY: Routledge.
[18]. van Schuur, W. H. (2003). Mokken scale analysis: Between the Guttman scale and parametric item response theory. Political Analysis, 11, p. 139-163. https://doi.org/10.1093/pan/mpg002
[19]. Sijtsma, K., & Verweij, A. C. (1992). Mokken scale analysis: Theoretical considerations and an application to transitivity tasks. Applied Measurement in Education, 5, p. 355-373.
[20]. Watson, R., van der Ark, L. A., Lin, L., Fieo, R., Deary, I. J., & Meijer, R. R. (2011). Item response theory: How Mokken scaling can be used in clinical practice. Journal of Clinical Nursing, 21, p. 2736-2746. https://doi.org/10.1111/j.1365-2702.2011.03893.x
[21]. Ligtvoet, R., van der Ark, L. A., Bergsma, W. P., Sijtsma, K. (2011). Polytomous latent scales for the investigation of ordering of items. Psychometrika, 76, p. 200-216. https://doi.org/10.1007/s11336-0109199-8
[22]. Ski, C. F., Thompson, D. R., Hare, D. L., Stewart, A. G., & Watson, R. (2012). Cardiac Depression Scale: Mokken scaling in heart failure patients. Health and Quality of Life Outcomes, 10(1), p. 1-5.
[23]. Chou, Y., Lee, C., Liu, C., & Hung, C. (2017). Construct validity of the Depression and Somatic Symptoms Scale: Evaluation by Mokken scale analysis. Neuropsychiatric Disease and Treatment, 13, p. 205-211. https://doi.org/10.2147/NDT.S118825
[24]. Rivas, T., Bersabe, R., & Berrocal, C. (2005). Application of the double monotonicity model to polytomous items: Scalability of the Beck Depression items on subjects with eating disorders. European Journal of Psychological Assessment, 21, p. 1-10. https://doi.org/10.1027/1015-5759.21.1.1
[25]. Munyombwe, T., Hofer, S., Fitzsimons, D., Thompson, D. R., Lane, D., Smith, K., & Astin, F. (2014). An evaluation of the Minnesota Living with Heart Failure Questionnaire using Rasch analysis. Quality of Life Research, 23, p. 1753-1765. https://doi.org/10.1007/s11136-013-1617-0
[26]. Ivarsson, B., & Malm, U. (2007). Self-reported consumer satisfaction in mental health services: Validation of a self-rating version of the UKU-Consumer Satisfaction Rating Scale. Nordic Journal of Psychiatry, 61, p. 194-200. https://doi.org/10.1080/08039480701352488
[27]. Tonnaer, F., Cima, M., Sijtsma, K., Uzieblo, K., & Lilienfeld, S. O. (2013). Screening for psychopathy: Validation of the Psychopathic Personality Inventory-Short Form with reference scores. Journal of Psychopathology and Behavioral Assessment, 35, p. 153-161. https://doi.org/10.1007/s10862-012-9333-2
[28]. Wismeijer, A. A. J., Sijtsma, K., van Assen, M. A. L. M., & Vingerhoets, A. J. J. M. (2008). A comparative study of the dimensionality of the Self-Concealment Scale using principal components analysis and Mokken scale analysis. Journal of Personality Assessment, 90, p. 323-334. https://doi.org/ 10.1080/00223890802107875
[29]. Finserås, T. R., Pallesen, S., Mentzoni, R. A., Krossbakken, E., King, D. L., & Molde, H. (2019). Evaluating an internet gaming disorder scale using mokken scaling analysis. Frontiers in Psychology, 10. https://doi.org/ 10.3389/fpsyg.2019.00911
[30]. Moussa, S. (2021). Measuring brand personality using emoji: findings from Mokken scaling. Journal of Brand Management, 28, p. 116–132. https://doi.org/10.1057/s41262-020-00220-8
[31]. Paas, L. J., & Sijtsma, K. (2008). Nonparametric item response theory for investigating dimensionality of marketing scales: A SERVQUAL application. Marketing Letters, 19, p. 157-170. https://doi.org/10.1007/s11002-007-9031-0
[32]. Font, J., Wojcieszak, M., & Navarro, C. J. (2015). Participation, representation and expertise: Citizen preferences for political decision-making processes. Political Studies, 63, p. 153-172. https://doi.org/10.1111/1467-9248.12191
[33]. Jacoby, W. G. (1995). The Structure of Ideological Thinking in the American Electorate. American Journal of Political Science, 39, p. 314-335.
[34]. van Elsas, E. (2015). Political trust as a rational attitude: A comparison of the nature of political trust across different levels of education. Political Studies, 63, p. 1158-1178. https://doi.org/10.1111/1467-9248.12148
[35]. Vargas-Vargas, M., Jiménez, J. M., Meseguer-Santamaría, M. L., Montero-Lorenzo, J. M., & Fernández-Avilés, G. (2010). American Journal of Business Education, 3, p. 1-6.
[36]. Cruise, R. J., Cash, W., & Bolton, D. L. (1985, August) Development and validation of an instrument to measure statistical anxiety. Paper presented at the annual meeting of the American Statistical Association, Las Vegas, NV.
[37]. Baloglu, M. (2002) Psychometric properties of the Statistics Anxiety Rating Scale, Psychological Reports, 90, p. 315-325. https://doi.org/10.2466/pr0.2002.90.1.315
[38]. DeVaney, T. A. (2016). Confirmatory factor analysis of the Statistical Anxiety Rating Scale with online graduate students. Psychological Reports, 118, p. 565-586. https://doi.org/10.1177/0033294116644093
[39]. Mji, A., & Onwuegbuzie, A. J. (2004) Evidence of score reliability and validity of the Statistical Anxiety Rating Scale among Technikon students in South Africa. Measurement and Evaluation in Counseling and Development, 36, p. 238-251.
[40]. Hsiao, T. (2010) The Statistical Anxiety Rating Scale: Further evidence for multidimensionality. Psychological Reports, 107, 977-982. https://doi.org/10.2466/07.11.PR0.107.6.977-982.
[41]. Papousek, I., Ruggeri, K., Macher, D., Paechter, M., Heene, M., Weiss, E. M., Schulter, G., & Freudenthaler, H. H. (2012) Psychometric evaluation and experimental validation of the Statistics Anxiety Rating Scale. Journal of Personality Assessment, 94, p. 82-91. https://doi.org/10.1080/00223891.2011.627959
[42]. Teman, E. D. (2013) A Rasch analysis of the Statistical Anxiety Rating Scale. Journal of Applied Measurement, 14, 414-434.
[43]. Chew, P. K. H., & Dillon, D. B. (2014) Statistics anxiety update: Refining the construct and recommendations for a new research agenda. Perspectives on Psychological Science, 9, p. 196-208. https://doi.org/10.1177/1745691613518077
[44]. Van der Ark, L. A. (2007). Mokken Scale Analysis in R. Journal of Statistical Software, 20, https://www.jstatsoft.org/article/view/v020i11
[2]. De Champlain, A. F. (2010). A primer on classical test theory and item response theory for assessment in medical education. Medical Education, 44, p. 109-117. https://doi.org/10.1111/j.1365-2923.2009.03425.x
[3]. Gulliksen, H. (1950). Theory of mental tests. New York, NY: John Wiley and Sons.
[4]. Lord, F. M., & Novick, M. R. (1968). Statistical theories of mental test scores. Reading, MA: Addison-Wesley.
[5]. Mahmud, J. (2017). Item response theory: A basic concept. Educational Research and Reviews, 12, p. 258-266. https://doi.org/10.5897/ERR2017.3147
[6]. Jabrayilov, R., Emons, W. H. M., & Sijtsma, K. (2016). Comparison of classical test theory and item response theory in individual change assessment. Applied Psychological Measurement, 40, p. 559-572. https://doi.org/10.1177/0146621616664046
[7]. Finch, W. H., Immekus, J. C., & French, B. F. (2014). Applied Psychometrics Using SAS. Charlotte, NC: Information Age.
[8]. Lord, F. M. (1980). Applications of item response theory to practical problems. Hillsdale, NJ: Lawrence Erlbaum Associates.
[9]. Penfield, R. D. (2014). An NCME instructional module on polytomous item response theory models. Educational Measurement: Issues and Practice, 33, p. 36-48.
[10]. Ostini, R., & Nering, M. L. (2010). New perspective and applications. In M. L. Nering & R. Ostini (Eds.), Handbook of polytomous item response theory models. New York, NY: Routledge.
[11]. Sijtsma, K., & Molenaar, I. W. (2002). Introduction to nonparametric item response theory. Thousand Oaks, CA: Sage.
[12]. Avsar, A. S., & Tavsancil, E. (2017. Examination of polytomous items’ psychometric properties according to nonparametric item response theory models in different test conditions. Educational Sciences: Theory & Practice, 17, p. 493-514. https://doi.org/10.12738/estp2017.2.0246
[13]. van der Ark, L. A. (2012). New developments in Mokken scale analysis in R. Journal of Statistical Software, 48(5).
[14]. Wind, S. A. (2017). An instructional module on Mokken Scale Analysis. Educational Measurement: Issues and Practice, 36(2), p. 50-66.
[15]. Roskam, E. E., van den Wollenberg, A. L., & Jansen, P. G. W. (1986). The Mokken scale: A critical discussion. Applied Psychological Measurement, 10, p. 265-277.
[16]. van Schuur, W. H. (2011). Ordinal item response theory: Mokken scale analysis. Los Angeles, CA: Sage.
[17]. Hambleton, R. K., van der Linden, W. J., & Wells, C. S. (2010). IRT models for the analysis of polytomously scored data. In M. L. Nering & R. Ostini (Eds.), Handbook of polytomous item response theory models. New York, NY: Routledge.
[18]. van Schuur, W. H. (2003). Mokken scale analysis: Between the Guttman scale and parametric item response theory. Political Analysis, 11, p. 139-163. https://doi.org/10.1093/pan/mpg002
[19]. Sijtsma, K., & Verweij, A. C. (1992). Mokken scale analysis: Theoretical considerations and an application to transitivity tasks. Applied Measurement in Education, 5, p. 355-373.
[20]. Watson, R., van der Ark, L. A., Lin, L., Fieo, R., Deary, I. J., & Meijer, R. R. (2011). Item response theory: How Mokken scaling can be used in clinical practice. Journal of Clinical Nursing, 21, p. 2736-2746. https://doi.org/10.1111/j.1365-2702.2011.03893.x
[21]. Ligtvoet, R., van der Ark, L. A., Bergsma, W. P., Sijtsma, K. (2011). Polytomous latent scales for the investigation of ordering of items. Psychometrika, 76, p. 200-216. https://doi.org/10.1007/s11336-0109199-8
[22]. Ski, C. F., Thompson, D. R., Hare, D. L., Stewart, A. G., & Watson, R. (2012). Cardiac Depression Scale: Mokken scaling in heart failure patients. Health and Quality of Life Outcomes, 10(1), p. 1-5.
[23]. Chou, Y., Lee, C., Liu, C., & Hung, C. (2017). Construct validity of the Depression and Somatic Symptoms Scale: Evaluation by Mokken scale analysis. Neuropsychiatric Disease and Treatment, 13, p. 205-211. https://doi.org/10.2147/NDT.S118825
[24]. Rivas, T., Bersabe, R., & Berrocal, C. (2005). Application of the double monotonicity model to polytomous items: Scalability of the Beck Depression items on subjects with eating disorders. European Journal of Psychological Assessment, 21, p. 1-10. https://doi.org/10.1027/1015-5759.21.1.1
[25]. Munyombwe, T., Hofer, S., Fitzsimons, D., Thompson, D. R., Lane, D., Smith, K., & Astin, F. (2014). An evaluation of the Minnesota Living with Heart Failure Questionnaire using Rasch analysis. Quality of Life Research, 23, p. 1753-1765. https://doi.org/10.1007/s11136-013-1617-0
[26]. Ivarsson, B., & Malm, U. (2007). Self-reported consumer satisfaction in mental health services: Validation of a self-rating version of the UKU-Consumer Satisfaction Rating Scale. Nordic Journal of Psychiatry, 61, p. 194-200. https://doi.org/10.1080/08039480701352488
[27]. Tonnaer, F., Cima, M., Sijtsma, K., Uzieblo, K., & Lilienfeld, S. O. (2013). Screening for psychopathy: Validation of the Psychopathic Personality Inventory-Short Form with reference scores. Journal of Psychopathology and Behavioral Assessment, 35, p. 153-161. https://doi.org/10.1007/s10862-012-9333-2
[28]. Wismeijer, A. A. J., Sijtsma, K., van Assen, M. A. L. M., & Vingerhoets, A. J. J. M. (2008). A comparative study of the dimensionality of the Self-Concealment Scale using principal components analysis and Mokken scale analysis. Journal of Personality Assessment, 90, p. 323-334. https://doi.org/ 10.1080/00223890802107875
[29]. Finserås, T. R., Pallesen, S., Mentzoni, R. A., Krossbakken, E., King, D. L., & Molde, H. (2019). Evaluating an internet gaming disorder scale using mokken scaling analysis. Frontiers in Psychology, 10. https://doi.org/ 10.3389/fpsyg.2019.00911
[30]. Moussa, S. (2021). Measuring brand personality using emoji: findings from Mokken scaling. Journal of Brand Management, 28, p. 116–132. https://doi.org/10.1057/s41262-020-00220-8
[31]. Paas, L. J., & Sijtsma, K. (2008). Nonparametric item response theory for investigating dimensionality of marketing scales: A SERVQUAL application. Marketing Letters, 19, p. 157-170. https://doi.org/10.1007/s11002-007-9031-0
[32]. Font, J., Wojcieszak, M., & Navarro, C. J. (2015). Participation, representation and expertise: Citizen preferences for political decision-making processes. Political Studies, 63, p. 153-172. https://doi.org/10.1111/1467-9248.12191
[33]. Jacoby, W. G. (1995). The Structure of Ideological Thinking in the American Electorate. American Journal of Political Science, 39, p. 314-335.
[34]. van Elsas, E. (2015). Political trust as a rational attitude: A comparison of the nature of political trust across different levels of education. Political Studies, 63, p. 1158-1178. https://doi.org/10.1111/1467-9248.12148
[35]. Vargas-Vargas, M., Jiménez, J. M., Meseguer-Santamaría, M. L., Montero-Lorenzo, J. M., & Fernández-Avilés, G. (2010). American Journal of Business Education, 3, p. 1-6.
[36]. Cruise, R. J., Cash, W., & Bolton, D. L. (1985, August) Development and validation of an instrument to measure statistical anxiety. Paper presented at the annual meeting of the American Statistical Association, Las Vegas, NV.
[37]. Baloglu, M. (2002) Psychometric properties of the Statistics Anxiety Rating Scale, Psychological Reports, 90, p. 315-325. https://doi.org/10.2466/pr0.2002.90.1.315
[38]. DeVaney, T. A. (2016). Confirmatory factor analysis of the Statistical Anxiety Rating Scale with online graduate students. Psychological Reports, 118, p. 565-586. https://doi.org/10.1177/0033294116644093
[39]. Mji, A., & Onwuegbuzie, A. J. (2004) Evidence of score reliability and validity of the Statistical Anxiety Rating Scale among Technikon students in South Africa. Measurement and Evaluation in Counseling and Development, 36, p. 238-251.
[40]. Hsiao, T. (2010) The Statistical Anxiety Rating Scale: Further evidence for multidimensionality. Psychological Reports, 107, 977-982. https://doi.org/10.2466/07.11.PR0.107.6.977-982.
[41]. Papousek, I., Ruggeri, K., Macher, D., Paechter, M., Heene, M., Weiss, E. M., Schulter, G., & Freudenthaler, H. H. (2012) Psychometric evaluation and experimental validation of the Statistics Anxiety Rating Scale. Journal of Personality Assessment, 94, p. 82-91. https://doi.org/10.1080/00223891.2011.627959
[42]. Teman, E. D. (2013) A Rasch analysis of the Statistical Anxiety Rating Scale. Journal of Applied Measurement, 14, 414-434.
[43]. Chew, P. K. H., & Dillon, D. B. (2014) Statistics anxiety update: Refining the construct and recommendations for a new research agenda. Perspectives on Psychological Science, 9, p. 196-208. https://doi.org/10.1177/1745691613518077
[44]. Van der Ark, L. A. (2007). Mokken Scale Analysis in R. Journal of Statistical Software, 20, https://www.jstatsoft.org/article/view/v020i11
Downloads
Published
2021-04-08
How to Cite
DeVaney, T. (2021). Mokken Scale Analysis: Discussion and Application. Advances in Social Sciences Research Journal, 8(3), 672–695. https://doi.org/10.14738/assrj.83.9949
Issue
Section
Articles
License
Authors wishing to include figures, tables, or text passages that have already been published elsewhere are required to obtain permission from the copyright owner(s) for both the print and online format and to include evidence that such permission has been granted when submitting their papers. Any material received without such evidence will be assumed to originate from the authors.
