Identifying CRAAP on the Internet: A Source Evaluation Intervention


  • Krista Renee Muis McGill University
  • Courtney Denton McGill University
  • Adam Dubé McGill University



epistemic cognition, source evaluation, intervention, digital literacy


Individuals of all ages struggle to determine the reliability of information on the internet. To address this common issue, many educational institutions have endorsed the CRAAP test (i.e., currency, relevance, authority, accuracy, and purpose) as an effective approach to support identification of unreliable information. The present study extended the CRAAP test by incorporating a modeling component on how to evaluate and integrate multiple sources of varying quality on the internet and evaluated the efficacy of this source evaluation training intervention. Eighty-two participants across Canada were recruited to evaluate six authentic webpages and then construct an argument on a specific topic. Half the sample received training to examine the currency, relevance, authority, accuracy, and purpose (i.e., CRAAP) of the webpages before completing the online activity. Results revealed that the intervention group provided more accurate rank-ordering of the webpages, but no differences were found between groups on source integration via an argumentative essay. These findings suggest that the CRAAP test is effective in improving individuals’ evaluations of online sources but is not effective in promoting better quality source integration.


Barnes, M. D., Penrod, C., Neiger, B. L., Merrill, R. M., Thackeray, R., Eggett, D. L., & Thomas, E. (2003). Measuring the relevance of evaluation criteria among health information seekers on the Internet. Journal of Health Psychology, 8(1), 71–82.

Barzilai, S., & Chinn, C. A. (2018). On the goals of epistemic education: Promoting apt epistemic performance. Journal of the Learning Sciences, 27(3), 353–389.

Barzilai, S., & Eshet-Alkalai, Y. (2015). The role of epistemic perspectives in comprehension of multiple author viewpoints. Learning and Instruction, 36, 86–103.

Barzilai, S., Mor-Hagani, S., Zohar, A. R., Shlomi-Elooz, T., & Ben-Yishai, R. (2020). Making sources visible: Promoting multiple document literacy with digital epistemic scaffolds. Computers & Education, 157, 103980.

Barzilai, S., Thomm, E., & Shlomi-Elooz, T. (2020). Dealing with disagreement: The roles of topic familiarity and disagreement explanation in evaluation of conflicting expert claims and sources. Learning and Instruction, 69, 101367.

Barzilai, S., & Zohar, A. (2014). Reconsidering personal epistemology as metacognition: A multifaceted approach to the analysis of epistemic thinking. Educational Psychologist, 49(1), 13–35.

Barzilai, S., Zohar, A. R., & Mor-Hagani, S. (2018). Promoting integration of multiple texts: A review of instructional approaches and practices. Educational Psychology Review, 1–27.

Batchelor, O. (2017). Getting out the truth: The role of libraries in the fight against fake news. Reference Services Review, 143-148.

Bates, B. R., Romina, S., Ahmed, R., & Hopson, D. (2006). The effect of source credibility on consumers’ perceptions of the quality of health information on the Internet. Medical Informatics and the Internet in Medicine, 31(1), 45–52.

Blakeslee, S. (2004). The CRAAP test. Loex Quarterly, 31(3), 4.

Braasch, J. L. G., Bråten, I., Strømsø, H. I., Anmarkrud, Ø., & Ferguson, L. E. (2013). Promoting secondary school students’ evaluation of source features of multiple documents. Contemporary Educational Psychology, 38(3), 180–195.

Bråten, I., Strømsø, H. I., & Andreassen, R. (2016). Sourcing in professional education: Do text factors make any difference? Reading and Writing, 29(8), 1599–1628.

Bråten, I., Strømsø, H. I., & Britt, M. A. (2009). Trust matters: Examining the role of source evaluation in students’ construction of meaning within and across multiple texts. Reading Research Quarterly, 44(1), 6–28.

Breakstone, J., McGrew, S., Smith, M., Ortega, T., & Wineburg, S. (2018). Why we need a new approach to teaching digital literacy. Phi Delta Kappan, 99(6), 27–32.

Britt, M. A., & Aglinskas, C. (2002). Improving students’ ability to identify and use source information. Cognition and Instruction, 20(4), 485–522.

Chevrier, M., Muis., K. R., Pekrun, R., Sinatra, G., & Denton, C. A. (2019). Thinking critically about socio-scientific issues: The roles of epistemic cognition and epistemic emotions [Unpublished manuscript]. Department of Educational and Counselling Psychology, McGill University.

Chiagouris, L., Long, M. M., & Plank, R. E. (2008). The consumption of online news: The relationship of attitudes toward the site and credibility. Journal of Internet Commerce, 7(4), 528–549.

Chinn, C. A., Buckland, L. A., & Samarapungavan, A. (2011). Expanding the dimensions of epistemic cognition: Arguments from philosophy and psychology. Educational Psychologist, 46(3), 141–167.

Chinn, C. A., Rinehart, R. W., & Buckland, L. A. (2014). Epistemic cognition and evaluating information: Applying the AIR model of epistemic cognition. In D. N. Rapp & J. L.G. Braasch (Eds.), Processing inaccurate information: Theoretical and applied perspectives from cognitive science and the educational sciences (pp. 425–453). MIT Press.

Chinn, C., & Sandoval, W. A. (2018). Epistemic cognition and epistemic development. In F. Fischer, C. E. Hmelo-Silver, S. R. Goldman, & P. Resiman (Eds.), International handbook of the learning sciences (pp. 46–55). Routledge.

Canadian Mental Health Association (CMHA). (2018a). Stigma and discrimination. Retrieved from

CMHA. (2018b). Myths about mental illness. Retrieved from:

Cohen, J., Cohen, P., West, S. G., & Aiken, L. S. (2013). Applied multiple regression/correlation analysis for the behavioral sciences. Routledge.

Cresswell, J. W., & Plano Clark, V. L. (2011). Designing and conducting mixed methods research (2nd ed.). Sage Publications.

Cresswell, J. W., & Plano Clark, V. L. (2017). Designing and conducting mixed methods research (3rd ed.). Sage Publications.

Denton, C. A., Muis, K.R., Armstrong, S., & Dubé, A. (in preparation). En-garde: Source evaluations in the digital age. Contemporary Educational Psychology.

Duby, H. R. (2018). What a load of CRAAP: Evaluating information in an era of "fake news". Tennessee Libraries, 68(4).

Eshet-Alkalai, Y., & Chajut, E. (2009). Changes over time in digital literacy. CyberPsychology & Behavior, 12(6), 713–715.

Eshet-Alkalai, Y., & Chajut, E. (2010). You can teach old dogs new tricks: The factors that affect changes over time in digital literacy. Journal of Information Technology Education: Research, 9(1), 173–181.

Faul, F., Erdfelder, E., Buchner, A., & Lang, A.-G. (2009). Statistical power analyses using G*Power 3.1: Tests for correlation and regression analyses. Behavior Research Methods, 41(4), 1149–1160.

Fielding, J. A. (2019). Rethinking CRAAP: Getting students thinking like fact-checkers in evaluating web sources. College & Research Libraries News, 80(11), 620.

Gerjets, P., Kammerer, Y., & Werner, B. (2011). Measuring spontaneous and instructed evaluation processes during Web search: Integrating concurrent thinking-aloud protocols and eye-tracking data. Learning and Instruction, 21(2), 220–231.

Green, J. (2019, Jan 22). Check yourself with lateral reading: Crash course navigating digital information #3 [Youtube video].

Greene, J. A., Cartiff, B. M., & Duke, R. F. (2018). A meta-analytic review of the relationship between epistemic cognition and academic achievement. Journal of Educational Psychology.

Greene, J. A., Copeland, D. Z., Deekens, V. M., & Seung, B. Y. (2018). Beyond knowledge: Examining digital literacy’s role in the acquisition of understanding in science. Computers & Education, 117, 141–159.

Greene, J. A., Yu, S. B., & Copeland, D. Z. (2014). Measuring critical components of digital literacy and their relationships with learning. Computers & Education, 76, 55–69.

Halverson, K. L., Siegel, M. A., & Freyermuth, S. K. (2010). Non-science majors’ critical evaluation of websites in a biotechnology course. Journal of Science Education and Technology, 19(6), 612–620.

Hämäläinen, E. K., Kiili, C., Marttunen, M., Räikkönen, E., González-Ibáñez, R., & Leppänen, P. H. T. (2020). Promoting sixth graders’ credibility evaluation of Web pages: An intervention study. Computers in Human Behavior, 110, 106372.

Jung, E. H., Walsh-Childers, K., & Kim, H.-S. (2016). Factors influencing the perceived credibility of diet-nutrition information web sites. Computers in Human Behavior, 58, 37–47.

Jung, W. S., Chung, M.-Y., & Rhee, E. S. (2018). The effects of attractiveness and source expertise on online health sites. Health Communication, 33(8), 962–971.

Kąkol, M., Nielek, R., & Wierzbicki, A. (2017). Understanding and predicting Web content credibility using the Content Credibility Corpus. Information Processing & Management, 53(5), 1043–1061.

Kiili, C., Laurinen, L., & Marttunen, M. (2008). Students evaluating Internet sources: From versatile evaluators to uncritical readers. Journal of Educational Computing Research, 39(1), 75–95.

Kobau, R., DiIorio, C., Chapman, D., Delvecchio, P., & SAMHSA/CDC Mental Illness Stigma Panel Members. (2010). Attitudes about mental illness and its treatment: Validation of a generic scale for public health surveillance of mental illness associated stigma. Community Mental Health Journal, 46(2), 164–176.

Landis, J. R., & Koch, G. G. (1977). The measurement of observer agreement for categorical data. Biometrics, 33(1), 159.

Machackova, H., & Smahel, D. (2018). The perceived importance of credibility cues for the assessment of the trustworthiness of online information by visitors of health-related websites: The role of individual factors. Telematics and Informatics, 35(5), 1534–1541.

Mason, L., Ariasi, N., & Boldrin, A. (2011). Epistemic beliefs in action: Spontaneous reflections about knowledge and knowing during online information searching and their influence on learning. Learning and Instruction, 21(1), 137–151.

Mason, L., Junyent, A. A., & Tornatora, M. C. (2014). Epistemic evaluation and comprehension of web-source information on controversial science-related topics: Effects of a short-term instructional intervention. Computers & Education, 76, 143–157.

Mason, L., Scrimin, S., Tornatora, M. C., Suitner, C., & Moè, A. (2018). Internet source evaluation: The role of implicit associations and psychophysiological self-regulation. Computers & Education, 119, 59–75.

McGrew, S. (2020). Learning to evaluate: An intervention in civic online reasoning. Computers & Education, 145, 103711.

McGrew, S., Smith, M., Breakstone, J., Ortega, T., & Wineburg, S. (2019). Improving university students’ web savvy: An intervention study. British Journal of Educational Psychology, bjep.12279.

Meppelink, C. S., Smit, E. G., Fransen, M. L., & Diviani, N. (2019). “I was right about vaccination”: Confirmation bias and health literacy in online health information seeking. Journal of Health Communication, 24(2), 129–140.

Meriam Library of CSU Chico. (2010). Evaluating information - Applying the CRAAP test [PDF document]. Meriam Library of CSU Chico.

Meyers, L. S., Gamst, G., & Guarino, A. J. (2016). Applied multivariate research: Design and interpretation. Sage Publications.

Morse, J. M. (2003). Principles of mixed methods and multimethod research design. In A. Tashakkori, & C. Teddlie (Eds.), Handbook of mixed methods in social and behavioral research (pp. 189–208). Sage Publications.

Pretorius, L. (2018). Experiential and self-discovery learning in digital literacy: Developing the discernment to evaluate source reliability. College & Undergraduate Libraries, 25(4), 388–405.

Queen's University Library. (2020). CRAAP criteria. Bachelor of health sciences toolbox.

Richter, T., & Maier, J. (2017). Comprehension of multiple documents with conflicting information: A two-step model of validation. Educational Psychologist, 52(3), 148–166.

Richter, T., & Schmid, S. (2010). Epistemological beliefs and epistemic strategies in self-regulated learning. Metacognition and Learning, 5(1), 47–65.

Rouet, J.-F., & Britt, M. A. (2011). Relevance processes in multiple document comprehension. In M. T. McCrudden, J. P. Magliano, & G. Schraw (Eds.), Text relevance and learning from text (pp. 19–52). IAP Information Age Publishing.

Salmerón, L., Gil, L., & Bråten, I. (2018). Effects of reading real versus print-out versions of multiple documents on students’ sourcing and integrated understanding. Contemporary Educational Psychology, 52, 25–35.

Sandoval, W. A. (2017). Situating practices of epistemic cognition. In T. G. Amin & O. Levrini (Eds.), Converging perspectives on conceptual change (1st ed., pp. 253–260). Routledge.

Smallwood, C. (2015). The complete guide to using Google in libraries: Instruction, administration, and staff productivity (Vol. 1). Rowman & Littlefield.

Stanford History Education Group. (2016). Evaluating information: The cornerstone of online civic reasoning [PDF document].

Stanford History Education Group. (2020). COR. Civic Online Reasoning.

Subramaniam, M., Taylor, N. G., St. Jean, B., Follman, R., Kodama, C., & Casciotti, D. (2015). As simple as that?: Tween credibility assessment in a complex online world. Journal of Documentation, 71(3), 550–571.

Tabachnick, B. G., & Fidell, L. S. (2013). Using multivariate statistics: International edition. Pearson.

Taylor, W. D., Johnson, G., Ault, M. K., Griffith, J. A., Rozzell, B., Connelly, S., Jensen, M. L., Dunbar, N. E., & Ness, A. M. (2015). Ideological group persuasion: A within-person study of how violence, interactivity, and credibility features influence online persuasion. Computers in Human Behavior, 51, 448–460.

The News Literacy Project. (2020). Checkology. Checkology.

Ulyshen, T. Z., Koehler, M. J., & Gao, F. (2015). Understanding the connection between epistemic beliefs and Internet searching. Journal of Educational Computing Research, 53(3), 345–383.

Van Strien, J. L., Brand-Gruwel, S., & Boshuizen, H. P. (2014). Dealing with conflicting information from multiple nonlinear texts: Effects of prior attitudes. Computers in Human Behavior, 32, 101–111.

Walraven, A., Brand-Gruwel, S., & Boshuizen, H. P. A. (2013). Fostering students’ evaluation behaviour while searching the internet. Instructional Science, 41(1), 125–146.

White, A. M., DeCuir-Gunby, J. T., & Kim, S. (2019). A mixed methods exploration of the relationships between the racial identity, science identity, science self-efficacy, and science achievement of African American students at HBCUs. Contemporary Educational Psychology, 57, 54–71.

Wiley, J., Goldman, S. R., Graesser, A. C., Sanchez, C. A., Ash, I. K., & Hemmerich, J. A. (2009). Source evaluation, comprehension, and learning in Internet science inquiry tasks. American Educational Research Journal, 46(4), 1060–1106.

Wineburg, S., & McGrew, S. (2017). Lateral reading: Reading less and learning more when evaluating digital information [PDF document].

Wineburg, S. S. (1991). On the reading of historical texts: Notes on the breach between school and academy. American Educational Research Journal, 28(3), 495–519.




How to Cite

Muis, K. R., Denton, C., & Dubé, A. (2022). Identifying CRAAP on the Internet: A Source Evaluation Intervention. Advances in Social Sciences Research Journal, 9(7), 239–265.