Page 1 of 18
403
Advances in Social Sciences Research Journal – Vol.8, No.1
Publication Date: January 25, 2021
DOI:10.14738/assrj.81.8457.
Simons, L. (2021). Student Learning Outcomes. An Evaluation of Academic and Cultural-Based Service-Learning as Critical Pedagogies
for Enhancing Student Learning. Advances in Social Sciences Research Journal, 8 (1) 403-420.
STUDENT LEARNING OUTCOMES
An Evaluation of Academic- and Cultural-Based Service-Learning as
Critical Pedagogies for Enhancing Student Learning
Lori Simons
Widener University
ABSTRACT
A cross-sectional study was conducted to evaluate differences in
students learning outcomes for 328 undergraduate students enrolled
in psychology courses that utilize academic- and cultural-based
service-learning as primary pedagogical methods. Results from a
repeated measures analyses of covariance indicate that students
improve their awareness of White privilege and understanding of
gender and racial discrimination from the beginning to the end of the
course. Cultural-based service-learners also had a deeper
understanding of White privilege, gender and racial discrimination,
and racial and economic disparities in the community, as well as
stronger perspective-taking skills, ethnocultural empathy, and
interpersonal engagement compared to academic-based service- learners. Results from a repeated measures analysis of variance
further indicate that upperclassmen develop a racial-cultural-ethnic
identity; while, underclassmen develop ethnocultural awareness over
time. Implications for incorporating critical service-learning
pedagogies in beginning, middle, and ending psychology courses are
discussed.
AN EVALUATION OF ACADEMIC- AND CULTURAL-BASED SERVICE-LEARNING AS CRITICAL
PEDAGOGIES FOR ENHANCING STUDENT LEARNING
During the past decade, institutions of higher education (IHEs) have developed and implemented
elaborate assessment strategies to measure student learning outcomes (SLOs). Student learning
outcomes are used for program review and institutional accreditation (Landrum, 2011). The
American Psychological Association (APA) has established guidelines for developing high quality
undergraduate programs in psychology (APA, 2013). Psychology Departments develop program
outcomes that align with the curricula based on these guidelines and assess student attainment
(APA, 2016). Departments use assessment results to revise the curricula in order to improve
student learning. Multiculturalism and social responsibility have been emphasized in the revised
guidelines for undergraduate psychology programs (APA, 2016;). Bringle, Ruiz, Brown and Reeb
(2016a) describe how incorporating service-learning in an undergraduate psychology curriculum
can be used to promote student learning in academic, personal and civic domains. The aim of this
study is to assess service-learning impacts on student learning for students enrolled in
undergraduate psychology courses.
Page 2 of 18
404
Advances in Social Sciences Research Journal (ASSRJ) Vol.8, Issue 1, January-2021
The University established an academic service-learning faculty development program (ASLDFP)
that trained faculty how to develop course objectives, connect with the community, incorporate
reflection activities and assignments, and assess student learning in 2001 (i.e., this program was
considered ad hoc until 2004) (Dowshen & Pickron-Davis, 2011). The Educational Psychology
course was one of the first service-learning courses offered at the University. Assessment of
student learning in the Educational Psychology course was used for course improvement. Pilot
data indicated that students increased their understanding of diversity and interest in working
with diverse recipients. This finding led to the refinement of the Educational Psychology course
and the development of a Multicultural Psychology course, which was developed to fill a void in
the undergraduate curriculum (i.e., the Psychology Department did not offer a specific course in
diversity). Both Educational and Multicultural Psychology courses included developmentally- appropriate practices for sophomores and juniors (Dunn, Wilson, Freeman, & Stowell, 2010) and
service-learning pedagogies (Amsel, Cheshire, Massen, Kowalewski, & Winniford, 2011); while,
the multicultural psychology course also included APA’s standards emphasizing cultural diversity
and instructional strategies to engage students in discussions about cultural privilege and social
responsibility (Eyler & Giles, 1999; Prieto, 2011).
In the psychology curricula, there are beginning (i.e., Introduction to Psychology) and ending (i.e.,
Capstone) courses (Dunn et al., 2010). Students take courses from a variety of content areas
taught by faculty representing diverse orientations and sub-disciplines. Academic-based service- learning (ABSL) is one pedagogical method that is used in undergraduate psychology courses and
combines academic study with community service (Eyler, 2002). Educators debate about whether
ABSL should be incorporated into beginning or ending courses (Amsel et al., 2011); while, other
scholars suggest ABSL should be embedded in multiple courses throughout the undergraduate
curriculum (Bringle, Reeb, Brown. & Ruiz, 2016b), because of the pedagogical benefits related to
student learning. The few studies that have measured differences in student learning for freshmen
and seniors suggest that there are different learning outcomes for underclassman and
upperclassmen (Eppler, Ironsmith, Dingle, Erickson, 2011; Molee, Henry, Sessa, & McKinney- Prupis, 2010). Molee et al. (2010) evaluated reflections for 39 undergraduate students and found
that seniors demonstrated greater critical thinking and civic engagement skills than freshmen.
Additional research is needed to understand the influence from development on service-learning
and student learning.
Several scholars suggest that ABSL is an effective pedagogy for developing culturally competent
critical thinkers and civic leaders in undergraduate students (Bell, Horn, & Roxas, 2007). Students
who participate in service-learning are often immersed in a cultural that differs from their own.
Students learn about community, forge relationships with service recipients, and acquire
information that negates preconceived views. In contrast to this perspective, several other
researchers propose that students retain stereotypes through participation in service activities
that reinforce their preconceived views (Dunlap, Scoggin, Green, & Davi, 2007; Hess, Lanig, &
Vaughan, 2007). In fact, students are likely to leave service experiences with their stereotypes
intact unless their assumptions are challenged. Mitchell (2008) suggests that ABSL does not
challenge students to rethink assumptions or reexamine attitudes. Therefore, students who
participate in ABSL are likely to retain their preexisting views unless they are required to
systematically analyze their service experiences and connect them to diversity content.
Page 3 of 18
URL: http://dx.doi.org/10.14738/assrj.81.8457. 405
Simons, L. (2021). Student Learning Outcomes. An Evaluation of Academic and Cultural-Based Service-Learning as Critical Pedagogies for Enhancing
Student Learning. Advances in Social Sciences Research Journal, 8 (1) 403-420.
Cultural-based service-learning (CBSL) is an extension of ABSL in which students are required to
connect the diversity content to the service context (Baldwin, Buchanan, & Rudisill, 2007; Sterling,
2007). CBSL requires students to examine their personal biases, think critically about power,
privilege, and oppression, and systematically analyze the conditions that lead to racial and
economic disparities in the community (Baldwin et al., 2007). Students who participate in CBSL
reformulate their attitudes and acquire a deeper understanding of their own systems of privilege
than those students who participate in ABSL, but only if diversity and social justice issues are
explicit course objectives and outcomes. Additional scholarship is necessary to disentangle
impacts from the service context and diversity content on student learning for underclassmen and
upperclassmen in psychology courses. The next step in this area of research is to measure
differences in student learning outcomes for students enrolled in psychology courses that utilize
ABSL and CBSL as primary pedagogical methods using a cross-sectional research design. Our
study was guided by two primary questions:
(1). Are there differences in student learning outcomes as measured in interpersonal problem- solving, social justice attitudes, awareness of White privilege, racism and institutional
discrimination, ethnocultural awareness, acceptance, perspective-taking, and expression of
feelings, ethnic identity, course content and climate, multicultural awareness, knowledge and
relationships, interpersonal and community engagement, and socially responsible leadership
from the beginning to the end of the semester for students enrolled in psychology courses
that utilize ABSL and CBSL as primary pedagogical methods?
a) Are there differences in student learning outcomes for students who have taken
previous service-learning courses?
b) Are there differences in student learning outcomes for students who have taken
previous diversity courses?
(2). Are there differences in student learning outcomes from the beginning to the end of the
semester for underclassmen and upperclassmen in psychology courses?
METHOD
Participants
A secondary analysis of 1,500 undergraduate students enrolled in psychology courses during
2002-2015 at a private teaching university in a northern metropolitan area took part in the study.
Of these students, 329 of them took part in either ABSL (75%) or CBSL (25%) courses. Most
students in this subsample of service-learners were White (79%) and female (75%) with a mean
age of 19.22 years (SD= 2.83). The remaining students self-identified as African-American (17%),
Latino/a (4%), Asian-American (5%), Indian (3%), and Biracial or Multiracial (2%) and male
(21%). There were no significant differences for gender and race. Thirty-six percent were
freshmen, 32% were sophomores, 19% were juniors, and 13% were seniors. Slightly more than
half (60%) of the students reported volunteering outside of school. Sixty-two percent of students
reported taking at least one service-learning course and 21% of them reported taking at least one
diversity course prior to the current service-learning course.
Course Content
Academic-Based Service-Learning (Simons et al., 2011). Educational Psychology: Early Learners
is a three-credit course intended to prepare students to work with children at a public school. The
class begins with a lecture on service-learning. The next two classes consist of a two-hour
orientation by guest speakers representing the public schools in the Chester-Upland School
Page 4 of 18
406
Advances in Social Sciences Research Journal (ASSRJ) Vol.8, Issue 1, January-2021
District. Students are required to complete 15 hours of service in which they tutor or mentor
children who are may differ from them in race, ethnicity, and/or socioeconomic status. Students
are also required to answer structured reflection questions before, during, and after their service
site visits. These questions require students to apply their service experiences to the course
content, integrate course readings to support their perspectives, and reflect on their thoughts and
feelings about both the service context and the course content. The rest of the class is devoted to
lecture and discussion that correspond to topics on measurement theory, achievement tests,
developmental, behavioral, and motivational theories, and diversity in school settings. Students
also complete three examinations and a paper. The paper assignment requires students to watch a
diversity film (i.e., Stand and Deliver, Dangerous Minds) so they can acquire a deeper
understanding of educational disparities. Students then summarize the movie, describe the main
character, compare and contrast the demographic characteristics of this character to the child
with whom they were paired at the school, and apply psychological theories from various scholars
(i.e., Bruner, Erikson, Piaget, Vygotsky, and Werner) to explain the development of the main
character. Students are required to describe what learned about multicultural education in a
public school. Finally, students take part in a closing reception at the service placement and
participate in a reflection on their learning throughout the semester.
Cultural-Based Service-Learning (Simons et al., 2012). Multicultural Psychology is a three-credit
course that fulfills a distribution requirement in the psychology and African and African American
Studies curricula. In-class time begins with a discussion on student concerns about taking this
class, guidelines for this course, and a lecture on multiculturalism. The next two classes consist of
an orientation on service-learning activities by guest speakers representing different placement
sites. Students are required to complete 15 hours of service in which they tutor or mentor children
who may differ from them in race, ethnicity, and/or socioeconomic status in either elementary or
secondary public schools in either the Chester Upland or Philadelphia School Districts. The rest of
the course is devoted to lecture, reflective and experiential activities, and discussion. Topics
covered in detail include multicultural psychology, stereotypes, ageism, sexism, and other isms,
classism and racism, racial identity development, oppression and privilege, prejudice reduction,
and cultural competence. Experiential activities (i.e., crossing-the-line) and video-clips (i.e., People
Like Us, Blue Eyed) are used to stimulate reflection and discussion.
Students are required to complete a multicultural observation paper, a movie critique of a
diversity film, an intercultural interview paper, and reflections about their course and service
experiences. The multicultural observation is designed as an immersion experience. Students
attend an activity associated with a culture or ethnic group that is distinctively different from
them. For example, some students attend a church service other than their own, dine at a
restaurant that serves ethnic food, or go to a part of the community or city to which they have
never been. Next, students write a short description about what they did, how it felt while they
were doing it, and what they learned. Students are also required to watch a diversity film (i.e.,
Crash, Save the Last Dance), apply diversity theories to explain the main theme of the movie, and
describe what they did or did not learn in terms of racial identity development and multicultural
competence (i.e., awareness, knowledge, & skills). In addition, students are required to complete
an intercultural interview paper. This assignment requires students to develop an interview on
any topic related to multicultural psychology (i.e., classism, ageism, racism), interview two
individuals who differ in one cultural characteristic (i.e., age, race, religion, sexuality, nationality,
Page 5 of 18
URL: http://dx.doi.org/10.14738/assrj.81.8457. 407
Simons, L. (2021). Student Learning Outcomes. An Evaluation of Academic and Cultural-Based Service-Learning as Critical Pedagogies for Enhancing
Student Learning. Advances in Social Sciences Research Journal, 8 (1) 403-420.
education, gender, or socioeconomic status), and compare and contrast their responses. Students
integrate theory and research to explain the main findings from the interviews. Students are also
required to complete structured reflection questions after each class and service experience so
they can critically analyze their thoughts and feelings about race and class concepts within the
course and service context over the semester. The course ends with a social network activity and a
reflective discussion about how student concerns about taking this class have changed throughout
the semester.
Measures and Procedure
A Demographic Questionnaire, developed by the researchers, was used to gather information on
gender, race, age, and year in school.
The Civic Attitudes and Skills Questionnaire (CASQ), developed by Moely, Mercer, Ilustre,
Miron, and McFarland (2002b), assessed civic attitudes and skills. The CASQ, an 84-item self- report questionnaire, yields scores on six scales: 1. Civic Action (respondents evaluate their
intentions to become involved in the future in some community service); 2. Interpersonal and
Problem-Solving Skills (respondents evaluate their ability to listen, work cooperatively,
communicate, make friends, take the role of the other, think logically and analytically, and solve
problems); 3. Political Awareness (respondents evaluate their awareness of local and national
events and political issues); 4. Leadership Skills (respondents evaluate their ability to lead); 5.
Social Justice Attitudes (respondents rate their agreement with items expressing attitudes
concerning the causes of poverty and misfortune and how social problems can be solved); and 6.
Diversity Attitudes (respondents describe their attitudes toward diversity and their interest in
relating to culturally different people). Internal consistencies for each scale reported by Moely et
al. (2002) ranged from .69 to .88, and test-retest reliabilities for each scale ranged from .56 to .81.
This scale has a strong level of reliability (α=.93) among the current student group. The
interpersonal problem-solving skills and social justice attitudes subscales were used in this study.
The Color-Blind Racial Attitude Scale (CoBRAS), developed by Neville, Lilly, Duran, Lee, and
Browne (2000), assessed contemporary racial attitudes. The CoBRAS, a 20-item self-report
measure, yields scores on three scales: 1. Unawareness of Racial Privilege (respondents evaluate
their lack of awareness of White racial privilege); 2. Unawareness of Institutional Discrimination
(respondents evaluate their lack of awareness of racial issues associated with social policies,
affirmative action, and discrimination); and 3. Unawareness of Blatant Racial Issues (respondents
evaluate their lack of awareness of blatant racial problems in the United States). Cronbach’s
coefficient alpha for each scale ranged from .86 to .88 (Neville, Lilly, Duran, Lee, & Browne, 2000).
The Multicultural Counseling Inventory (MCI), developed by Sodowsky, Taffe, Gutkin, and Wise
(1994) measured cultural competence on four scales: 1. Awareness (respondents assess the
degree of their cultural awareness); 2. Knowledge (respondents assess the degree of their cultural
knowledge); 3. Skills (respondents assess the degree of their cultural skills); and 4. Relationship
(respondents assess their interactional process and relationships with others who differ from
them). Cronbach's coefficient alpha for each scale ranged from .68 to .80. This scale has a strong
level of reliability (α = .81) among the current student group. The awareness, knowledge and
relationship subscales were used in this study.
Page 6 of 18
408
Advances in Social Sciences Research Journal (ASSRJ) Vol.8, Issue 1, January-2021
The Multicultural Environmental Inventory (MEI), developed by Pope-Davis, Liu, Nevitt, and
Toporek. (2000) assessed the degree to which multiculturalism is integrated in the curriculum on
four subscales: 1. Curriculum and Supervision (respondents assess the degree to which
multicultural issues are integrated into the course); 2. Climate and Comfort (respondents assess
the degree to which their comments are valued in class); 3. Honesty in Recruitment (respondents
assess the degree to which they are honest about the climate when recruiting); and 4.
Multicultural Research (respondents assess how much multicultural issues are integrated in
research). The internal consistency reliability estimates for the four subscales ranged from .83 to
.92. The curriculum and supervision and the climate and comfort subscales that were used in this
study were slightly modified to evaluate the course content and classroom climate in an
undergraduate diversity course.
The Multigroup Ethnic Identity Measure (MEIM), developed by Phinney (1992), measured two
aspects of students' ethnic identity: 1. Ethnic Identity Achievement based on exploration and
commitment; and 2. Sense of Belonging to and attitudes toward, one's ethnic group. Mean scores
were calculated to produce two subscale scores. Reliability for this scale is strong (alpha = .80).
The Scale of Ethnocultural Empathy, developed by Wang, Davidson, Yakushko, Savoy, Tan, and
Bleier (2003), cultural empathy on four subscales: 1. Empathetic Feeling and Expression
(respondents assess their ability to take a position when they are offended by a joke or comment
about a group who culturally differs from them); 2. Empathetic Perspective-Taking (respondents
assess the degree to which they can put themselves in the shoes of someone who is culturally
different); 3. Acceptance of Cultural Differences (respondents assess their acceptance of others
who culturally differ from them); and 4. Empathetic Awareness (respondents assess their ability
to recognize how society portrays people based on racial or ethnic stereotypes). Items are added
together to produce a total scale and four subscales. Cronbach's coefficient alpha for the total scale
and the four subscales ranged from .71 to .91. The four subscales were used in this study.
The Socially Responsible Leadership Scale, developed by Dugan (2006), measured
characteristics associated with leadership on eight subscales: 1. Consciousness of Self
(respondents assess their level of comfort in expressing oneself); 2. Congruence (respondents
assess their ability to take a stand when I believe in something); 3. Commitment (respondents
assess their ability to follow through on tasks); 4. Common Purpose (respondents assess their
ability to work with others who share collective values); 5. Collaboration (respondents assess
their belief in having better outcomes as a result of people working together); 6. Controversy with
Civility (respondents assess the belief that hearing differences in opinions enriches thinking); 7.
Citizenship (respondents assess the importance of playing an active role in communities); and 8.
Change (respondents assess the degree to which they work well in changing environments). The
internal consistency for each scale ranged from .69 to .92. The citizenship subscale was used in
this study and has strong level reliability (α=.90).
The Three Aspects of Engagement, developed by Gallini and Moely (2003), assessed students'
views of their engagement. This 27-item self-report questionnaire yields scores on three scales: 1.
Community Engagement (respondents evaluate the extent to which their attitudes changed as a
result of course participation, working with people of different backgrounds, and feeling
connected to the community): 2. Academic Engagement (respondents describe their satisfaction
Page 7 of 18
URL: http://dx.doi.org/10.14738/assrj.81.8457. 409
Simons, L. (2021). Student Learning Outcomes. An Evaluation of Academic and Cultural-Based Service-Learning as Critical Pedagogies for Enhancing
Student Learning. Advances in Social Sciences Research Journal, 8 (1) 403-420.
with the academic course and university, and their connectedness to their studies and field of
interest): and 3. Interpersonal Engagement (respondents evaluate the course's influence on their
ability to effectively work with others, communicate with other students, and make friends). Items
are added together to produce three subscale scores. Cronbach's coefficient alpha for each scale
ranged from .85 to .98. The community and interpersonal engagement subscales were used in this
study.
A cross-sectional research design was used to measure differences in student learning for
students enrolled in psychology courses that utilize ABSL or CBSL as their primary pedagogical
strategy from 2002 through 2015. All students completed an informed consent form, a
demographic information sheet, and a survey that measured Civic Action Skills Questionnaire
(CASQ), the Color-blind Attitudes Scale (CoBRAS), the Multicultural Counseling Inventory (MCI),
the Multicultural Environmental Inventory (MEI), the Multigroup Ethnic Identity Measure (MEIM),
the Scale of Ethnocultural Empathy (SEE), the Socially Responsible Leadership Scale (SRLS), and
the Three Aspects of Engagement. Students completed the survey in class and gave it directly to
the researcher. An online version replaced the paper-and-pencil survey in spring 2010. Students
were given a link to access the survey at the beginning and end of the term.
RESULTS
A repeated measures analyses of covariance (ANCOVAs) was conducted to measure differences in
student learning outcomes (i.e., CASQ, CoBRAS, MCI, MEI, MEIM, SEE, SRLS, and Three Aspects of
Engagement) between academic- and cultural-based service-learners from the beginning to the
end of the semester. The type of service-learning course (i.e., ABSL, CBSL) was used as the
independent variable, pretest and posttest scores were used as dependent variables, and previous
service-learning courses served as the covariate. The significant interaction and main effects are
shown in Tables 1 and 2.
Bonferroni t tests were conducted to detect differences in adjusted means for academic- and
cultural-based service-learners while controlling the Family Wise (FW) Error Rate (Howell, 1992).
Results from pairwise comparisons (p<.05) indicate that cultural-based service-learners had
greater awareness of White privilege and understanding of gender and racial discrimination by
the end of the semester compared to academic-based service-learners after controlling for prior
service-learning. One-Way Analyses of Variance (ANOVA) were conducted on posttest scores to
further control for Type I errors. ANOVAs are shown in Table 3. Cultural-based service-learners
had higher posttest White privilege, social justice, multicultural relationship, ethnocultural
empathetic expression, perspective-taking and interpersonal engagement scores compared to
academic-based service-learners. Finally, main effects for time indicate that students increased
their awareness of White privilege, understanding of gender and racial discrimination, and
awareness of institutional discrimination from the beginning to the end of the semester regardless
if students are exposed to ABSL or CBSL.
Independent t-test were conducted to measure differences in post-test student learning outcomes
for students with and without prior service-learning. Students with prior service-learning
(
=51.11,
= 20.96) had higher post-test scores for interpersonal problem-solving skills t (238)
= -3.91, p<.001 and social justice attitudes t (251) = -4.21, p<.05 compared to those students
without prior service (
=47.01,
=16.24). Students with prior service (
=15.61,
=17.32) also
Page 9 of 18
URL: http://dx.doi.org/10.14738/assrj.81.8457. 411
Simons, L. (2021). Student Learning Outcomes. An Evaluation of Academic and Cultural-Based Service-Learning as Critical Pedagogies for Enhancing
Student Learning. Advances in Social Sciences Research Journal, 8 (1) 403-420.
beginning of the semester, cultural-based service-learners made more positive appraisals of their
own attitudes and skills by the end of the semester compared to academic-based service-learners.
Compared to academic-based service-learners, cultural-based service-learners increased their
intercultural relationships, understanding of social justice issues in the community, and
interpersonal engagement by the end of the course. This group of service-learners also increased
their awareness of White privilege, acquired perspective-taking skills, and improved their ability
to express empathetic feelings. Academic- and cultural-based service-learners mentored and
tutored children who culturally differed from them throughout the semester and connected their
service experiences to the course content through reflections, discussions and assignments. Unlike
students in the ABSL course, students in the CBSL course were required to systematically analyze
race and class concepts in each class. Experiential and reflection activities were used daily to
challenge assumptions; while, documentaries, role-playing and other connective activities were
used to demonstrate how students are similar to and different from each other. The combination
of the diversity content and service context may have contributed to cultural-based service- learners’ improved attitudes and skills. It is further possible that students formed relationships
with peers because of class assignments also contributing to their interpersonal engagement
skills. For instance, some students visited each other’s religious organization as part of the
multicultural observation assignment; while, other students interviewed each other for the
multicultural interview assignment. In addition, White privilege was a major topic covered in the
Multicultural Psychology course. Cultural-based service-learners learned and practiced
perspective-taking skills through small and large group activities. Cultural-based service-learners
also learned how to address stereotypes through role-playing and reflection activities. It is not
surprising that cultural-based service-learners increased their awareness of White privilege,
improved their perspective-taking skills, and acquired an ability to recognize and challenge
stereotypes, because these skills reflect the diversity content and instructional strategies that
were used in the Multicultural Psychology course. Our findings are consistent with previous
research that suggests service-learning is a promising pedagogy for student engagement in
courageous conversations about cultural privilege, but only if diversity is intentionally addressed
in the course through the content, activities, and assignments (Baldwin et al., 2007; Dunlap et al.,
2007).
Critical service-learning is another term that is used to describe the combination of service- learning, course content, and instructional strategies to intentionally challenge students to think
critically about diversity and social justice issues (Mitchell, 2008). Critical service-learning is
related to student learning outcomes of cultural competence and civic engagement (Harkins et al.,
2018). Student learning outcomes related to cultural competence and civic engagement are also
known as critical benefits (Harkins et al., 2018). Within this context, CBSL is a form of critical
service-learning and student learning outcomes are considered critical benefits. In contrast to
critical service-learning, there were no significant differences in student learning outcomes
between students who did and did not take diversity courses as part of their undergraduate
studies except for community engagement. Students who participated in a diversity course
reported feeling more connected to the community and were more likely to change their
perceptions as a result of the course compared to students who did not participate in a diversity
course, incongruent with previous research suggesting that diversity courses foster an enriched
learning environment that challenges students to examine issues related to cultural privilege
(Case, 2007; Prieto, 2010). Few students in the current study reported taking a diversity course
Page 10 of 18
412
Advances in Social Sciences Research Journal (ASSRJ) Vol.8, Issue 1, January-2021
which may explain the discrepancies in the findings from the current study and in previous
research. It is further possible that students who reported taking a diversity course may have
taken a didactic course in a discipline outside of psychology as part of their liberal arts studies
(i.e., Modern Language, Art History). Didactic courses may not challenge students to examine their
own assumptions related to cultural privilege.
A third and final objective was to measure differences in student learning outcomes for
underclassmen and upperclassmen. Upperclassmen enrolled in ABSL and CBSL courses showed an
expected change in their racial-cultural-ethnic identity development compared to underclassmen
from the beginning to the end of the semester, congruent with previous research on racial identity
interaction paradigms (Cross, 1991; Helms, 1990). Helms (1990) proposes that White students
transform their White racial identity development through intercultural interactions with others
who are racially and culturally different from them. Intercultural interactions enhance identity
development for White students. White students transform their views of themselves as privileged
and colorblind to racialized and less racist. Upperclassmen may have had more opportunities for
formal and informal intercultural interactions, thus contributing to their development of their
racial-cultural-ethnic identity development.
In addition, underclassmen enrolled in ABSL and CBSL courses showed unexpected changes in
their evaluation of the integration of diversity content and ethnocultural awareness of racial and
ethic stereotypes over the semester compared to upperclassmen. It is conceivable that freshmen
and sophomores had limited exposure to diversity and social justice issues prior to college. These
students may have rated the integration of such material in both Educational and Multicultural
Psychology courses stronger than upperclassmen because of their limited knowledge.
Underclassmen also increased their ethnocultural awareness of racial and ethnic stereotypes
compared to upperclassmen. Underclassmen’s exposure to diversity content in undergraduate
psychology courses may have contributed to their increased ethnocultural awareness, consistent
with previous research on multicultural and ethnocultural awareness (Sodowsky et al., 1994;
Wang et al., 2003).
Congruent with previous service-learning research (Eyler & Giles, 1999), upperclassmen and
underclassmen increased their interpersonal problem-solving skills, interpersonal and
community engagement, and civic responsibility scores from the beginning to the end of the term.
Both student groups also evaluated Educational and Multicultural Psychology courses as
encouraging student perspectives and increased their awareness of White privilege, institutional
discrimination, and gender and racial inequality over time. Our findings indicate that there are
different yet important critical benefits from service-learning for upperclassmen and
underclassmen. Freshmen and sophomores acquired ethnocultural awareness; while, juniors and
seniors developed their cultural-ethnic-racial identity. These student learning outcomes indicate
that the developmental levels of students influence how students engage in ABSL and CBSL
courses. Therefore, beginning and ending psychology courses should incorporate service-learning
pedagogies to address the different development levels and learning needs of upperclassmen and
underclassmen.
Engaging students in courageous conversations about cultural privilege is more than getting
students to be active learners in class (Prieto, 2010). Educators need to challenge students to
Page 11 of 18
URL: http://dx.doi.org/10.14738/assrj.81.8457. 413
Simons, L. (2021). Student Learning Outcomes. An Evaluation of Academic and Cultural-Based Service-Learning as Critical Pedagogies for Enhancing
Student Learning. Advances in Social Sciences Research Journal, 8 (1) 403-420.
examine their own privilege, as well as their assumptions about oppression. They will also need to
teach active listening and perspective-taking skills so students can engage in open dialogues about
racial and cultural privilege. ABSL and CBSL are pedagogical techniques that can be used to foster
student engagement in discussions about cultural privilege. However, CBSL intentionally
addresses diversity and may be more effective for nurturing student engagement in these
discussions. Educators should consider infusing both service-learning (i.e, ABSL) and critical
service-learning (i.e., CBSL) pedagogies in undergraduate psychology curricula and courses. It may
be advantageous to use a developmental approach for infusing service-learning in undergraduate
psychology courses. ABSL could be used in beginning and CBSL could be incorporated in ending
psychology courses, because the combination of the course content and service context
contributes to different student learning outcomes. Educators should also develop psychology
courses that are specific for juniors and seniors. Middle-level psychology courses should include
student learning objectives and outcomes that align with critical benefits. Critical service-learning
pedagogies should be infused in middle-level psychology courses. Exposing students to service- learning and critical service-learning pedagogies in beginning, middle-level, and ending
psychology courses may contribute to sustained engagement beyond their academic studies.
Our findings add to the research on service-learning as a critical pedagogy for underclassmen and
upperclassmen in psychology courses; however, the results should be viewed considering a few
limitations. An overwhelming number of participants in this study were freshmen and
sophomores, and, as a result, there is the potential for Type I errors to be associated with the
results for underclassmen and upperclassmen. In addition, there were unequal groups of students
who did and did not take a diversity course as part of their undergraduate studies. In fact, very
few students reported taking a diversity course. Type II errors may be associated with student
learning outcomes. It is possible that there are differences in students learning outcomes beyond
community engagement for students who do and do not take diversity courses. More research is
needed on diversity learning and student learning outcomes. Data was collected with self-report
measures from a large sample of students enrolled in service-learning courses taught by the same
instructor over the past decade. Students reported on measures that inquired about color-blind
attitudes, intercultural relationships, and ethnocultural empathy at different points in time. Most
participants were White and female and came from middle-class backgrounds. Students of Color
and students from diverse backgrounds may have answered the questions inquiring about White
privilege, intercultural relationships, and perspective-taking differently compared to White female
students from middle-class backgrounds. Self-report limitations and social desirability effects are
most likely associated with reports of student learning. Researchers may want to expand this
study by having an equal number of upperclassmen and underclassmen in ABSL, CBSL, and
diversity courses to draw more definitive conclusions.
In conclusion, teaching psychology courses that promote the goals of cultural competence and
social responsibility is both challenging and rewarding. ABSL has the potential; while, CBCL is a
promising pedagogy that promotes student learning. Students increased their intercultural
relationships, acquired an understanding of White privilege and social justice issues in the
community, developed perspective-taking skills, and improved their ability to express empathetic
feelings through participation in a CBSL course. Engaging students in courageous conversations
about cultural privilege is also a demanding yet an enriching experience. CBSL and ABSL can foster
student engagement in these discussions for underclassmen and upperclassmen. Freshmen and
Page 13 of 18
URL: http://dx.doi.org/10.14738/assrj.81.8457. 415
Simons, L. (2021). Student Learning Outcomes. An Evaluation of Academic and Cultural-Based Service-Learning as Critical Pedagogies for Enhancing
Student Learning. Advances in Social Sciences Research Journal, 8 (1) 403-420.
Gallini, S. M., & Moely, B. E. (2003). Service-learning and engagement, academic challenge, and retention. Michigan
Journal of Community Service-Learning, 10 (1), 5-14.
Harkins, D. A., Kozak, K., & Ray, S. (2018). Service-Learning: A case study of student outcomes.
Journal of Service-Learning in Higher Education. Retrieved from https://eric.ed.gov/?id=EJ1193091.
Helms, J. E. (1990). Black and white racial identity: Theory, research, and practice. Westport, CT: Greenwood Press.
Hess, D. J., Lanig, H., & Vaughan, W. (2007). Educating for equity and social justice: A conceptual model for cultural
engagement. Multicultural Perspectives, 9 (1), 32-39.
Howell, D.C. (1992). Statistical methods for psychology. Massachusetts: PWS-KENT.
Landrum, R.E. (2011). Documenting hthe benefits of student engagement: Assessing student learning outcomes. . In
R.L. Miller, E. Amsel, B. M. Kowalewski, B.C., Beins, & B. F. Peden (Eds.), Promoting student engagement (Vol. 1, pp.34-
38). Retrieved from the Society of Teaching for Psychology. Retrieved
from:http://teachpsych.org/ebooks/pse2011/index.php
Moely, B. E., Mercer, S. H., Ilustre, V., Miron, D., & McFarland, M. (2002). Psychometric properties and correlates of the
civic attitudes and skills questionnaire (CASQ): A measure of student’s attitudes related to service-learning. Michigan
Journal of Community Service Learning, 8(2), 15-26.
Molee, L. M., Henry, M. E., Sessa, V. I., & McKinney-Prupis, E. R. (2010). Assessing learning in service-learning courses
through critical reflection. Journal of Experiential Education, 33(3), 293-257.
Mitchell, T.D. (2010). Challenges and possibilities: linking social justice and student learning. Michigan Journal of
Community Service Learning, 17 (1), 94-97.
Neville, H. A., Lilly, R. L., Duran, G., Lee, R. M., & Browne, L. (2000). Construction and initial validation of the Color- Blind Racial Attitude Scale (CoBRAS). Journal of Counseling Psychology, 47, 59-70.
Phinney, J. (1992). The Multigroup Ethnic Identity Measure: A new scale for use with adolescents and young adults
from diverse groups. Journal of Adolescent Research, 7, 156-176.
Pope-Davis, D.B., Liu, W.M., Nevitt, J., & Toporek, R.L. (2000). The development and validation of the multicultural
environmental inventory: A preliminary investigation. Cultural Diversity and Ethnic Minority Psychology, 6 (1), 57-64.
Prieto, L. (2011). Engaging students: Issues of cultural privilege. In R.L. Miller, E. Amsel, B. M.
Kowalewski, B.C., Beins, & B. F. Peden (Eds.), Promoting student engagement (Vol. 1, pp.34-38).
Retrieved from the Society of Teaching for Psychology. Retrieved from: http://teachpsych.org/ebooks/pse2011/index.php
Sodowsky, G. R., Taffe, R. C., Gutkin, T. B., & Wise, S. L. (1994). Development of the multicultural
counseling inventory: A self-report measure of multicultural competencies. Journal of Counseling Psychology, 41 (2),
137-148.
Sperling, R. (2007). Service-learning as a method of teaching multiculturalism to White college students. Journal of
Latinos and Education, 6 (4), 309-322.
Wang, Y.W., Davidson, M.M., Yakushko, O.F., Savoy, H.B., Tan, J.A., & Bleier, J. K. (2003). The scale of ethnocultural
empathy: Development, validation, and reliability. Journal of Counseling Psychology, 50 (2), 221-234.
Page 14 of 18
416
Advances in Social Sciences Research Journal (ASSRJ) Vol.8, Issue 1, January-2021
Table 1. Repeated Measures Analysis of Covariance of Pretest and Posttest Scores as a Function of
Type of Service-Learning with Prior Service-Learning Courses as Covariates
Source df SS MS F
CoBras
White Privilege 1 190.19 190.19 8.61**
Prior Service-Learning (SL) 1 3.50 3.50 .09
Course 1 285.44 285.44 7.68**
WP x Prior SL 1 86.38 86.38 3.91*
WP x Course 1 .00 .00 0.00
Error 151 3332.45 22.06
CoBras Total Score 1 501.83 501.83 7.53**
Prior SL 1 99.27 99.27 .38
Course 1 1098.53 1098.53 8.53**
CoBras Total x Prior SL 1 280.91 280.91 4.11*
CoBras Total x Course 1 .45 .45 0.00
Error 141 18146.96 128.70
Note. 1Differential change by service-learners, reflected in an interaction of Group by Time, at
**<.01, *p<.05. ANOVA F ratios are Wilk’s approximation for Group X Time interactions.
2Differential change as a function of group, p<.05. 3Differential change as a function of time, p<.05.
CoBRAS rating scales 1 strongly disagree; 6 strongly agree). Lower posttest scores indicate an
increase in awareness of racial privilege, institutional discrimination, and blatant racial issues.
MEIM rating scales: 1= strongly disagree; 5 = strongly agree.
Table 2. Pretest and Posttest Mean Scores and Standard Deviations as a Function of Type of
Service-Learning
Variable Pretest Posttest
M SD M SD
CoBras
ABSL 21.26 5.39 19.21 5.33
CBSL 18.91 5.72 17.38 5.43
Total 20.35 5.62 18.50 5.42
CoBras Total Score
ABSL 54.34 8.95 51.52 10.10
CBSL 49.54 9.60 47.50 11.47
Total 52.51 9.47 49.99 10.79
Page 15 of 18
URL: http://dx.doi.org/10.14738/assrj.81.8457. 417
Simons, L. (2021). Student Learning Outcomes. An Evaluation of Academic and Cultural-Based Service-Learning as Critical Pedagogies for Enhancing
Student Learning. Advances in Social Sciences Research Journal, 8 (1) 403-420.
Table 3. Means, Standard Deviations and One-Way Analysis of Variance for Posttest Student
Learning Outcomes as a Function of ABSL and CBSL
Variables M SD df F
CASQ
Interpersonal Problem-Solving 251 .01
ABSL 48.49 8.30
CBSL 48.35 7.75
Social Justice 263 4.53*
ABSL 17.30 8.67
CBSL 19.92 9.09
CoBras
White Privilege 175 4.03*
ABSL 18.88 5.54
CBSL 17.16 5.31
Institutional Discrimination 184 1.90
ABSL 16.63 3.98
CBSL 17.74 4.64
Racism 177 1.61
ABSL 13.07 3.75
CBSL 12.32 3.32
Total Scale 169 3.63
ABSL 50.92 10.19
CBSL 47.70 11.22
MCI
Awareness 250 .11
ABSL 20.11 4.82
CBSL 20.35 5.89
Interpersonal Relationships 253 24.68***
ABSL 15.61 4.13
CBSL 18.54 4.23
SRLS
Citizenship 234 1.18
ABSL 33.78 5.26
CBSL 34.61 5.33
Note. ***p<.001, **p<.01, *p<.05.
Page 16 of 18
418
Advances in Social Sciences Research Journal (ASSRJ) Vol.8, Issue 1, January-2021
Table 3 (continued).
Variables M SD df F
MEI
Course 250 8.14**
ABSL 25.86 6.29
CBSL 28.43 4.80
Climate 249 .05
ABSL 30.12 5.92
CBSL 29.92 4.75
SEE
Acceptance 101 1.57
ABSL 9.68 1.63
CBSL 9.12 1.66
Awareness 162 .06
ABSL 15.64 3.66
CBSL 15.50 3.24
Empathetic Expression 177 4.13*
ABSL 53.43 11.11
CBSL 57.86 8.66
Perspective-Taking 173 7.77**
ABSL 17.52 4.32
CBSL 18.87 3.77
Three Aspects of Engagement
Community Engagement 191 4.03
ABSL 8.87 1.23
CBSL 8.98 1.24
Interpersonal Engagement 187 75.58***
ABSL 12.80 2.15
CBSL 17.03 4.50
Note. ***p<.001, **p<.01, *p<.05.
Page 17 of 18
URL: http://dx.doi.org/10.14738/assrj.81.8457. 419
Simons, L. (2021). Student Learning Outcomes. An Evaluation of Academic and Cultural-Based Service-Learning as Critical Pedagogies for Enhancing
Student Learning. Advances in Social Sciences Research Journal, 8 (1) 403-420.
Table 4. Means, Standard Deviations and F Ratios for Pretest and Posttest Scores for CASQ,
CoBras, MCI, MEI, MEIM, SEE, SRLS, and Three Aspects of Engagement as a Function of Year
in School
Time Points F rations
Pretest Posttest
Measures M SD M SD df Time Group Time x Group
CASQ
Interpersonal Problem-Solving 225 9.11** 12.73***.00
Underclassmen 47.71 7.81 49.43 8.80
Upperclassmen 44.36 5.55 46.01 7.85
Total 46.68 7.34 48.36 8.15
CoBras
White Privilege 151 10.94*** 12.64*** .41
Underclassmen 21.55 5.78 19.37 5.82
Upperclassmen 18.71 5.01 17.25 4.64
Total 20.36 5.63 18.48 5.44
Institutional Discrimination 170 7.89** .75 3.76
Underclassmen 19.14 3.38 18.85 3.99
Upperclassmen 19.35 4.13 17.75 4.43
Total 19.23 3.72 18.37 4.21
Total Scale 141 7.79** 8.32** .93
Underclassmen 53.80 8.70 51.98 10.96
Upperclassmen 50.81 10.29 47.06 10.00
Total 52.55 9.48 49.92 10.81
MEI
Course 156 7.86** 4.32* 10.10**
Underclassmen 27.01 4.70 27.24 5.61
Upperclassmen 27.54 4.95 23.87 7.83
Total 27.25 4.81 25.72 6.71
Climate 152 24.99*** 6.69* 1.73
Underclassmen 27.98 4.61 29.96 5.13
Upperclassmen 28.81 4.38 32.21 5.50
Total 28.34 4.51 30.92 5.39
Note. ***p<.001, **p<.01, *p<.05.