Page 1 of 9
Advances in Social Sciences Research Journal – Vol. 11, No. 8
Publication Date: August 25, 2024
DOI:10.14738/assrj.118.16706.
Troy, J., Grambow, S., Neely, M., Pomann, G.-M., Davenport, C., Ashner, M., & Samsa, G. (2024). Rationalizing the Mathematics
Requirements for a Master of Biostatistics Program: A Case Study and Commentary. Advances in Social Sciences Research Journal,
11(8). 265-273.
Services for Science and Education – United Kingdom
Rationalizing the Mathematics Requirements for a Master of
Biostatistics Program: A Case Study and Commentary
Jesse Troy
Department of Biostatistics and Bioinformatics,
Duke University, Durham NC, USA
Steven Grambow
Department of Biostatistics and Bioinformatics,
Duke University, Durham NC, USA
Megan Neely
Department of Biostatistics and Bioinformatics,
Duke University, Durham NC, USA
Gina-Maria Pomann
Department of Biostatistics and Bioinformatics,
Duke University, Durham NC, USA
Clemontina Davenport
Department of Biostatistics and Bioinformatics,
Duke University, Durham NC, USA
Marissa Ashner
Department of Biostatistics and Bioinformatics,
Duke University, Durham NC, USA
Greg Samsa
Department of Biostatistics and Bioinformatics,
Duke University, Durham NC, USA
ABSTRACT
Within a 2-year Master of Biostatistics program, we reconsidered the mathematics
requirements for admission, and also the premise that the most distinguishing
feature of a top candidate is deep exposure to mathematics. Our assessment took
place within a broad curriculum review intended to enhance alignment: aligning
programmatic goals with job skills, aligning the use of mathematics within our
curriculum with programmatic goals, aligning our admission criteria with our
curriculum, and aligning our application materials with these admission criteria.
We developed a specific list of mathematical skills required by the curriculum, and
are revising the application materials to include self-report on applicant's exposure
to and functional mastery of those skills. We illustrate how functional mastery is
operationally defined. Our criteria for identifying top candidates was broadened to
include those with especial skills in analytics, biology and/or communication (i.e.,
Page 2 of 9
266
Advances in Social Sciences Research Journal (ASSRJ) Vol. 11, Issue 8, August-2024
Services for Science and Education – United Kingdom
the three conceptual pillars of our program). Deep mathematical training is one of
many ways to become a top candidate. Within STEM fields such as biostatistics, and
despite the commonly held assumption that admission criteria should emphasize
depth of mathematical training, a systematic analysis suggests that this assumption
imposes a gratuitous requirement on applicants. Reconsidering this assumption
can help remove unnecessary barriers, reduce challenges, and support success for
aspiring or emerging biostatisticians from diverse or multifaceted backgrounds. It
is one way to contribute to a more inclusive and equitable learning environment in
our STEM discipline. We believe similar programs might benefit from performing
this type of analysis and reflection.
Keywords: Biostatistics, curriculum review, mathematics requirements, STEM pipeline.
INTRODUCTION
Lorem A basic premise of program development is that of constructive alignment [1] -- for
example, curriculum content should be aligned with overall educational goals, and admission
criteria should be aligned with curriculum content. Moreover, educational goals should be
realistic. For example, the curriculum of a professional master's program should be focused on
tasks that graduates will accomplish on the job (currently, and in the future), and gratuitous
requirements should be avoided.
Here, we consider a 2-year Master of Biostatistics (MB) program. Originally approved as a
terminal professional degree program, it now serves two types of students. Approximately 30%
immediately proceed to doctoral study after graduation. The other 70% enter the workforce. A
distinctive feature of the MB program is its broad focus, consistent with a mission to train
students who will contribute to interdisciplinary team science, encompassing not only
Analytical skills but Biological knowledge and Communication skills as well (i.e., the "ABCs of
biostatistics") [2-10]. Graduates often take positions with significant responsibility and
autonomy, and our intention is to train students for this type of "high-end" application.
The current case study and commentary pertain to the alignment between curriculum content
and admission criteria. Because a biostatistics curriculum is mathematically intensive students
must be "good at math", and our question is how that construct can be best operationalized.
Previously, our measure of this construct was based on grades and course lists. At a minimum,
we required calculus through multi-variable calculus and linear algebra (i.e., the two main
languages of instruction), with the default expectation that the multi-variable calculus grade
was an A. Moreover, top candidates were expected to have additional coursework in real
analysis, differential equations, etc. In other words: (1) the primary criterion for basic
competency was demonstrated excellence in multi-variable calculus; and (2) a requirement for
being considered as a top candidate was deep training in mathematics, and the deeper the
better. Other master's programs apply similar admission criteria.
We continue to believe that mastery of multi-variable calculus is an excellent measure of basic
competency. However, the seemingly intuitive criteria for identifying top candidates had some
unintended consequences. We begin by describing those consequences, and then how our
admission criteria are being modified in response. We believe similar programs could benefit
from performing this type of analysis and reflection.
Page 3 of 9
267
Troy, J., Grambow, S., Neely, M., Pomann, G.-M., Davenport, C., Ashner, M., & Samsa, G. (2024). Rationalizing the Mathematics Requirements for
a Master of Biostatistics Program: A Case Study and Commentary. Advances in Social Sciences Research Journal, 11(8). 265-273.
URL: http://dx.doi.org/10.14738/assrj.118.16706
UNINTENDED CONSEQUENCES
Our response to these unintended consequences began with a comprehensive and ongoing
curriculum review. An initial step was to identify the skill set required by our graduates and to
revise the curriculum so that (1) each important skill was addressed; and (2) less important
skills were deemphasized, eliminated, or made optional. This step led to big-picture changes
such as adding and dropping courses, revising the mapping of courses into tracks, etc. These
changes supported better alignment between the overall curriculum and overall educational
goals (i.e., made concrete by the desired student skill set).
Next, we reviewed how effectively important skills were taught within specific classes (e.g.,
active learning approaches were encouraged). This was the point at which we assessed the
requisite degree of mathematical coverage. For example, our first inference course was revised
to replace some (but certainly not all) calculus with demonstrations via simulation, and to
replace formal proofs with other intuition-building exercises [9]. This supported the second
element of alignment: namely, better alignment between course goals and course content
(especially, mathematical content).
Once this initial process of course revision was completed, we considered how mathematics is
used within individual courses. Upon reflection, two differing skill sets are needed: (1) the
general ability to use mathematics to formulate and solve statistically-based problems; and (2)
mastery of specific prerequisite knowledge required by our program. The second and more
specific skill set was simpler to operationally define.
Regarding prerequisite knowledge, we reviewed the courses that a job-bound student would
be likely to take (thus, for example, selecting elective courses with an applied focus) and
recorded the specific applications of mathematics. To our surprise, this list (Appendix 1) is not
voluminous. It contains deep working knowledge of some basic concepts, such as the nature of
functions. It also includes a relatively small subset of the material typically covered in an
undergraduate course in calculus which focuses on the trajectory of objects as they move
through space and similar concepts from physics. Additionally, it includes (1) facility with
differentiation and integration, which are required (among others) for manipulating the
probability distributions used in statistics; and (2) ability to manipulate functions -- especially,
to maximize and minimize them.
A similar phenomenon was observed for linear algebra -- rather than the entire content of a
typical linear algebra course, the information that is most important to our program is limited
to (1) the ability to manipulate matrices (e.g., to add them, to invert them); (2) the ability to
state statistical models in matrix terms (i.e., as this allows ideas to be presented succinctly and
in greater generality); and (3) the ability to project high-dimensional spaces onto lower
dimensions (i.e., this being critical to statistical modeling, and requiring facility with content
such as eigenvalues and eigenvectors).
Regarding the first skill set, the underlying construct (i.e., general ability to use mathematics to
formulate and solve statistically-based problems) was straightforward to state, but not
necessarily trivial to assess. Presumably, having good grades in undergraduate courses in
mathematics, or which extensively use mathematics, is a necessary but not fully sufficient
condition for assessing this construct, because students can master the technical elements of
Page 4 of 9
268
Advances in Social Sciences Research Journal (ASSRJ) Vol. 11, Issue 8, August-2024
Services for Science and Education – United Kingdom
mathematics as an exercise in symbol manipulation but not truly understand the underlying
concepts (and, thus, not be able to extend their knowledge to different problems). In any event,
we had created a list of what students should be able to do: namely, use calculus to accomplish
a specific set of tasks, use linear algebra to accomplish another specific set of tasks, and apply
mathematics more generally to define and solve statistical problems. For each of these
constructs, our admission materials could only provide a rough surrogate, albeit a surrogate
that we hoped to improve.
RESPONSE
Our response to these unintended consequences began with a comprehensive and ongoing
curriculum review. An initial step was to identify the skill set required by our graduates and to
revise the curriculum so that (1) each important skill was addressed; and (2) less important
skills were deemphasized, eliminated, or made optional. This step led to big-picture changes
such as adding and dropping courses, revising the mapping of courses into tracks, etc. These
changes supported better alignment between the overall curriculum and overall educational
goals (i.e., made concrete by the desired student skill set).
Next, we reviewed how effectively important skills were taught within specific classes (e.g.,
active learning approaches were encouraged). This was the point at which we assessed the
requisite degree of mathematical coverage. For example, our first inference course was revised
to replace some (but certainly not all) calculus with demonstrations via simulation, and to
replace formal proofs with other intuition-building exercises [9]. This supported the second
element of alignment: namely, better alignment between course goals and course content
(especially, mathematical content).
Once this initial process of course revision was completed, we considered how mathematics is
used within individual courses. Upon reflection, two differing skill sets are needed: (1) the
general ability to use mathematics to formulate and solve statistically-based problems; and (2)
mastery of specific prerequisite knowledge required by our program. The second and more
specific skill set was simpler to operationally define.
Regarding prerequisite knowledge, we reviewed the courses that a job-bound student would
be likely to take (thus, for example, selecting elective courses with an applied focus) and
recorded the specific applications of mathematics. To our surprise, this list (Appendix 1) is not
voluminous. It contains deep working knowledge of some basic concepts, such as the nature of
functions. It also includes a relatively small subset of the material typically covered in an
undergraduate course in calculus which focuses on the trajectory of objects as they move
through space and similar concepts from physics. Additionally, it includes (1) facility with
differentiation and integration, which are required (among others) for manipulating the
probability distributions used in statistics; and (2) ability to manipulate functions -- especially,
to maximize and minimize them.
A similar phenomenon was observed for linear algebra -- rather than the entire content of a
typical linear algebra course, the information that is most important to our program is limited
to (1) the ability to manipulate matrices (e.g., to add them, to invert them); (2) the ability to
state statistical models in matrix terms (i.e., as this allows ideas to be presented succinctly and
in greater generality); and (3) the ability to project high-dimensional spaces onto lower
Page 5 of 9
269
Troy, J., Grambow, S., Neely, M., Pomann, G.-M., Davenport, C., Ashner, M., & Samsa, G. (2024). Rationalizing the Mathematics Requirements for
a Master of Biostatistics Program: A Case Study and Commentary. Advances in Social Sciences Research Journal, 11(8). 265-273.
URL: http://dx.doi.org/10.14738/assrj.118.16706
dimensions (i.e., this being critical to statistical modeling, and requiring facility with content
such as eigenvalues and eigenvectors).
Regarding the first skill set, the underlying construct (i.e., general ability to use mathematics to
formulate and solve statistically-based problems) was straightforward to state, but not
necessarily trivial to assess. Presumably, having good grades in undergraduate courses in
mathematics, or which extensively use mathematics, is a necessary but not fully sufficient
condition for assessing this construct, because students can master the technical elements of
mathematics as an exercise in symbol manipulation but not truly understand the underlying
concepts (and, thus, not be able to extend their knowledge to different problems). In any event,
we had created a list of what students should be able to do: namely, use calculus to accomplish
a specific set of tasks, use linear algebra to accomplish another specific set of tasks, and apply
mathematics more generally to define and solve statistical problems. For each of these
constructs, our admission materials could only provide a rough surrogate, albeit a surrogate
that we hoped to improve.
MODIFYING THE ADMISSION PROCESS
Considering the above, we have made two major changes to the admission process. The first is
an attempt to better translate course titles into an assessment of the mastery of the specific
items in Appendix 1. Toward this end, we had previously asked that applicants describe the
level of rigor associated with their mathematically-related courses, and many applicants had
additionally copied course descriptions from a catalog. Starting with the next application cycle,
we will enhance this report by (1) asking students to self-rate their facility with the items in
Appendix 1; and (2) provide example responses that illustrate how we conceptualize mastery
(see Appendix 2). In other words, we will ask applicants to provide more specific self-reports
about their mastery of the content that will be important to their success in our program, and
also provide examples which help them calibrate these reports.
The second change was to reconsider what it means to be a top candidate. Rather than simply
ranking students on the number of mathematically-related courses taken as an undergraduate
(1) we continue to require all successful applicants to demonstrate basic competency in multi- variable calculus and linear algebra; and (2) for applicants satisfying this first condition, we
now define a top candidate as someone with especial skills in analytics, biology and/or
communication (i.e., the three conceptual pillars of our program). In other words, additional
mathematical training remains one way to become a top candidate, but isn't the only one.
Indeed, we had previously been making such judgments on an ad hoc basis, and have found that
doing so in a systematic, transparent and reproducible basis helps to simplify the application
review process as well as making it more consistent.
DISCUSSION
We have described the process by which our admission requirements around mathematics
were systematically reviewed and updated. Our intention is to more effectively align the use of
mathematics in our curriculum with broader programmatic goals, to better align our admission
criteria with how mathematics is used, and to better align the information in our application
materials with these admission criteria. Reconsideration of admission criteria was a final step
in a broader process of curriculum review. We were encouraged to find that the specific list of
mathematics requirements was of modest length, and favored depth of knowledge over
Page 6 of 9
270
Advances in Social Sciences Research Journal (ASSRJ) Vol. 11, Issue 8, August-2024
Services for Science and Education – United Kingdom
breadth. We also recognized a mismatch between this list of mathematics requirements and the
information currently available on our application (e.g., lists of undergraduate math-related
courses and their grades), and are revising our application materials accordingly. In part
because of the ability to perform web searches, to use generative artificial intelligence, etc., we
do not directly evaluate mathematical skills within the application (e.g., we don't ask applicants
to answer questions such as those in Appendix 2), but instead provide information about our
interpretation of functional mastery of mathematical skills to help support their own self- evaluation.
An additional benefit of this review is that potential applicants will receive more specific
information about the preparation they need to succeed in our program -- for example, to assist
them in their selection of undergraduate courses. Indeed, we are in the process of updating the
descriptions of our program (e.g., on our website) to make our mathematical requirements
more explicit.
Although not the primary rationale for revising our admission process, we note that our
approach is relevant to the question of the "shrinking pipeline in the STEM disciplines" [11-13].
As applied to mathematics and statistics, this pipeline begins in middle school (and before) with
students who are interested in mathematics. Some drop out of the pipeline in high school,
because math courses are uninteresting, irrelevant, poorly taught, and/or too hard for their
level of preparation. The same applies to their undergraduate experience, especially if they
encounter a "weed-out course", causing them to decide against a mathematically-related major.
Students with fewer math courses or having non-math majors are less likely to pursue graduate
study in a STEM field and ultimately receive a graduate degree. At each step, these challenges
disproportionately fall on students from disadvantaged groups, due to a combination of beliefs
(e.g., stereotyping), individual behavior, institutional behavior, resources (both formal and
informal) and inadequate support. The ultimate result is that too few students ultimately
succeed in the STEM fields, and also that the distribution of those who do is skewed toward
students from more advantaged backgrounds.
The points at which our program can intervene fall late in the process: (1) by broadening
admission requirements; (2) by enforcing consistency between admission requirements and
the curriculum; and (3) by striving to provide a supportive environment to all students,
recognizing that doing so requires due consideration of individual characteristics and social
context. Here, we have focused on the first two elements of the list: in essence, by returning
selected non-math majors who are traditionally assumed to have exited the pipeline back into
it, and by providing a curriculum for which they are adequately prepared. Broadening
admission requirements is especially reasonable for a "team science discipline" such as
biostatistics where a successful practitioner could, for example, build upon deep expertise in
biology and competence in mathematics rather than the reverse (which works well, too, of
course). Doing so would be less reasonable, for example, for those students who aspire to
doctoral study in theoretical mathematics, as this is a highly specialized discipline that directly
builds upon the depth of their previous mathematical training. We posit that an assumption
that is often unexamined is that biostatistics is a highly specialized and mathematically
intensive discipline rather like theoretical mathematics: this assumption holds true for a
minority of graduate programs (whose admission criteria should reflect this) but not for most
graduate programs and not for most practitioners.
Page 7 of 9
271
Troy, J., Grambow, S., Neely, M., Pomann, G.-M., Davenport, C., Ashner, M., & Samsa, G. (2024). Rationalizing the Mathematics Requirements for
a Master of Biostatistics Program: A Case Study and Commentary. Advances in Social Sciences Research Journal, 11(8). 265-273.
URL: http://dx.doi.org/10.14738/assrj.118.16706
As a practical matter, one recommendation we can provide to others is that implementing this
sort of change in admission requirements requires proper framing. In particular, the goal of
these changes was not to "admit students who are weaker at math", but instead to "enhance
consistency between what we teach and who we admit". For our program, who we admit
requires, at a minimum, acceptable background in each of the ABCs of biostatistics, with
mathematics being part of the analytic competency. Once they have entered the pool of those
with acceptable qualifications, applicants can distinguish themselves along multiple
dimensions. To those who might have worried that requirements are being weakened, we
clarified that (1) we continue to seek strong students, now with the criteria for what counts as
"strong" being broadened; and (2) every admitted student has sufficient background in
mathematics to succeed. Of course, we remain happy to admit students whose distinctive
strength falls within the domain of mathematics and whose exposure extends far beyond the
minimum.
CONCLUSION
In conclusion, we believe that removing unnecessary mathematical barriers and ensuring
alignment between our mathematics requirements, curriculum content, and admission
requirements is educationally sound and can also help increase access, reduce challenges, and
support success for aspiring or emerging biostatisticians from diverse or multifaceted
backgrounds. It is one way to contribute to a more inclusive and equitable learning
environment in our STEM discipline. We believe similar programs might benefit from
performing this type of analysis and reflection.
7 APPENDIX 1: ADMISSION CRITERIA FOR MATHEMATICS
In
Admission requirement Justification
Calculus
Good grades in math-related courses Some exposure to math is a surrogate for a generic ability
to define and solve statistical problems using math
Facility with basic mathematical concepts
such as functions
Used as a building block for what follows
Facility with integrals and derivatives for one
variable
Used for working with statistical distributions -- for
example, for transforming probability density functions to
cumulative distribution functions
Facility with integration in 2 variables
strongly preferred
Used for working with marginal and conditional
distributions
Ability to manipulate functions (e.g., to find
maxima and minima)
Used in working with likelihood functions, among others
Linear algebra
Ability to manipulate matrices (e.g., add,
invert, transpose)
Used as a building block for what follows
State models in matrix language Used to succinctly describe statistical models
Project high-dimensional data into lower
dimensions (e.g., eigenvalues, eigenvectors)
Used in multi-predictor models
Page 8 of 9
272
Advances in Social Sciences Research Journal (ASSRJ) Vol. 11, Issue 8, August-2024
Services for Science and Education – United Kingdom
8 APPENDIX 2: ILLUSTRATION OF FUNCTIONAL MASTERY OF CALCULUS
One of our admission criteria pertains to mastery of functions. To illustrate what is intended,
consider the following question:
"Is exp{-2t}, t>0, a function of t? How would you find its maximum value? Please
discuss "why" in addition to "how".
A possible answer is as follows:
A function is a rule, which takes inputs and uniquely assigns values for its outputs.
Here, the inputs are the positive numbers (i.e., "t>0"). For each value of t, the output
is f(t) = exp{-2t}. This is a special case where the usual calculus procedure for
finding a maximum of a function, which begins by finding the derivative of that
function and setting it equal to 0, doesn't work.
To see why the usual approach doesn't work, this derivative is (-2) *exp{-2t}.
According to the rules of exponents, exp{-2t} = 1 / exp{2t}, and there is no value of
x for which 1/x equals 0.
We can, nevertheless, proceed using logic. Simply plugging in values of t makes it
apparent that as t increases f(t) decreases. In essence, as t increases so does exp{2t}
and so 1 / exp{2t} decreases, as does 2 / exp{2t}. So, the maximum value of f(t)
corresponds to the smallest value of t within its range.
Now, if the range was t >= 0 rather than t > 0, we could plug t=0 into f(t) and obtain
the maximum value f(t)=1. As we allow t to decrease and approach 0, the value of
f(t) approaches 1 as closely as you like. Indeed, in the world of calculus, "approaches
1 as closely as you like" has the same interpretation as "is 1", and so the maximum
value of f(t) is 1.
Comment: To understand the principles behind the above solution, a working knowledge of the
nature of functions suggests that you should start by plotting f(t) on the y-axis and t on the x- axis and hope that this provides a clue about the location of the maximum. Plugging in a few
values of t makes it clear that f(t) decreases as t increases, and so the maximum must occur at
"t=0". This conclusion can be checked using rules of exponents and quotients, which are basic
mathematical techniques. Finally, the fundamental calculus principle of a limit translates
"approaches 1" to "equals 1".
The solution only relies upon basic mathematical manipulations, but does require the ability to
use of the construct of general mathematical thinking in order to set up the analytical approach.
In other words, general mathematical thinking suggests drawing a graph and using the shape
of that graph to discover the likely value of the maximum.
References
[1]. Biggs, J. Enhancing teaching through constructive alignment. High Educ 32, 347–364 (1996).
https://doi.org/10.1007/BF00138871
[2]. Neely ML, Troy JD, Gschwind GT, Pomann G-M, Grambow SC, Samsa GP. Preorientation curriculum: an
approach for preparing students with heterogeneous backgrounds for training in a Master of Biostatistics
Page 9 of 9
273
Troy, J., Grambow, S., Neely, M., Pomann, G.-M., Davenport, C., Ashner, M., & Samsa, G. (2024). Rationalizing the Mathematics Requirements for
a Master of Biostatistics Program: A Case Study and Commentary. Advances in Social Sciences Research Journal, 11(8). 265-273.
URL: http://dx.doi.org/10.14738/assrj.118.16706
program. Journal of Curriculum and Teaching. Online Published April 21, 2022. DOI:10.5430/jct.
v11n4p77.
[3]. Pomann G-M, Boulware LE, Chan CC, Grambow SC, Hanlon AL, Neely ML, Peskoe SB, Samsa G, Troy JD,
Yang LZ, Thomas SM. Experiential learning methods for biostatistics students: A model for embedding
student interns in academic medical centers. Stat. online 09Sep2022. https://doi.org/10.1002/sta4.506.
[4]. Samsa GP, LeBlanc TW, Locke SC, Troy JD, Pomann, GM. A model of cross-disciplinary communication for
collaborative statisticians: implications for curriculum design. 2018. Journal of Curriculum and Teaching.
Online August 3, 2018. DOI: 10.5430.jct.v7n2p1.
[5]. Samsa G. Using coding interviews as an organizational and evaluative framework for a graduate course in
programming. Journal of Curriculum and Teaching, 2020, 9(3), 107-140.
[6]. Samsa G. Evolution of a qualifying examination from a timed closed-book format to a collaborative take- home format: a case study and commentary. Journal of Curriculum and Teaching. 2021. 10(1). 47-55.
Doi:10.5430/jct. v10n1p47.
[7]. Troy JD, Neely ML, Grambow SC, Samsa GP. The Biomedical Research Pyramid: A model for the practice of
biostatistics. Journal of Curriculum and Teaching, 2021, 10(1), 10-17.
[8]. Troy JD, Granek J, Samsa GP, Pomann G-M, Updike S, Grambow SC, Neely ML. A course in biology and
communication skills for master of biostatistics students. Journal of Curriculum and Teaching. April 21,
2022. DOI: https://doi.org/10.5430/jct.v11n4p77.
[9]. Troy JD, McCormack K, Grambow SC, Pomann G-M, Samsa GP. Redesign of a first-year theory sequence in
biostatistics. Journal of Curriculum and Teaching. 2022;11(8),1-12. Doi:10.5430/jct.v11n8p1.
[10]. Troy JD, Pomann G-M, Neely ML, Grambow SC, Samsa GP. Are simulated coding interviews a fair and
practical examination format for non-professional programmers enrolled in a master's degree program in
biostatistics? Journal of Curriculum and Teaching. 2023, 11(15), DOI:10.5430/jct.v12n6p253.
[11]. Brown BA, Henderson JB, Gray S, Donovan B, Sullivan S, Patterson A, Waggstaff W. From description to
explanation: an empirical exploration of the African-American pipeline problem in STEM. Journal of
Research in Science Teaching, 2016, 53(1), 146-177.
[12]. Cannady MA, Greenwald E, Harris KN. Problematizing the STEM pipeline metaphor: is the STEM pipeline
metaphor serving our students and the STEM workforce? Science Education, 2014, 98(3), 443-460,
DOI:10.1102/sce.21108.
[13]. Estrada M, Burnett M, Campbell AG, Campbell PB, Denetclaw WF, Gutierrez CG, Hurtado S, John GH, Matsui
J, McGee RM, Okpodu CM, Robinson TJ, Summers MF, Werner-Washburne M, Zavala ME. Improving
underrepresented minority student persistence in STEM. CBE-Life Sciences Education. September 1, 2016
15:es5 DOI:10.10.1187/cbe.16-01-0038.