For mixed methods research in psychology to expand, a body of psychologists, and psychology academics who have the knowledge and expertise to conduct and review mixed methods research is required. Reviews of mixed methods articles in psychology (e.g., Bartholomew and Brown, 2012; Bartholomew and Lockard, 2018) have highlighted issues related to the lack of clarity of methods used in published mixed methods psychological research. These include the failures to identify the type of mixed method design, the mixed methods research question, the qualitative analysis methodology, and to explicitly state the process for integrating data. These findings highlight the lack of training for psychologists in both conducting and reviewing mixed methods research. Whilst the new American Psychological Association reporting standards for mixed methods research (Levitt et al., 2018) are helpful for authors and reviewers with some mixed methods expertise, they must be complemented by at least foundational training in mixed methods research to ensure the quality and rigor of mixed methods research published in psychology journals.
In order to develop a body of psychologists and psychology academics who have the knowledge and expertise to conduct and review mixed methods studies, we need to teach current and future generations of psychology students about mixed methods research. At present, mixed methods research training, where offered at all, is typically provided at the doctoral level and across disciplines (Christ, 2009; Baran, 2010; Poth, 2014). Whilst Onwuegbuzie et al. (2011) argued that we can expect mixed methods to be routinely taught in the majority of higher degree by research programs in the future, Poth (2014) viewed one-off mixed methods courses as inadequate preparation, calling instead for the teaching of mixed methods research to begin earlier, positioned alongside the teaching of qualitative and quantitative research. Similarly, while not specific to psychology, the Mixed Methods Task Force report on the future of mixed methods (Mertens et al., 2016a) has recommended the inclusion of mixed methods approaches in the undergraduate curriculum, describing this as “the natural starting point” (p. 19).
We echo the sentiment of the Task Force by calling for the inclusion of mixed methods research in the undergraduate psychology curriculum, beginning with teaching an appreciation of mixed methods research. The last decade has seen an increase in the number of psychology undergraduate programs in the United Kingdom, United States of America and Canada1 teaching both quantitative and qualitative research methods (Gibson and Sullivan, 2018; McMullen and Winston-Proctor, 2018). Although these methods are often poorly integrated, and many educators tend to focus excessively on differences between qualitative and quantitative paradigms (Fielden et al., 2012), this teaching does provide a foundation for the introduction of mixed methods research. As exposure to quantitative and qualitative research increases, psychology students' perceptions of the incompatibility of qualitative and quantitative research methods decrease (Roberts and Povee, 2014). While the teaching of both quantitative and qualitative research methods equips students for multimethods research (the use of multiple methodologies to address different goals within a research project), mixed methods research varies from multimethods research in the focus on integration, as both qualitative and quantitative components are addressing the same objective (Anguera et al., 2018). Integration can occur during design, methods, interpretation or reporting (Fetters et al., 2013) and can result in additional insights and understanding. When moving from teaching qualitative and quantitative research methods separately to teaching mixed methods research, it is this mixing and integration that needs to be taught. Previous research has demonstrated that undergraduate psychology students are receptive to mixed methods, despite some misconceptions about mixed methods research and skepticism about the motivation and practices of mixed methods researchers (Povee and Roberts, 2015) that may stem from their unfamiliarity with this type of research.
Burgeoning literature on mixed methods research pedagogy at the postgraduate level (e.g., Earley, 2007; Christ, 2009; Baran, 2010; Onwuegbuzie et al., 2011; Frels et al., 2012, 2014; Poth, 2014) can be used to inform the development of mixed methods research courses at the undergraduate level. Frels et al. (2014) reported that leading mixed methodologists emphasized application and integration as learning goals when teaching mixed methods research. However, there is no “one size fits all” in terms of how mixed methods training can be delivered. Research conducted with experienced mixed methods teachers (Onwuegbuzie et al., 2011) indicates that courses vary in their orientation (importance placed on coverage of quantitative and qualitative methodologies before covering mixed methods), application (from conceptual to applied), and structure (from highly structured to exploratory/experiential), highlighting the possible range of teacher positions for mixed methods courses. This diversity of teaching approaches reflects the diversity of mixed methods research designs. Regardless of the teaching approach selected, Mertens et al. (2016a) highlight the role of educators to support students in aligning mixed methods research choices with philosophical assumptions.
A limited number of example models (e.g., Onwuegbuzie et al., 2013; Ivankova and Plano Clark, 2018) and syllabi (e.g., Earley, 2007; Christ, 2009) for mixed methods research courses are available. We encourage academics who teach mixed methods to undergraduates to share their syllabi, resources and experiences. We note that there are currently no materials that relate to teaching mixed methods research in the “statistics and research methods” teaching resources and “research methods” project syllabi sections of the Society for the Teaching of Psychology website (http://teachpsych.org/)2. The recently published Oxford Handbook of Undergraduate Psychology Education (Dunn, 2015) is similarly silent with regard to teaching mixed methods.
Whilst the availability of teaching resources, example syllabi and authoritative guides will no doubt encourage some instructors to introduce mixed methods into undergraduate research methods classes, the inclusion of mixed methods in our accreditation guidelines and textbooks will have a much larger and more immediate impact. In Australia, where the first author works, the Australian Psychology Accreditation Council (2018) require that research methods be taught to all undergraduate psychology students, but are not prescriptive about the range of methods students should be exposed to. In the United Kingdom (home of the second author), the British Psychological Society (2017a,b) specify that undergraduates should learn how to conduct quantitative and qualitative research, but make no mention of mixed methods research. The regulatory landscape in the United States is similar (American Psychological Association, 2016). Furthermore, the current3 bestselling psychology research methods textbook on Amazon.com (Morling, 2018) does not include any coverage of qualitative, let alone mixed methods research. Coolican (2014), the current best seller on Amazon.co.uk, does provide coverage of both, although mixed methods is relegated to a handful of brief mentions in chapters on measurement and qualitative methods.
The positioning of mixed methods teaching within the undergraduate psychology curriculum needs careful consideration. Ideally, the way we teach research methods will change, integrating the teaching of qualitative and quantitative methods from the beginning, rather than teaching each separately and in opposition to each other (for an illustration of this approach, see Onwuegbuzie et al., 2010). This will overcome two key challenges currently faced in teaching mixed methods: the preconceived bias and misperceptions of students about quantitative and qualitative methods and the diversity of competence in qualitative and quantitative research of students entering postgraduate mixed methods research courses (Frels et al., 2012).
Where this restructuring of the research methods curriculum is not possible, there are a number of alternatives. First, it is possible to teach quantitative, qualitative and mixed methods courses sequentially throughout the undergraduate curriculum, such as is currently done at the first author's institution. A potential consequence of this sequential approach is that quantitative research is set up as the main research methodology for psychology, with qualitative research then seen as an alternative approach (Roberts and Castell, 2016). Mertens et al. (2016a) note that where either quantitative or qualitative approaches are priveliged in teaching, a cultural shift will be required to support mixed methods approaches. The priveliging of quantaitive research methods over qualitative research methods in order of teaching potentially leads to a misperception that the qualitative component of mixed methods research is tokenistic (Povee and Roberts, 2015).
Beyond this, decisions on how to incorporate the teaching of mixed methods research in the undergraduate psychology curriculum can be informed by the three components of a student-centered pedagogy for research methods identified by Kilburn et al. (2014): making the research process visible, engaging students in conducting research, and reflecting on the research process. Examples of activities that fit within this framework might include embedding mixed methods assignments and presenting and discussing relevant mixed methods research findings within subject courses. Greene (2010) asks students to select a mixed methods article and lead a class discussion on the quality of the study. The first author has individual and group assessments that require students to take the perspective of a journal article reviewer and critically review mixed methods articles. These tasks explicitly ask students to reflect on how well the qualitative and quantitative components of the research haved been integrated.
Regardless of the way in which mixed methods research is implemented within the undergraduate psychology curriculum, evaluation will be essential in order to determine “what works.” Guetterman et al. (2017) have developed a reliable and validated self-rated measure of mixed methods skills that can be used to track students' self-ratings of skill development, and may be used by course coordinators to improve curricula in areas where students report they have limited skills.
We are not suggesting that introducing mixed methods to the undergraduate psychology curriculum will be an easy task. A key challenge facing mixed methods research educators is the time required to cover key quantitative and qualitative methods in addition to the mixing of methods (Onwuegbuzie et al., 2011; Frels et al., 2012). Students in graduate courses report finding mixed methods terminology and the concept of mixing paradigms challenging (Hesse-Biber, 2015; Gilmartin and Esterhuizen, 2017). This is not surprising given the “kaleidoscope” of mixed methods approaches and terminology (Mertens et al., 2016b) and questions over the relevant content and skills to be taught (Mertens et al., 2016b). Teachers of mixed methods courses often have not been formally taught mixed methods themselves, and may have strengths in only qualitative or quantitative research, indicating that a team-teaching approach may in some cases be required (Hesse-Biber, 2015). Further, in our experience, not all students are interested in learning mixed methods research, with some undergraduate students already expressing a clear preference for either quantitative or qualitative research methodologies by their final undergraduate year. Despite these challenges, we envisage there will be benefits to the discipline and employers in future years in terms of producing psychology graduates who are able to select from the full range of research designs: quantitative, qualitative and mixed methods.
Statements
Author contributions
LR was responsible for leading the writing of the paper. PA contributed to the design, ideas and writing.
Conflict of interest
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
Footnotes
1.^The teaching of qualitative research methods in undergraduate psychology in other countries has not been documented in the academic literature. We acknowledge that there are likely to be large variations in the amount of qualitative research methods taught across countires, and this will impact on the feasibility of introducing mixed methods research.
2.^We note that the only graduate level research methods project syllabus listed also neglects mixed methods research.
3.^At 28 September 2018.
References
1
American Psychological Association (2016). Guidelines for the undergraduate psychology major: version 2.0. Am. Psychol.71, 102–111. 10.1037/a0037562
2
AngueraM. T.Blanco-VillaseñorA.LosadaJ. L.Sánchez-AlgarraP.OnwuegbuzieA. J. (2018). Revisiting the difference between mixed methods and multimethods: Is it all in the name?Qual. Quant. 52, 2757–2770. 10.1007/s11135-018-0700-2
3
Australian Psychology Accreditation Council (2018). Accreditation Standards for Psychology Programs. Melbourne, VIC: Author.
4
BaranM. (2010). Teaching multi-methodology research courses to doctoral students. Int. J. Mult. Res. Appro.4, 19–27. 10.5172/mra.2010.4.1.019
5
BartholomewT. T.BrownJ. R. (2012). Mixed methods, culture, and psychology: a review of mixed methods in culture-specific psychological research. Int. Perspect. Psychol.1, 177–190. 10.1037/a0029219
6
BartholomewT. T.LockardA. J. (2018). Mixed methods in psychotherapy research: A review of method (ology) integration in psychotherapy science. J. Clin. Psychol. 74, 1687-1709. 10.1002/jclp.22653
7
British Psychological Society (2017a). Standards for the accreditation of undergraduate, conversion and integrated masters programmes in psychology. Leicester, UK: Author.
8
British Psychological Society (2017b). Supplementary Guidance for Research and Research Methods on Society Accredited Undergraduate and Conversion Programmes. Leicester: Author.
9
ChristT. W. (2009). Designing, teaching, and evaluating two complementary mixed methods research courses. J. Mix. Methods Res.3, 292–325. 10.1177/1558689809341796
10
CoolicanH. (2014). Research Methods and Statistics in Psychology, 6th ed. Hove: Psychology Press.
11
DunnD. S. (2015). (Ed.). Oxford Handbook of Undergraduate Psychology Education.New York, NY: Oxford University Press. 10.1093/oxfordhb/9780199933815.001.0001
12
EarleyM. A. (2007). Developing a syllabus for a mixed-methods research course. Int. J. Soc. Res. Methodol.10, 145–162. 10.1080/13645570701334118
13
FettersM. D.CurryL. A.CreswellJ. W. (2013). Achieving integration in mixed methods designs—principles and practices. Health Serv. Res.48(6 pt 2), 2134–2156. 10.1111/1475-6773.12117
14
FieldenA. L.GoldieS.SillenceE. (2012). Taking another look: developing a sustainable and expandable programme of qualitative research methods in psychology. Psychol. Learn. Teach.11, 46–51. 10.2304/plat.2012.11.1.46
15
FrelsR. K.OnwuegbuzieA. J.LeechN. L.CollinsK. M. (2012). Challenges to teaching mixed research courses. J. Effect. Teach.12, 23–44.
16
FrelsR. K.OnwuegbuzieA. J.LeechN. L.CollinsK. M. (2014). Pedagogical strategies used by selected leading mixed methodologists in mixed research courses. J. Effect. Teach.14, 5–34.
17
GibsonS.SullivanC. (2018). A changing culture? Qualitative methods teaching in U.K. psychology. Qual. Psychol.5, 197–206. 10.1037/qup0000100
18
GilmartinJ.EsterhuizenP. (2017). Shifting pedagogical priorities in facilitating mixed methods research including postgraduate student's reflections. GSTF J. Nurs. Health Care5, 1–7. 10.5176/2345-718X_5.1.154
19
GreeneJ. C. (2010). Foreword: beginning the conversation. Int. J. Mult. Res. Appro.4, 2–5. 10.5172/mra.2010.4.1.002
20
GuettermanT. C.CreswellJ. W.WittinkM.BargF. K.CastroF. G.DahlbergB.et al. (2017). Development of a self-rated mixed methods skills assessment: The NIH mixed methods research training program for the health sciences. J. Continu. Educ. Health Profess.37, 76–82. 10.1097/CEH.0000000000000152
21
Hesse-BiberS. (2015). The problems and prospects in the teaching of mixed methods research. Int. J. Soc. Res. Methodol.18, 463–477. 10.1080/13645579.2015.1062622
22
IvankovaN. V.Plano ClarkV. L. (2018). Teaching mixed methods research: using a socio-ecological framework as a pedagogical approach for addressing the complexity of the field. International J. Soc. Res. Methodol. 21, 409–424. 10.1080/13645579.2018.1427604
23
KilburnD.NindM.WilesR. (2014). Learning as researchers and teachers: The development of a pedagogical culture for social science research methods?Br. J. Educ. Stud.62, 191–207. 10.1080/00071005.2014.918576
24
LevittH. M.BambergM.CreswellJ. W.FrostD. M.JosselsonR.Suárez-OrozcoC. (2018). Journal article reporting standards for qualitative primary, qualitative meta-analytic, and mixed methods research in psychology: the APA Publications and Communications Board task force report. Am. Psychol.73, 26–46. 10.1037/amp0000151
25
McMullenL. M.Winston-ProctorC. E. (2018). Qualitative inquiry in undergraduate psychology curricula and research training: Contexts and practices for transforming the optional into the obligatory. Qual. Psychol.5, 191–196. 10.1037/qup0000123
26
MertensD. M.BazeleyP.BowlegL.FieldingN.MaxwellJ.Molina-AzorínJ. F.et al. (2016a). The Future of Mixed Methods: A Five Year Projection to 2020. Available online at: https://mmira.wildapricot.org/resources/Documents/MMIRA%20task%20force%20report%20Jan2016%20final.pdf
27
MertensD. M.BazeleyP.BowlegL.FieldingN.MaxwellJ.Molina-AzorinJ. F.et al. (2016b). Expanding thinking through a kaleidoscopic look into the future: implications of the mixed methods international research association's task force report on the future of mixed methods. J. Mix. Methods Res.10, 221–227. 10.1177/1558689816649719
28
MorlingB. (2018). Research Methods in Psychology: Evaluating a World of Information, 3rd ed. New York, NY: W.W. Norton.
29
OnwuegbuzieA. J.FrelsR. K.CollinsK. M.LeechN. L. (2013). Conclusion: a four-phase model for teaching and learning mixed research. Int. J. Mult. Res. Appro.7, 133–156. 10.5172/mra.2013.7.1.133
30
OnwuegbuzieA. J.FrelsR. K.LeechN. L.CollinsK. M. (2011). A mixed research study of pedagogical approaches and student learning in doctoral-level mixed research courses. Int. J. Mult. Res. Appro.5, 169–199. 10.5172/mra.2011.5.2.169
31
OnwuegbuzieA. J.LeechN. L.MurtonenM.TähtinenJ. (2010). Utilizing mixed methods in teaching environments to reduce statistics anxiety. Int. J. Mult. Res. Appro.4, 28–39. 10.5172/mra.2010.4.1.028
32
PothC. (2014). What constitutes effective learning experiences in a mixed methods research course? An examination from the student perspective. Int. J. Mul. Res. Appro.8, 74–86. 10.5172/mra.2014.8.1.74
33
PoveeK.RobertsL. D. (2015). Attitudes toward mixed methods research in psychology: the best of both worlds?Int. J. Soc. Res. Methodol.18, 41–57. 10.1080/13645579.2013.872399
34
RobertsL. D.CastellE. (2016). “Having to shift everything we've learned to the side”: expanding research methods taught in psychology to incorporate qualitative methods. Front. Psychol.7:688. 10.3389/fpsyg.2016.00688
35
RobertsL. D.PoveeK. (2014). A brief measure of attitudes toward mixed methods research in psychology. Front. Psychol.5:1312. 10.3389/fpsyg.2014.01312
Summary
Keywords
mixed methods research, undergraduate psychology, research methods, pedagogy, undergraduate students
Citation
Roberts LD and Allen PJ (2019) A Call for the Inclusion of Mixed Methods Research in the Undergraduate Psychology Curriculum. Front. Psychol. 9:2709. doi: 10.3389/fpsyg.2018.02709
Received
01 October 2018
Accepted
17 December 2018
Published
08 January 2019
Volume
9 - 2018
Edited by
M. Teresa Anguera, University of Barcelona, Spain
Reviewed by
Conrad Izquierdo, Autonomous University of Barcelona, Spain; Angel Blanco-Villaseñor, University of Barcelona, Spain; Lisa Zimmerman, University of South Africa, South Africa
Updates
Copyright
© 2019 Roberts and Allen.
This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: Lynne D. Roberts lynne.roberts@curtin.edu.au
This article was submitted to Quantitative Psychology and Measurement, a section of the journal Frontiers in Psychology
Disclaimer
All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article or claim that may be made by its manufacturer is not guaranteed or endorsed by the publisher.