The Specificity of Observational Studies in Physical Activity and Sports Sciences: Moving Forward in Mixed Methods Research and Proposals for Achieving Quantitative and Qualitative Symmetry

Abstract

Mixed methods studies are been increasingly applied to a diversity of fields. In this paper, we discuss the growing use—and enormous potential—of mixed methods research in the field of sport and physical activity. A second aim is to contribute to strengthening the characteristics of mixed methods research by showing how systematic observation offers rigor within a flexible framework that can be applied to a wide range of situations. Observational methodology is characterized by high scientific rigor and flexibility throughout its different stages and allows the objective study of spontaneous behavior in natural settings, with no external influence. Mixed methods researchers need to take bold yet thoughtful decisions regarding both substantive and procedural issues. We present three fundamental and complementary ideas to guide researchers in this respect: we show why studies of sport and physical activity that use a mixed methods research approach should be included in the field of mixed methods research, we highlight the numerous possibilities offered by observational methodology in this field through the transformation of descriptive data into quantifiable code matrices, and we discuss possible solutions for achieving true integration of qualitative and quantitative findings.

Sport and physical activity as a new substantive area in mixed methods research

In the late 1990s, Biddle (1997) found very little diversity in research methods used in empirical studies in two of the most prestigious sport and physical activity journals he chose to study—The Journal of Sport and Exercise Psychology (JSEP), a leading research journal in the field, and The International Journal of Sport Psychology (IJSP), which was the first journal in this field. Most of the quantitative research was based on regression techniques and discriminant analysis, while most of the qualitative research drew on interviews and content analysis. During the same period, Morris (1999) reported that observational and case studies accounted for just 2% of scientific production in this field between 1979 and 1998.

In a study published shortly afterwards, Biddle et al. (2001) presented a detailed analysis of the methods used in both quantitative and qualitative sport and exercise psychology research, with a focus on discriminant analysis, hierarchical regression, stepwise statistical procedures (although it should be noted that stepwise procedures have been debunked by numerous statisticians; cf. Thompson, 1995; Onwuegbuzie and Daniel, 2003), and meta-analysis in the area of quantitative research and thematic analysis (mostly interviews) in the area of qualitative research. Biddle et al. (2001) words were particularly enlightening:

The extent to which such diverse approaches could or should be integrated is a matter for the reader to decide. Some have stated that qualitative and quantitative approaches reflect fundamentally different paradigms, such as when people refer to qualitative vs. quantitative methods. Although there are obvious differences in the two approaches, there are many cases when the two are combined. (p. 778)

As we will discuss in the last section, one of the main shortcomings of studies that involve an attempt to combine the two approaches is the failure to successfully integrate qualitative and quantitative data. This is consistent with Bazeley's (2010) conclusion that “there are surprisingly few published studies reporting results from projects which make more than very elementary use of the capacity to integrate data and analyses using computers” (p. 434). True integration in applied studies is not easy task, but the aim of this paper is to show how a novel methodological approach grounded within systematic observation can help to overcome some of the challenges involved.

Based on our experience and work, we can now confidently state that the “multifaceted” perspective (Tashakkori and Teddlie, 2010, p. 274) offered by a mixed methods research approach (Johnson et al., 2007) is now widely present in the field of sport and physical activity (van der Roest et al., 2015). An optimal approach would be to take a wide-angle perspective while resisting the temptation to pose an overly broad research question, with the ultimate aim of making future research more effective.

To gain a better perspective on the use of mixed methods research in sports and physical activity studies worldwide, we conducted what Alise and Teddlie (2010) refer to as prevalence rate studies, which represents “a line of inquiry into research methods in the social/behavioral sciences [referring to the proportion of articles using a particular methodological approach]” (p. 104), which is undertaken by assessing (a) the prevalence rates of MM [mixed methods] in those fields and (b) the degree to which disciplines are still dominated by the traditional postpositivist QUAN [quantitative] approaches” (p. 107). Specifically, we performed a literature search of ISI-indexed journals in the Web of Science and the ISI Web of Knowledge (Journal Citation Reports) to determine the number of articles applying a mixed methods research approach in this field. We placed no restrictions on language, year, or geographic location.

Table 1 presents a list of the journals analyzed, together with their JCR impact factor and the number of articles that used mixed methods research approaches. They key search term used was mixed methods and we did not place any limits on publication dates, although our results show that the majority of articles retrieved were published after the year 2000. Our findings show that, compared with the situation described by Biddle (1997) and Morris (1999), a considerable number of ISI-indexed journals now publish mixed methods research studies. We have included all studies that, based on their keywords, can be considered mixed methods studies from the time the mixed methods movement emerged. The results from the last 15 years highlight the growing number of mixed methods studies published in the field of sport and physical activity. These studies include a considerable number of conceptual and methodological papers on different aspects of mixed methods, which have undoubtedly contributed to the growth of applied empirical studies in this area. Indeed, the 203 mixed methods research articles identified among this set of 67 journals yielded a mean of 3.03 mixed methods research articles (SD = 4.98). This represents an important advance, not only because of the increase in studies of this type, but also because it shows that prestigious peer-reviewed journals are now publishing these studies.

Inclusion of purely observational sports and physical activity studies in the field of mixed methods research

Studies in the field of sport and physical activity frequently address immediate research concerns that require a scientific answer to questions related to multiple aspects of learning, training, and performance. Such realities are multifaceted in any field, but we are referring to the specific—and possibly unique—case of studies in which the primary and often the only goal is to capture what is actually happening, with no regard for the administration of standardized tests or the opinions or feelings of the agents involved. Studies in the field of sport and physical activity provide numerous examples of such cases, which, due to their singularity, we believe deserve special consideration (Castañer et al., 2013).

Let us imagine, for example, that we are interested in studying the suitability of a certain tactic in an elite individual or team competition (e.g., a judo or soccer match). A fitting research design would be systematically to observe the athlete's behavior (systematic direct observation) and to conduct an in-depth interview with the athlete and/or his or her trainer after the event (indirect observation). Logically, the responses given by the athlete or trainer might be different to the information portrayed by the video recording (referred by Greene et al., 1989; as initiation, which involves discovering paradoxes and contradictions that emerge when findings from the two analytical strands are compared), because opinions regarding performance can understandably vary and can be elaborated on in an interview situation. To meet the goal of our study, we would need to merge the quantitative and qualitative findings by comparing the results of the interview (presuming that these are purely qualitative) with the information captured in the video recordings (as annotation of the behaviors observed in the successive images analyzed produces a systematized, quantifiable dataset built through the coding of data guided by a structured ad-hoc observation instrument).

Although interviews as a research method can sometimes raise concerns due, for example, to doubts about sample representativeness (Sandelowski, 1995; Onwuegbuzie, 2003), this is not the case in the example described. The issue of interviews in observational methodology studies of sport and physical activity is very different, and poses more serious questions, as illustrated by the following example.

Let us now imagine that we are studying the fouls committed by an athlete in a competition. If we did not modify our approach, we would be contrasting a visual record of what actually happened with the athlete's interpretation of what happened, with the additional risk that this interpretation could be tainted by considerable cognitive baggage. If the purpose of the study is to analyze the fouls committed by an athlete, what use is it for the athlete to say that he or she did not commit the foul if we have an image showing the contrary? The discrepancies between the two realities could be considerable, both in volume and nature, but that aside, we do not actually need the opinion of the athlete, because the answer to our research question lies in the analysis of fragments of what actually happened. This issue becomes even more complicated if we decide to include quantitative data, such as distances covered, number of steps taken, or heart rate, or if we administer a personality test before and after the competition, because none of this information can shed light on our research question or enrich our findings.

In our opinion, the ideal solution for situations like this (which are very common) is to apply the successive steps defined within observational methodology. These include selecting dimensions and subdimensions designed to answer the research question, taking decisions on segmentation of the observable date into units, proposing a design for each research objective, building a purpose-designed observation instrument, creating a computerized coded dataset that allows the data to be arranged into matrices of codes, checking the reliability and variability of the data collected, and analyzing the behavioral patterns hidden within the code matrices using robust analytical techniques for categorical data. Systematic observation is the main procedure used to collect data in event analysis (Happ et al., 2004) and there is ample experience with its use and evidence of its potential (Anguera, 1979, 2003; Portell et al., 2015b).

The study of spontaneous behavior is characterized by a richness of information that can only be captured by video or sound recordings, without elicitation (Anguera and Hernández-Mendo, 2016), and the possibilities offered in this area have been greatly enhanced by recent technological advances. Examples are (a) integration of data through merging, connecting, and embedding strategies (Plano Clark and Sanders, 2015); (b) integration of multisensor data through data fusion (Liggins et al., 2017), which consists of combining signal- and image-processing techniques with pattern-recognition techniques and artificial intelligence to create multimodal databases; (c) integration of heart rate data captured during exercise with observational data on physical activity through hidden Markov chains (Castañer et al., 2017b); and (d) application of deep learning techniques, which automatically extract multilevel characteristics that maximize the identification of predefined behavioral patterns (Ordóñez and Roggen, 2016). The resulting information is also richer in terms of veracity, as the data are not tainted by a personal opinion but based on an objective recording of what happened.

A careful choice of observation units is a central component of observational research (Anguera and Izquierdo, 2006). The choice of units in the field of sport will be determined by the research question and by the rules of the sport, each with its nuances, and the units must be captured through the careful, rigorous use of video cameras, which is not without its technical complexities. In soccer, for example, a move may be a macro-unit (with the condition that only the team in possession of the ball is observed) but it can also be divided into smaller units depending on, for instance, how a given player establishes contact with the ball or with different team mates or areas of the pitch.

Systematic observation differs from other methods in that the observation instrument must be built ad-hoc—that is, it must be purpose-designed in accordance with the theoretical framework of the study. The main instrument used in studies of this type combines a field format system and category systems tailored to the research question (Anguera et al., 2007).

The field format (Sánchez-Algarra and Anguera, 2013) is a multidimensional system. For each field format, it is necessary to draw up a catalog of behaviors (a list of mutually exclusive behaviors for each dimension) that is considered to be permanently open; it is constructed using a decimal coding system that allows the behaviors to be hierarchically arranged according to the degree of molecularization required. The final dataset acquires the form of a matrix of codes consisting of columns containing the different dimensions/subdimensions and rows consisting of the successive units into which the episode observed has been segmented. The category system (Anguera, 2003) is unidimensional and requires a theoretical framework, which, combined with empirical information on the situation being observed, enables the construction of a series of exhaustive, mutually exclusive categories. Instruments that combine field format and category systems aim to harness the strengths of the two systems (flexibility in the first case and support from a theoretical framework in the second) and compensate for their weaknesses (inadequacy of the category system in dynamic processes and multidimensional studies and weakness of the field format system in studies that lack a theoretical framework or in which this framework has been rejected).

Numerous examples have been described in the literature, particularly in recent years, and have been applied to a wide range of sporting contexts, including motor skill analysis (Castañer et al., 2009), physical activity (Castañer et al., 2016b), middle- and long-distance races (Aragón et al., 2015, 2017), basketball (Fernández et al., 2009), soccer (Jonsson et al., 2006; Castañer et al., 2016a, 2017a; Casal et al., 2017; Diana et al., 2017), judo (Gutiérrez-Santiago et al., 2011), hockey (Hernández-Mendo and Anguera, 2002), futsal (Lapresa et al., 2013b), and kinesics (Castañer et al., 2013). Ad-hoc instruments have been shown to be equally effective in amateur (Arana et al., 2013) and elite (Barreira et al., 2014) sport. The growing use of combined field-format/category system instruments has undoubtedly has been favored by the increase in observational studies in the field of sport and physical activity. We believe, however, that it is also attributable to the fact that observational methodology is widely applicable and offers an optimal balance between rigor and flexibility.

The number of software programs specifically designed for observational studies has increased in recent years. Apart from general-purpose programs, such as Microsoft Excel and Access, researchers now have access to numerous open-access programs that can be used to record, to display, and to analyze data, as well as to perform quality checks. Our research group has designed several freely accessible software programs to support the scientific community (Hernández-Mendo et al., 2014). Examples are LINCE (Gabin et al., 2012; http://observesport.com), HOISAN (Hernández-Mendo et al., 2012; http://www.menpas.com), MOTS (Castellano et al., 2008; http://www.menpas.com), and SOCCEREYE (Barreira et al., 2013). Another very useful freeware program that our group has been systematically using for years to record observational data and to perform lag sequential analysis is SDIS-GSEG (Bakeman and Quera, 2011).

The concepts and technicalities of quantification (also known as quantitizing; Tashakkori and Teddlie, 1998) and data transformation are a recurrent theme in works written by eminent figures in the field of mixed methods research (Sandelowski, 2001; Creswell et al., 2003; Bazeley, 2009b; Sandelowski et al., 2009). Quantification in observational methodology is particularly robust, because apart from simple frequency counts, it contemplates other essential primary parameters, such as order and duration (Bakeman, 1978; Anguera et al., 2001; Bakeman and Quera, 2011), thereby providing the researcher with the means to map the different components of a behavior as it occurs. In observational methodology, the term progressive order of inclusion refers to the fact that frequency is the parameter that provides the least information; order provides information on frequency and something else (i.e., sequence of behaviors); and duration provides information on frequency and order (by adding the number of time units for each occurrence of a behavior). This specific consideration of the order parameter is crucial for detecting hidden structures through the quantitative analysis of relationships between different codes in systematized observational datasets.

Precisely because it contains information on order and duration, the initial data set, which is derived from an extremely rich qualitative component, can be analyzed using a wide range of quantitative techniques, producing a set of quantitative results that are then interpreted qualitatively, permitting seamless integration. With observational methodology, we are no longer talking about complementing qualitative and quantitative findings, but rather about integrating them. As stated by Fetters (2016), in his article drawing comparisons between developments in mixed methods research and the transition from the horseless carriage to the modern automobile, innovation is both needed and will occur.

The wide scope of opportunities available for processing data derived from observation supports the idea that purely observational studies should be considered as mixed methods research studies, even though they constitute a somewhat special case and do not follow traditional patterns. Although this is a somewhat controversial topic, as Freshwater (2015, p. 296) stated, “disagreement and debate is fundamental to achieving excellence in scholarship.”

The way forward: overcoming the shortcomings of integration and symmetry

We are at a critical time for the future of mixed methods research and we believe that the time has come to stop and to take stock, just as we do in our everyday lives, as we come up against different obstacles and challenges. Considering the relatively recent surge in mixed methods research articles around the globe, we believe it is methodologically “healthy” to, as they say in Spain, “put our finger in the blister” and make a humble but firm call for reflection on what we believe to be the two major barriers to the successful implementation of mixed methods research designs: the lack of integration and the lack of symmetry.

The barrier of integration

Integration of qualitative and quantitative research approaches is a central theme in the mixed methods research literature, and the title of a recent editorial by Fetters and Freshwater (2015b)−1 + 1 = 3—graphically showed that a whole is greater than is the sum of the individual parts. Although it is understandable that researchers from a given discipline typically will follow the traditions of their research communities, it is necessary to bear in mind that the respective findings will be mutually informative—that is, they will talk to each other (O'Cathain et al., 2010).

Quantitative methods address questions such as causality, generalizability, and magnitude of effects, whereas qualitative methodologies are used to develop theories, to describe occurrences, and to explore the contexts surrounding different phenomena (Fetters and Freshwater, 2015b). Qualitative data also can be used to design quantitative instruments (Onwuegbuzie et al., 2010). Just like in an orchestra, each of the components in a mixed methods research design has an important role, but the sum of these components form a greater whole. However, as Bazeley (2009b) pointed out in an interesting study that described how different qualitative and quantitative methodologies could be positioned along a continuum, not all types of data or analysis can be integrated.

Although numerous leading figures in the field of mixed methods research have stressed the importance of integrating qualitative and quantitative data (Creswell, 2003, 2015; O'Cathain et al., 2010), a large number of researchers, not surprisingly, still struggle to merge the two approaches and end up publishing their results separately.

We believe that the failure to successfully integrate qualitative and quantitative data is largely due to the nature of the data involved (Bazeley, 2009b) and that this is where we need to focus our efforts, through reflection, inquiry, and exploration of solutions. This lack of data integration might also stem from quantitative and qualitative research questions that are addressed separately within a mixed methods research study (cf. Plano Clark and Badiee, 2010). Qualitative and quantitative data, however, can be integrated using what is known as the weaving approach, which involves presenting the respective findings together according to a specific theme or concept. Consequently, we propose that researchers who encounter difficulties merging qualitative and quantitative data in studies of sport and physical activity contemplate an initial exploratory phase in which they search for ways of weaving together their data, at least until a suitable methodological solution is found.

The barrier of symmetry

Unlike other approaches in experimental studies, which from an enactive framework (Lutz et al., 2002) show the difference between first-person approaches (based on phenomonological data) and third-person approaches (based on physiological and behavioral data obtained objectively using a range of instruments), enabling thus the problem of asymmetry to be overcome, in observational studies of spontaneous behavior in natural settings, where nothing is “artificial” or “staged,” asymmetry acquires a different meaning, as described below.

Mixed methods research studies typically involve the adoption of either a qualitative-dominant or a quantitative-dominant approach (Onwuegbuzie and Combs, 2010), but an additional issue is that studies are frequently characterized by a lack of symmetry between the two approaches. Mixed methods studies typically focus more on qualitative than quantitative data and accordingly miss the opportunity to explore the wealth of information that a quantitative analysis of qualitative data can provide. Although it is true that some researchers apply robust statistical methods and even multiple techniques to analyze quantitative data (Onwuegbuzie et al., 2007), they frequently fail to move beyond a descriptive analysis (Ross and Onwuegbuzie, 2014), and consequently miss out on the opportunity to explore the richness of information within the qualitative component (Bazeley, 2009a; Onwuegbuzie, 2016).

As a step toward achieving this qualitative-quantitative symmetry, we agree with Happ et al. (2004) that it is necessary to quantitize the qualitative data and qualitize the quantitative data using different event analysis techniques, such as, for example, segmenting episodes of behavior into events or measuring duration of behaviors. O'Cathain et al. (2010) also refer to quantitization and qualitization, but argue that it is not sufficient simply to use the qualitative data to inform the quantitative findings, stressing instead the need to mix together the two types of data to create new variables.

Onwuegbuzie et al. (2011) identified 58 types of quantitative analyses, which they grouped into four categories according to level of complexity: number of independent variables, number of dependent variables, measurement scales for independent variables (nominal, ordinal, interval, and ratio scales), and measurements scale for dependent variables. Obviously, each of these categories can be further broken down into additional categories (Ross and Onwuegbuzie, 2014). What we are proposing in this paper, and with specific reference to research in the field of sport and physical activity, is an approach that strengthens the analytical processing of quantitative data derived from the qualitative component of the study. The concept of independent and dependent variables, almost omnipresent in experimental and quasi-experimental studies, is not relevant to systematic observation, because this involves observing spontaneous behaviors in natural settings.

Techniques for analyzing quantitative data obtained from qualitative research are generally complex. Anguera et al. (2014) presented a list of the techniques used in sport and physical activity research. In Table 2, we present an updated version of this list, which also now includes techniques for analyzing data from quantitative sources (Sanchez-Algarra, 2006).

Table 2 shows the wide range of possibilities that exist for analyzing the qualitative and quantitative data that coexist in observational studies in the field of sport and physical activity. As the table demonstrates, however, what is novel about our approach from a mixed methods perspective is the way in which we integrate or mix the two types of data. Generally speaking, “there are three ways in which mixing occurs: merging or converging the two datasets by actually bringing them together, connecting the two datasets by having one build on the other, or embedding one data set within the other so that one type of data provides a supportive role for the other data set” (Creswell and Plano Clark, 2007, p. 7). For our proposal, we chose the second form: connecting two databases by having one build on the other. According to Sandelowski et al. (2009), this connection can be achieved through transformation, i.e., by quantitizing qualitative data or by qualitizing quantitative data. We use the sequentiality method, shown in the last row in Table 2, which takes as its starting point the annotation of the order of occurrence of all the behaviors included in a given observational dataset. This sequentiality permits the transformation of initially qualitative data into a format that can be analyzed quantitatively and robustly, achieving thus successful integration.

None of the standard research designs conceptualized for mixed methods research are applicable to the transformation of qualitative data (derived from video or sound recordings in natural settings, or from texts resulting from indirect observation) into quantitative data for analysis using specific quantitative techniques, such as variability analysis, comparison of proportions, categorical variance, log-linear analysis, logit analysis, lag sequential analysis, polar coordinate analysis, T-pattern detection, and so forth. Perhaps the use of such techniques will enable the weaving approach called for in mixed methods research. Several data analysis techniques that are specific to the study of sequences of behavior, such as lag sequential analysis, polar coordinate analysis, and T-pattern detection, have a particularly important role in observational methodology due to the assignment of parameters of frequency, order, and duration to the initial qualitative data (Anguera et al., 2001; Blanco-Villaseñor et al., 2003) and thereby providing the necessary conditions for subsequent quantitative analysis using robust, non-standard, statistical techniques that offer highly relevant structural results.

As an epilog, we would like to stress that there is wide consensus in the mixed methods research field on the value of using merging, connecting, and embedding strategies to integrate qualitative and quantitative data (Plano Clark and Sanders, 2015). In this article, we have focused on an approach for connecting these two perspectives and shown that it is perfectly possible to transform qualitative datasets featuring behaviors whose order of occurrence has been recorded into matrices of code that can subsequently be analyzed using powerful quantitative techniques.

Discussion

The scientific literature of the past 25 years has presented, from varying and sometimes opposing standpoints, a wide spectrum of theoretical positions and empirical findings in relation to studies that are claimed to represent mixed methods research studies (López-Fernández and Molina-Azorín, 2011). Mixed methods research is growing in most disciplines—as demonstrated by Onwuegbuzie and Corrigan (2016) via their recent meta-prevalence rate study. And, in our opinion, the field of sport and physical activity is a particularly fertile area in which studies merging quantitative and qualitative approaches have begun to flourish. These studies typically involve the analysis of behaviors and motor-related skills in a wide range of sports and activities that are grounded in a theoretical framework, can be performed at a professional or amateur level, offer big learning/training opportunities due to their vast scope, and have the potential for causing considerable impact in the scientific community and media at large. They are, as such, particularly deserving of attention. As shown by our review of the literature, summarized in Table 1, the volume of mixed methods research publications in ISI-indexed sports and physical activity journals varies widely from one journal to the next. Perhaps, as Fetters (2016) suggests via his analogy of the horseless carriage, we were unaware that the many decades of tinkering with mixed methods would spawn a period of accelerated development. We should also, however, bear in mind the saying “do not put vintage wine into new wineskins lest it sour.” That stated, although these “wineskins” are new, they will have benefited from the experience gradually accumulated in multiple substantive areas over the past two decades.

As the guiding principle of this paper was to show the specificities of observational studies in the field of sport and physical activity, it is only logical that we also critically appraise alternative options described in the scientific literature. With reference to studies based on first-person and third-person descriptions, it is important to note that the former refer to “lived experiences” linked to cognitive and mental events, while the latter refer to descriptive experiences linked to the study of other natural phenomena. Both, however, can be connected using a phenomenological approach. In an experimental setting designed to analyze processes, such as attention or memory, for example, subjects are asked to perform a specific task but while doing this, they experience what can be termed “lived content.” As this is something that can be described and analyzed, it exists. What sets our proposal apart is that we always analyze spontaneous behavior in natural settings. The subject is given no instructions, as the setting is natural, not artificial. Consequently, despite the vitality of lived experiences and the issues regarding the what, why, and how of first-person methodologies (Varela and Shear, 1999a,b), the start and end points of first- and third-person methodologies are different, although it is perfectly possible to integrate qualitative and quantitative data in both approaches if there is sufficient symmetry.

One interesting point for reflection was recently proposed by Depraz et al. (2017), who addressed the challenges associated with generating productive interaction between first-person data (in this case micro-phenomenological interviews) and third-person data (objective physiological data) by co-interpreting both analyses (each from the other) using an approach based on “co-validation” and “mutual constraints” (Varela, 1996). Their proposal was discussed in depth in subsequent open peer commentaries (also available in Varela, 1996). In our field, reflection on the issues they discuss could give rise to a line of research focused on the perceivable expression of spontaneous behaviors, which would influence the categorization stage of our method.

We hope that the continued development of research within systematic observation will trigger reflection and inquiry and contribute to the creation of knowledge in the field of sport and physical activity. Systematic observation has commendable strengths, but it also requires a certain sacrifice in terms of time, labor, and pursuit of scientific rigor. The strength of systematic observation is that it provides a flexible yet rigorous framework for the objective analysis of behaviors in a natural setting and context; this is the only way to study spontaneous behaviors in a natural environment. In this article, we have discussed the essential role of observational studies in this respect and highlighted the benefits of transforming rigorously annotated, sequential (ordered) qualitative data into code matrices for subsequent analysis using powerful quantitative analytical techniques. Systematic observation in its current form permits the seamless integration of qualitative and quantitative data (Sánchez-Algarra and Anguera, 2013), while meeting the requirements of multi-method research studies and harnessing the potential offered by the quantitization of qualitative data using non-standard statistical techniques and the subsequent interpretive qualitization of the quantitative results (which are largely structural) using behavior sequence analysis techniques.

Finally, we would like to express our concern that the fundamental meaning of mixed methods research, at times, has been misunderstood, leading to the publication of pseudo-studies in the form of qualitative research supplemented by some quantitative results or quantitative research adorned with a qualitative component, such as an interview or autobiographical text. We believe that the unorthodox use and understanding of the term mixed methods has generated confusion that has not been adequately addressed. The literature contains many examples of studies that the authors claimed to represent mixed methods that are not, but it also contains studies that are mixed methods research studies but whose findings are limited by a lack of integration and/or symmetry. The difficulty of integrating qualitative and quantitative research approaches has been widely acknowledged (Fetters and Freshwater, 2015b) and is largely linked to a lack of a strong scientific culture. Achieving the symmetry between the two approaches will be a difficult process because they originate from very different traditions. We are aware of the difficulties that lie ahead in our field but we believe that we need to move forward and to assume the responsibility of providing methodological training focused precisely on these two weak points: integration and symmetry.

To conclude, we would like to acknowledge the initiative shown by Fetters and Freshwater (2015a) in calling for contributions that stimulate open reflection and dialog among researchers from multiple disciplines interested in helping to define and to conceptualize mixed methods research. We, for one, are very interested in engaging in this dialog.

References

• 1

  AliseM. A.TeddlieC. (2010). A continuation of the paradigm wars? Prevalence rates of methodological approaches across the social/behavioral sciences. J. Mix. Meth. Res. 4, 103–126. 10.1177/1558689809360805

  • CrossRef
  • Google Scholar

• 2

  AngueraM. T. (1979). Observational typology. Qual. Quan. 13, 449–484.

  • Google Scholar

• 3

  AngueraM. T. (2003). Observational methods (general), in Encyclopedia of Psychological Assessment, Vol. 2, ed Fernández-BallesterosR. (London: Sage), 632–637.

  • Google Scholar

• 4

  AngueraM. T.Blanco-VillaseñorA.LosadaJ. L. (2001). Diseños Observacionales, cuestión clave en el proceso de la metodología observacional. (Observational designs, a key question in the observational methodology process). Metodol. C. Comport. 3, 135–161.

  • Google Scholar

• 5

  AngueraM. T.CamerinoO.CastañerM.Sánchez-AlgarraP. (2014). Mixed methods en actividad física y deporte. Rev. Psicol. Dep. 23, 123–130.

  • Google Scholar

• 6

  AngueraM. T.Hernández-MendoA. (2016). Avances en estudios observacionales de Ciencias del Deporte desde los mixed methods. (Advances in observational studies in sports sciences from a mixed methods approach). Cuad. Psicol. Dep. 16, 17–30.

  • Google Scholar

• 7

  AngueraM. T.IzquierdoC. (2006). Methodological approaches in human communication. From complexity of situation to data analysis, in From Communication to Presence: Cognition, Emotions and Culture Towards the Ultimate Communicative Experience, eds RivaG.AngueraM. T.WiederholdB. K.MantovaniF. (Amsterdam: IOS Press), 203–222.

  • Google Scholar

• 8

  AngueraM. T.JonssonG. K.Sánchez-AlgarraP. (2017). Liquefying text from human communication processes: a methodological proposal based on t-pattern detection. J. Multimodal Comm. Studies4, 10–15.

  • Google Scholar

• 9

  AngueraM. T.MagnussonM. S.JonssonG. K. (2007). Instrumentos no estándar [Nonstandard instruments]. Avances en Medición5, 63–82.

  • Google Scholar

• 10

  AragónS.LapresaD.AranaJ.AngueraM. T.GarzónB. (2017). An example of the informative potential of polar coordinate analysis: sprint tactics in elite 1500 m track events. Measur. Phys. Educ. Exerc. Sci. 16, 279–286. 10.1080/1091367X.2016.1245192

  • CrossRef
  • Google Scholar

• 11

  AragónS.LapresaD.AranaX.AngueraM. T.GarzónB. (2015). Tactical behaviour of winning athletes in major championship 1500-m and 5000-m track finals. Eur. J. Sport Sci. 16, 279–286. 10.1080/17461391.2015.1009494

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 12

  AranaJ.LapresaD.AngueraM. T.GarzónB. (2013). Adapting football to the child: an application of the logistic regression model in observational methodology. Qual. Quant. 47, 3473–3480. 10.1007/s11135-012-9734-z

  • CrossRef
  • Google Scholar

• 13

  AriasE.AngueraM. T. (2017). Observation of interactions in adolescent group therapy: a mixed methods study. Front. Psychol. 8:1188. 10.3389/fpsyg.2017.01188

  • CrossRef
  • Google Scholar

• 14

  BakemanR. (1978). Untangling streams of behavior: sequential analysis of observation data, in Observing Behavior, Vol. 2: Data Collection and Analysis Methods, ed SackettG. P. (Baltimore, MA: University of Park Press), 63–78.

  • Google Scholar

• 15

  BakemanR.GottmanJ. M. (1997). Observing Interaction. Introduction to Sequential Analysis, 2nd Edn.Cambridge: Cambridge University Press.

  • Google Scholar

• 16

  BakemanR.QueraV. (2011). Sequential Analysis and Observational Methods for the Behavioral Sciences. Cambridge: Cambridge University Press.

  • Google Scholar

• 17

  BarreiraD.GargantaJ.CastellanoJ.AngueraM. T. (2013). SoccerEye: a software solution to observe and record behaviours in sport settings. Open Sports Sci. J. 6, 47–55. 10.2174/1875399X01306010047

  • CrossRef
  • Google Scholar

• 18

  BarreiraD.GargantaJ.GuimarãesP.MachadoJ.AngueraM. T. (2014). Ball recovery patterns as a performance indicator in elite soccer. J. Sports Engin. Technol. 228, 61–72. 10.1177/1754337113493083

  • CrossRef
  • Google Scholar

• 19

  BazeleyP. (2009a). Analysing qualitative data: more than “identifying themes”. Malasyan J. Qual. Res. 2, 6–22.

  • Google Scholar

• 20

  BazeleyP. (2009b). Editorial: integrating data analyses in mixed methods research. J. Mix. Meth. Res. 3, 203–207. 10.1177/1558689809334443

  • CrossRef
  • Google Scholar

• 21

  BazeleyP. (2010). Computer assisted integration of mixed methods data sources and analyses, in Handbook of Mixed Methods in Social and Behavioral Research, 2nd Edn., eds TashakkoriA.TeddlieC. (Thousand Oaks, CA: Sage), 431–467.

  • Google Scholar

• 22

  BiddleS. J. H. (1997). Current trends in sport and exercise psychology research. Psychol. Bull. Brit. Psych. Soc. 10, 63–69.

  • Google Scholar

• 23

  BiddleS. J. H.MarklandD.GilbourneD.ChatzisarantisN. L. D.SparkesA. C. (2001). Research methods in sport and exercise psychology: quantitative and qualitative issues. J. Sport Sci. 19, 777–809. 10.1080/026404101317015438

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 24

  Blanco-VillaseñorA.LosadaJ. L.AngueraM. T. (2003). Analytic techniques in observational designs in environment-behavior relation. Medio Ambiente y Comport. Hum. 4, 111–126.

  • Google Scholar

• 25

  CamerinoO.CastañerM.AngueraM. T. (eds.). (2012a). Mixed Methods Research in the Movement Sciences: Case Studies in Sport, Physical Education and Dance. Abingdon: Routledge.

  • Google Scholar

• 26

  CamerinoO.ChaverriJ.AngueraM. T.JonssonG. K. (2012e). Dynamics of the game in soccer: detection of T-patterns. Eur. J. Sport Sci.12, 216–224. 10.1080/17461391.2011.566362

  • CrossRef
  • Google Scholar

• 27

  CamerinoO.ChaverriJ.JonssonG.Sánchez-AlgarraP.AngueraM. T. (2012b). Influence of the use of space on the dynamics of play in basketball, in Mixed Methods Research in the Movement Sciences: Cases in Sport, Physical Education and Dance, eds CamerinoO.CastañerM.AngueraM. T. (Abingdon: Routledge), 41–57.

  • Google Scholar

• 28

  CamerinoO.IglesiasX.Gutiérrez-SantiagoA.PrietoI.CampaniçoJ.AngueraM. T. (2012c). Optimizing techniques and dynamics in individual sports, in Mixed Methods Research in the Movement Sciences: Case Studies in Sport, Physical Education and Dance, eds CamerinoO.CastañerM.AngueraM. T. (Abingdon: Routledge), 82–116.

  • Google Scholar

• 29

  CamerinoO.JonssonG.Sánchez-AlgarraP.AngueraM. T.LopesA.ChaverriJ. (2012d). Detecting hidden patterns in the dynamics of play in team sports, in Mixed Methods Research in the Movement Sciences: Case Studies in Sport, Physical Education and Dance, eds CamerinoO.CastañerM.AngueraM. T. (Abingdon: Routledge), 31–81.

  • Google Scholar

• 30

  CasalC. A.ManeiroR.ArdáT.LosadaJ. L.RialA. (2015). Analysis of corner kick success in elite football. Int. J. Perform. Anal. Sport15, 430–451. 10.1080/24748668.2015.11868805

  • CrossRef
  • Google Scholar

• 31

  CasalC. A.ManeiroR.ArdáT.MaríF. J.LosadaJ. L. (2017). Possession zone as a performance indicator in football. The Game of the Best Teams. Front. Psychol. 8:1176. 10.3389/fpsyg.2017.01176

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 32

  CastañerM.AnduezaJ.Sánchez-AlgarraP.AngueraM. T. (2012). The laterality of motor skills: a complex merging of postural support and gestural precision, in Mixed Methods Research in the Movement Sciences: Case Studies in Sport, Physical Education and Dance, eds CamerinoO.CastañerM.AngueraM. T. (Abingdon: Routledge), 119–145.

  • Google Scholar

• 33

  CastañerM.BarreiraD.CamerinoO.AngueraM. T.CantonA.HilenoR. (2016a). Goal scoring in soccer: a polar coordinate analysis of motor skills used by Lionel Messi. Front. Psychol. 7:806. 10.3389/fpsyg.2016.00806

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 34

  CastañerM.BarreiraD.CamerinoO.AngueraM. T.FernandesT.HilenoR. (2017a). Mastery in goal scoring, T-pattern detection and polar coordinate analysis of motor skills used by Lionel Messi and Cristiano Ronaldo. Front. Psychol. 8:741. 10.3389/fpsyg.2017.00741

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 35

  CastañerM.CamerinoO.AngueraM. T.JonssonG. K. (2013). Kinesics and proxemics communication of expert and novice PE teachers. Qual. Quant. 47, 1813–1829. 10.1007/s11135-011-9628-5

  • CrossRef
  • Google Scholar

• 36

  CastañerM.CamerinoO.LandryP.ParésN. (2016b). Quality of physical activity of children in exergames: sequential body movement analysis and its implications for interaction design. Int. J. Hum. Comput. Stud. 96, 67–78. 10.1016/j.ijhcs.2016.07.007

  • CrossRef
  • Google Scholar

• 37

  CastañerM.PuigarnauS.BenítezR.ZurloniV.CamerinoO. (2017b). How to merge observational and physiological data? A case study of motor skills patterns and heart rate in exercise programs for adult women. Anal. Psic.33, 442–449. 10.6018/analesps.33.3.271011

  • CrossRef
  • Google Scholar

• 38

  CastañerM.TorrentsC.AngueraM. T.DinušováM.JonssonG. (2009). Identifying and analyzing motor skill responses in body movement and dance. Behav. Res. Meth. 41, 857–867. 10.3758/BRM.41.3.857

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 39

  CastellanoJ.PereaA.AldayL.Hernández-MendoA. (2008). The measuring and observation tool in sports. Behav. Res. Meth. 40, 898–905. 10.3758/BRM.40.3.898

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 40

  CreswellJ. W. (2003). Research Design: Qualitative, Quantitative, and Mixed Methods Approaches, 2nd Edn. Thousand Oaks, CA: Sage.

  • Google Scholar

• 41

  CreswellJ. W. (2015). A Concise Introduction to Mixed Methods Research. Thousand Oaks, CA: Sage.

  • Google Scholar

• 42

  CreswellJ. W.Plano ClarkV. L. (2007). Designing and Conducting Mixed Methods Research, 3rd Edn. Thousand Oaks, CA: Sage.

  • Google Scholar

• 43

  CreswellJ. W.Plano ClarkV. L.GutmannM. L.HansonW. E. (2003). Advanced mixed methods research designs, in Handbook of Mixed Methods in Social and Behavioral Research, eds TashakkoriA.TeddlieC. (Thousand Oaks, CA: Sage), 209–240.

  • Google Scholar

• 44

  DeprazN.GyemantM.DesmidtT. (2017). A first-person analysis using third-person data as a generative method. Construc. Found.12, 190–203.

  • Google Scholar

• 45

  DianaB.ZurloniV.EliaM.CavaleraC. M.JonssonG. K.AngueraM. T. (2017). How game location affects soccer performance: T-pattern analysis of attack actions. Front. Psychol. 8:1415. 10.3389/fpsyg.2017.01415

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 46

  FernándezJ.CamerinoO.AngueraM. T.JonssonG. (2009). Identifying and analyzing the construction and effectiveness of offensive plays in basketball by using systematic observation. Behav. Res. Methods41, 719–730. 10.3758/BRM.41.3.719

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 47

  FettersM. D. (2016). “Haven't we always been doing mixed methods research?” Lessons learned from the development of the horseless carriage. J. Mix. Meth. Res.10, 3–11. 10.1177/1558689815620883

  • CrossRef
  • Google Scholar

• 48

  FettersM. D.FreshwaterD. (2015a). Publishing a methodological mixed methods research article. J. Mix. Meth. Res. 9, 203–213. 10.1177/1558689815594687

  • CrossRef
  • Google Scholar

• 49

  FettersM. D.FreshwaterD. (2015b). The 1+1 = 3 integration challenge. J. Mix. Meth. Res. 9, 115–117. 10.1177/1558689815581222

  • CrossRef
  • Google Scholar

• 50

  FreitasO.OliveiraC.LeitaoC.AngueraM. T.CampaniçoJ. (2010). Detection of counter-attack and fast attack interaction patterns in handball. Int. J. Sport Psychol.41, 71–72.

  • Google Scholar

• 51

  FreshwaterD. (2015). Mixed methods research: in pursuit of excellence. J. Mix. Meth. Res. 9, 295–296. 10.1177/1558689815606008

  • CrossRef
  • Google Scholar

• 52

  GabinB.CamerinoO.AngueraM. T.CastañerM. (2012). Lince: multiplatform sport analysis software. Procedia Soc. Behav. Sci. 46, 4692–4694. 10.1016/j.sbspro.2012.06.320

  • CrossRef
  • Google Scholar

• 53

  GreeneJ. C.CaracelliV. J.GrahamW. F. (1989). Toward a conceptual framework for mixed-method evaluation designs. Educ. Eval. Pol. Anal. 11, 255–274. 10.3102/01623737011003255

  • CrossRef
  • Google Scholar

• 54

  Gutierrez-SantiagoA.PrietoI.CamerinoO.AngueraM. T. (2013). Sequences of errors in the judo throw Morote Seoi Nage and their relationship to the learning process. J. Sports Engin. Techn. 227, 57–63. 10.1177/1754337111431916

  • CrossRef
  • Google Scholar

• 55

  Gutiérrez-SantiagoA.PrietoI.CamerinoO.AngueraM. T. (2011). The temporal structure of judo bouts in visually impaired men and women. J. Sport Sci. 29, 1443–1451. 10.1080/02640414.2011.603156

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 56

  HappM. B.SwigartV.TateJ.CrightonM. (2004). Event analysis techniques. Adv. Nurs. Sci. 27, 239–248. 10.1097/00012272-200407000-00008

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 57

  Hernández-MendoA.AngueraM. T. (2002). Behavioral structure in sociomotor sports: roller-Hockey. Qual. Quant. 36, 347–378. 10.1023/A:1020905411739

  • CrossRef
  • Google Scholar

• 58

  Hernández-MendoA.CastellanoJ.CamerinoO.JonssonG. K.Blanco-VillaseñorA.LopesA.et al. (2014). Programas informáticos de registro, control de calidad del dato, y análisis de datos. (Software programs for data recording, quality checks, and analysis). Rev. Psic. Dep. 23, 111–121.

  • Google Scholar

• 59

  Hernández-MendoA.López-LópezJ. A.CastellanoJ.Morales-SánchezV.PastranaJ. L. (2012). Hoisan 1.2: programa informático para uso en metodología observacional. (Hoisan 1.2: a software program for use in observational methodology). Cuad. Psic. Dep.12, 55–78. 10.4321/S1578-84232012000100006

  • CrossRef
  • Google Scholar

• 60

  IglesiasX.AngueraM. T. (2012). The influence of environmental factors in the context of elite fencing, in Mixed Methods Research in the Movement Sciences: Cases in Sport, Physical Education and Dance, eds CamerinoO.CastañerM.AngueraM. T. (Abingdon: Routledge), 83–93.

  • Google Scholar

• 61

  IvankovaN. V.KawamuraY. (2010). Emerging trends in the utilization of integrated designs in the social, behavioral, and health sciences, in Sage Handbook of Mixed Methods in Social and Behavioral Research, 2nd Edn, eds TashakkoriA.TeddlieC. (Thousand Oaks, CA: Sage), 581–611.

  • Google Scholar

• 62

  JohnsonR. B.OnwuegbuzieA. J.TurnerL. A. (2007). Toward a definition of mixed methods research. J. Mix. Meth. Res. 1, 112–133. 10.1177/1558689806298224

  • CrossRef
  • Google Scholar

• 63

  JonssonG. K.AngueraM. T.Blanco-VillaseñorA.LosadaJ. L.Hernández-MendoA.ArdáT.et al. (2006). Hidden patterns of play interaction in soccer using SOF-CODER. Behav. Res. Meth. 38, 372–381. 10.3758/BF03192790

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 64

  LapresaD.AngueraM. T.AlsasuaR.AranaJ.GarzónB. (2013a). Comparative analysis of T-patterns using real time data and simulated data by assignment of conventional durations: the construction of efficacy in children's basketball. Int. J. Perform. Anal. Sport13, 321–339. 10.1080/24748668.2013.11868651

  • CrossRef
  • Google Scholar

• 65

  LapresaD.ÁlvarezL.AranaJ.GarzónB.CaballeroV. (2013b). Observational analysis of the offensive sequences that ended in a shot by the winning team of the 2010 UEFA Futsal Championship. J. Sport Sci. 31, 1731–1739. 10.1080/02640414.2013.803584

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 66

  LigginsM. E.HallM. D.LlinasJ. (eds.). (2017). Handbook of Multisensor Data Fusion: Theory and Practice. Oxford: CRC Press.

  • Google Scholar

• 67

  López-FernándezO.Molina-AzorínJ. (2011). The use of mixed methods research in the field of behavioural sciences. Qual. Quant. 45, 1459–1472. 10.1007/s11135-011-9543-9

  • CrossRef
  • Google Scholar

• 68

  LutzA.LachauxJ.-P.MartinerieJ.VarelaF. J. (2002). Guiding the study of brain dynamics by using first-person data: synchrony patterns correlate with ongoing conscious states during a simple visual task. Proc. Natl. Acad. Sci. U.S.A.99, 1586–1591. 10.1073/pnas.032658199

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 69

  ManeiroR.LosadaJ. L.CasalC.ArdáA. (2017). Multivariate analysis of indirect free kick in the FIFA World Cup 2014. Anal. Psic. 33, 461–470. 10.6018/analesps.33.3.271031

  • CrossRef
  • Google Scholar

• 70

  ManolovR.LosadaJ. L. (2017). Simulation theory applied to direct systematic observation. Front. Psychol. 8:905. 10.3389/fpsyg.2017.00905

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 71

  Morales-SánchezV.Pérez-LópezV.AngueraM. T. (2014). Tratamiento metodológico de la observación indirecta en la gestión de organizaciones deportivas. (Methodological processing of indirect observation in the management of sports organizations). Rev. Psic. Dep. 23, 201–207.

  • Google Scholar

• 72

  MorrisT. (1999). The message of methods: developing research methodology in sport psychology. J. Wuhan Inst. Phys. Educ. 33, 20–27.

  • Google Scholar

• 73

  O'CathainA.MurphyE.NichollJ. (2010). Three techniques for integrating data in mixed methods studies. Br. Med. J.341:c4587. 10.1136/bmj.c4587

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 74

  OnwuegbuzieA. J.SlateJ. R.LeechN. L.CollinsK. M. T. (2007). Conducting mixed analyses: a general typology. Int. J. Mult. Res. App.1, 4–17. 10.5172/mra.455.1.1.4

  • CrossRef
  • Google Scholar

• 75

  OnwuegbuzieA. J. (2003). Effect sizes in qualitative research: a prolegomenon. Qual. Quant.37, 393–409. 10.1023/A:1027379223537

  • CrossRef
  • Google Scholar

• 76

  OnwuegbuzieA. J. (2016). A call for conducting multivariate mixed analyses. J. Educ. Issues2, 1–30. 10.5296/jei.v2i2.9316

  • CrossRef
  • Google Scholar

• 77

  OnwuegbuzieA. J.BustamanteR. M.NelsonJ. A. (2010). Mixed research as a tool for developing quantitative instruments. J. Mix. Meth. Res. 4, 56–78. 10.1177/1558689809355805

  • CrossRef
  • Google Scholar

• 78

  OnwuegbuzieA. J.CombsJ. P. (2010). Emergent data analysis techniques in mixed methods research: a synthesis, in Sage Handbook of Mixed Methods in Social and Behavioral Research, 2nd Edn., eds TashakkoriA.TeddieC. (Thousand Oaks, CA: Sage), 397–430.

  • Google Scholar

• 79

  OnwuegbuzieA. J.CorriganJ. A. (2016). Prevalence of mixed methods research in special education, in Paper Presented at the Annual Meeting of the American Educational Research Association (Washington, DC).

  • Google Scholar

• 80

  OnwuegbuzieA. J.DanielL. G. (2003, February 12). Typology of analytical interpretational errors in quantitative qualitative educational research. Current Issues Education6, 1–34. Available online at: https://cie.asu.edu/ojs/index.php/cieatasu/article/view/1609/651

  • Google Scholar

• 81

  OnwuegbuzieA. J.LeechN. L.CollinsK. M. T. (2011). Toward a new era for conducting mixed analyses: the role of quantitative dominant and qualitative dominant crossover mixed analyses, in The Sage Handbook of Innovation in Social Research Methods, eds WilliamsM.VogtW. P. (Thousand Oaks, CA: Sage), 353–384.

  • Google Scholar

• 82

  OrdóñezF. J.RoggenD. (2016). Deep convolutional and LSTM recurrent neural networks for multimodal wearable activity recognition. Sensors16:115. 10.3390/s16010115

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 83

  Pérez-LópezR.Morales-SánchezV.AngueraM. T.Hernández-MendoA. (2016). Modelo tridimensional de la calidad en organizaciones deportivas en usuarios/as infantiles [Tridimensional model of quality in sport organizations: emotional quality in children users]. Cuad. Psicl Dep. 16, 143–150.

  • Google Scholar

• 84

  Plano ClarkV. L.BadieeM. (2010). Research questions in mixed methods research, in Handbook of Mixed Methods in Social and Behavioral Research, 2nd Edn., eds TashakkoriA.TeddlieC. (Thousand Oaks, CA: Sage), 275–304.

  • Google Scholar

• 85

  Plano ClarkV. L.SandersK. (2015). The use of visual displays in mixed methods research, in Use of Visual Displays in Research and Testing, eds McCruddenM. T.SchrawG.BuckendahlC. W. (Charlotte, NC: Information Age Publishing, Inc.), 177–206.

  • Google Scholar

• 86

  PortellM.AngueraM. T.Chacón-MoscosoS.SanduveteS. (2015a). Guidelines for reporting evaluations based on observational methodology. Psicothema27, 283–289. 10.7334/psicothema2014.276

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 87

  PortellM.AngueraM. T.Hernández-MendoA.JonssonG. K. (2015b). Quantifying biopsychosocial aspects in everyday contexts: an integrative methodological approach from the behavioral sciences. Psych. Res. Behav. Manage. 8, 153–160. 10.2147/PRBM.S82417

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 88

  Rodríguez-ZamoraL.IglesiasX.BarreroA.TorresL.ChaverriD.RodríguezF. A. (2014). Monitoring internal load parameters during competitive synchronized swimming duet routines in elite athletes. J. Strength Cond. Res. 28, 742–751. 10.1519/JSC.0b013e3182a20ee7

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 89

  RossA.OnwuegbuzieA. J. (2014). Complexity of quantitative analyses used in mixed research articles published in a flagship mathematics education journal. Int. J. Mult. Res. Appr. 8, 63–73. 10.5172/mra.2014.8.1.63

  • CrossRef
  • Google Scholar

• 90

  Sanchez-AlgarraP.(Coord.). (2006). Métodos Estadísticos Aplicados (Applied Statistical Methods). Barcelona: Publicacions i Edicions de la Universidad de Barcelona.

  • Google Scholar

• 91

  Sánchez-AlgarraP.AngueraM. T. (2013). Qualitative/quantitative integration in the inductive observational study of interactive behaviour: impact of recording and coding among predominating perspectives. Qual. Quant. 47, 1237–1257. 10.1007/s11135-012-9764-6

  • CrossRef
  • Google Scholar

• 92

  SandelowskiM. (1995). Sample size in qualitative research. Res. Nurs. Health18, 179–183. 10.1002/nur.4770180211

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 93

  SandelowskiM. (2001). Real qualitative researchers do not count: the use of numbers in qualitative research. Res. Nurs. Health24, 230–240. 10.1002/nur.1025

  • CrossRef
  • Google Scholar

• 94

  SandelowskiM.VoilsC. I.KnaflG. (2009). On quantitizing. J. Mix. Meth. Res. 3, 208–222. 10.1177/1558689809334210

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 95

  TarragóR.IglesiasX.LapresaD.AngueraM. T.Ruiz-SanchísL.AranaJ. (2017). Analysis of diachronic relationships in successful and unsuccessful behaviors by world fencing champions using three complementary techniques. Anal. Psicol. 33, 471–485. 10.6018/analesps.33.3.271041

  • CrossRef
  • Google Scholar

• 96

  TashakkoriA.TeddlieC. (1998). Mixed Methodology: Combining Qualitative and Quantitative Approaches. Applied Social Research Methods Series (Vol. 46). Thousand Oaks, CA: Sage.

  • Google Scholar

• 97

  TashakkoriA.TeddlieC. (2010). Putting the human back in “human research methodology”: the researcher in mixed methods research. J. Mix. Meth. Res. 4, 271–277. 10.1177/1558689810382532

  • CrossRef
  • Google Scholar

• 98

  ThompsonB. (1995). Stepwise regression and stepwise discriminant analysis need not apply here: a guidelines editorial. Educ. Psych. Measur. 55, 525–534. 10.1177/0013164495055004001

  • CrossRef
  • Google Scholar

• 99

  ThompsonE.VarelaF. J. (2001). Radical embodiment: neural dynamics and consciousness. Trends Cogn. Sci.5, 418–425. 10.1016/S1364-6613(00)01750-2

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 100

  van der RoestJ.-W.SpaaijR.anf van BottenburgM. (2015). Mixed methods in emerging academic subdisciplines: the case of sport management. J. Mix. Meth. Res. 9, 70–90. 10.1177/1558689813508225

  • CrossRef
  • Google Scholar

• 101

  VarelaF. J. (1996). Neurophenomenology: a methodological remedy for the hard problem. J. Conscious. Stud. 3, 330–349

  • Google Scholar

• 102

  VarelaF. J.ShearJ. (1999a). First-person methodologies: what, why, how?J. Conscious. Stud. 6, 1–14.

  • Google Scholar

• 103

  VarelaF. J.ShearJ. (1999b). The View from Within. First-Person Approaches to the Study of Consciousness.Exeter: Imprint Academic.

  • Google Scholar

• 104

  ZurutuzaU.CastellanoJ.EcheazarraI.CasamichanaD. (2017). Absolute and relative training load and its relation to fatigue in football. Front. Psychol. 8:878. 10.3389/fpsyg.2017.00878

  • Pubmed Abstract
  • CrossRef
  • Google Scholar