Interrogating the Effectiveness of Educational Practices: A Critique of Evidence-Based Psychosocial Treatments for Children Diagnosed with Attention-Deficit/Hyperactivity Disorder
- 1Support for Learning Department, St. Luke’s Hospital, Thessaloniki, Greece
- 2Department of Psychology, University of Roehampton, London, UK
In this paper, we critically evaluate the literature of evidence-based psychosocial interventions for children diagnosed with attention-deficit/hyperactivity disorder (ADHD). Our paper is structured around five main criticisms. First, the evidence-based psychosocial interventions are formulated on the basis of the categorical diagnosis of ADHD, which is an ambiguous diagnostic entity. Second, they claim homogeneity among children diagnosed with ADHD. Third, they diffuse non-pedagogical language in education. The fourth criticism refers to the methodological bias by which evidence is produced. The fifth and final criticism describes how this body of literature promotes a pedagogical practice that relies on manualized approaches to be considered effective.
In this paper, we intend to critically evaluate the approach of evidence-based practice (EBP) as a model which “scientifically” investigates the effectiveness of and thus promote certain types of evidence-based psychosocial treatments (EBPTs) for children diagnosed with attention-deficit/hyperactivity disorder (ADHD). According to this model, research is often conducted through the employment of randomized control trials (RCTs) and the evidence/outcome of these trials is usually presented through publication. The educational experts, teachers, and trainers are then expected to assess and interpret the evidence and incorporate the most effective psychosocial treatments in their practice in an ongoing way. In this respect, this model, which was initially developed to evaluate rigor in medical research in the 1970s, has been currently extended to support advice to educational practitioners about which psychosocial treatments should be used that have well-documented effectiveness with children diagnosed with ADHD and which would, hopefully, replace (or work in combination with) the dominant pharmacological treatments. Our critique focuses on the body of literature investigating psychosocial interventions relating to the educational inclusion of children diagnosed with ADHD. For the purposes of this article, the general term psychosocial treatments will be used to represent the variety of interventions subsumed in this literature (e.g., child behavior management, parent training, classroom management, peer interventions, etc). After a careful review of the literature, we have the following five specific criticisms to offer:
EBPTs are Formulated on the Basis of an Ambiguous Categorical Diagnosis
It is self-evident that EBPTs which target children diagnosed with ADHD are diagnosis driven. Formulated on the basis of the categorical diagnosis, EBPTs implicitly or explicitly convey the certainty that the ADHD diagnosis is a valid, useful, and unproblematic diagnostic entity. What appears to be immediately given is, therefore, uncritically endorsed; ADHD as a diagnostic category and the current educational practices and policies are “taken for granted” and are not properly interrogated.
Attention-deficit/hyperactivity disorder has indeed established a “hegemonic position” in contemporary children mental health and education discourses (Timimi and Timimi, 2015, p.139). The so called “science of ADHD” continues to be one of the most controversial issues, fueling intense debates between academics, practitioners, and parents in such an extent that it has been depicted as the epitomy of skepticism against the current main-stream mental health epistemology (Rafalovich, 2005). Proponents of the dominant medicalized perspective lay their credence in abundant research that allegedly proves the preeminence of the “neurobiological and genetic origin of the disorder” (e.g., Barkley, 2002). However, structural and functional neuroimaging studies have not managed to identify a unique etiology, no genetic marker has been identified with consistency, while heritability studies often contain methodological problems and are open to different interpretations about the role of environmental factors (Joseph, 2004, 2006; Furman, 2005; Matthews et al., 2014).
Evidence for the validity and reliability of ADHD as a distinct diagnostic entity is controversial and inconclusive (Timimi et al., 2004; Moncrieff and Timimi, 2013; Singh et al., 2015; Whitely, 2015). Nothing portrays more convincingly the validity issues of ADHD diagnosis than the findings of Morrow et al. (2012) who conducted a cohort study involving 937,943 children aged 6–12 years old in British Columbia, Canada. Their analyses revealed a relative age effect in the diagnosis ADHD, meaning that boys who were born in December were 30% more likely to receive a diagnosis of ADHD than boys born in January, while girls born in December were 70% more likely to receive a diagnosis than girls born in January (December 31 was the annual cutoff date of birth for entry to school in British Columbia). Four recent, independent of each other studies conducted in different countries (Taiwan, Turkey, Spain, and Germany) have also come to the conclusion that the probability of displaying ADHD symptoms is greater among children who begin school at an earlier age (Librero et al., 2015; Chen et al., 2016; Schwandt and Wuppermann, 2016; Gökçe et al., 2017). As Furman (2005) has concisely articulated it “ADHD is not a disease per se but rather a group of symptoms representing a final common behavioral pathway for a gamut of emotional, psychological, and/or learning problems” (p. 994). Regardless of the inconclusive evidence and skepticism presented above, ADHD in the EBP literature is consistently framed as an undisputable medical disorder, which reinforces a circular reasoning, that is, “he is inattentive, impulsive and hyperkinetic because he has ADHD and ADHD makes him inattentive, impulsive and hyperkinetic.”
EBPTs Insinuate That There is Within-Group Homogeneity
Adopting an “EBP for ADHD” approach insinuates that there are particular practices that are more effective for children who share a “common neurobehavioral quality.” Such an essentialist, “one size fits all” approach, fails to acknowledge critical factors that may affect children’s behavior (e.g., gender, personality, family system, and social expectations) and at the same time assumes within-group homogeneity. Given that children diagnosed with ADHD are often also diagnosed with “associated conditions,” from trauma, insecure attachment, and depression (Leuzinger-Bohleber and Fischman, 2010; Storebø et al., 2016) to learning disabilities (Mayes et al., 2000; DuPaul et al., 2013) and autism (Reiersen and Todd, 2008; Antshel et al., 2016), it is very unlikely that a particular EBPT will be helpful for all children grouped under the ADHD term. Data from qualitative neuropsychological evaluations confirm the complexity and heterogeneity of the profiles of children diagnosed with ADHD (Akhutina and Pylaeva, 2012; Solovieva and Rojas, 2014, 2015).
EBPTs Diffuse Non-Pedagogical Language in Education
Evidence-based practice literature is rich of a seemingly medical vocabulary, which surrounds ADHD and when reading it one repeatedly encounters terms as “core symptoms,” “chronic disorder,” “co-morbidity,” and so forth, while seldom reads about “learning theory,” “pedagogy,” “mediation,” “personality,” etc. This vocabulary affects the language used by teachers and at the same time may implicitly direct their pedagogical thinking since language has the power to open but also narrow windows to experience (Wertsch, 1998). As Lev Vygotsky put it “language, scientific language in particular, is a tool of thought, an instrument of analysis, and it suffices to examine which instruments a science utilizes to understand the character of its operations” (Rieber and Wollock, 1997, p. 281). One can argue, therefore, that when medical language dominates the field of education, then the “treatment of the symptom” takes priority over the process of learning which in itself becomes uncomplicated and manageable once the “disorder is fixed” (Wenger, 1998). This contradicts the conclusions of many teachers and educational researchers whose language tends to regard the learning process as being fundamentally complex, relational, and a potentially unsettling activity, deeply embedded in the particularities of individual lives (Bainbridge and West, 2012).
In the case of ADHD, the “disorder”-related jargon has infil-trated educational discourses to the point that it is hard for teachers to observe, describe, and conceptualize children’s school behavior without invoking such terms (Danforth and Navarro, 2001; Graham, 2006). ADHD metaphors are an illustrative example of how language may affect educational thought. Danforth and Kim (2008) used systemic metaphor analysis to explore the dominant metaphors that give structural and semantic content to the construction of ADHD. After analyzing the possible implications of the two dominant ADHD metaphors in the work of celebrated ADHD expert Russel Barkley (i.e., “Brain as cybernetic control system,” “People with ADHD as prisoners”), they concluded that such metaphors do not offer any conceptual orientation to effective or purposeful action to teachers and may even detract them from inclusive educational efforts (Danforth and Kim, 2008). Accordingly, Freedman (2015) found that the medicalized discourse has two major educational implications: (a) teachers tend to limit their pedagogical responses to children diagnosed with ADHD in favor of relying upon medical knowledge and (b) they are discouraged to construct policies that are consistent with the goals of inclusive education.
EBPTs are Produced by and Reproduce a Methodological Bias
The EBP literature on ADHD privileges particular kinds of evidence, namely evidence that has been produced by RCTs. RCT studies are generally considered either as the best source of evidence or, in some instances, as the only source of evidence. For example, the Campbell Collaboration—closely affiliated with the Cochrane Collaboration—has thus far published one review on psychosocial treatments for ADHD titled “Parent training interventions for attention deficit hyperactivity disorder” (Zwi et al., 2012) in which only RCT studies or quasi-RCT studies are included. To include another example, the influence of what works clearinghouse can be seen in “Evidence-Based Psychosocial Treatments for Children and Adolescents with Attention-Deficit/Hyperactivity Disorder” (Evans et al., 2014). In order to describe the quality of the studies included in their review, the authors adopted WWC’s “Evidence Standards for Reviewing Studies” together with the classification proposed by Nathan and Gorman (2002) both of which advantage RCTs. The bias toward RCT studies is also made apparent in the review titled “Psychosocial Treatments for Attention-Deficit/Hyperactivity Disorder” (Knight et al., 2008). The authors included a reference and recommended reading list at the end of their article in which they rated certain studies either as “of importance” or “of major importance.” Interestingly, only 2 out of the 49 references have been rated as “of major importance” and both refer to RCT studies. Another example is the review of Bjornstad and Montgomery (2005) on the effectiveness of family therapy for children with ADHD in which only RCT studies were included. Ten years later, the aforementioned review received the highest rate in a systematic appraisal of the evidence carried out by Watson et al. (2015). Not coincidentally, the review ranked as of the second highest quality in the aforementioned study also included only RCT studies (i.e., Schachar et al., 2002).
There are two major criticisms regarding this methodological bias. The first critique relates to the very nature of psychosocial interventions. Unlike other interventions (e.g., pharmacotherapy), they are very difficult—if not impossible—to be standardized. For example, how children respond to a particular psychosocial treatment depends not only on the treatment’s specifications but also on how children interpret and make meaning of the intervention. In RCT studies, experience, agency, and meaning are either excluded as not relevant or reduced to observable behaviors to be controlled and technically manipulated. What is forgotten in this quest for empirical “evidence” is that “evidence” is always our evidence—the product of our interpretations. What is given (and, thus, what counts) as evidence is always mediated in many ways by the inner historicity of our conscious and unconscious experiences, which are the result and product of social and historical practices. Second, research methods stipulate in advance what is to be ascertained since they are prioritized over research questions. It should be expected that in science a question first is formulated and the methods for answering it are selected afterward (Toomela, 2011). If it is pre-decided that RCTs must be used for research to climb up the “credibility hierarchy” (Hammersley, 2001) and be considered as “top quality,” many research questions that emerge would be impossible to be asked and examined because of the restricted choice of methods. This “methodolatry” (Janesick, 1998) (re)produces the exclusion of certain forms of research and represents a form of totalitarianism since it is leaving very little room for heterogeneous thinking (Holmes et al., 2006) and plurality of perspectives in what is considered as evidence based.
EBPTs Promote a Manualized Pedagogical Practice
Systematic literature reviews of the EBPTs reveal the usefulness, effectiveness, and preference for psychosocial interventions, which mainly promote behavior management and organizational and/or social skills training (Eiraldi et al., 2012; Evans et al., 2014; Watson et al., 2015; Gaastra et al., 2016). The “advantage” of these interventions is that they meet two criteria that are absolutely necessary for evidence-based research to be carried out: first, they can be completely standardized (i.e., manualized by developing a protocol-like approach) in order to exclude the impact of individual teachers or trainers on the effectiveness outcomes and, second, they are brief so as to avoid other influences to occur due to the length of time.
It is very clear that any other form of intervention which is either non-standardized or long-term is not valid within this approach. This means that only a very limited number of interventions (mainly behavioral ones) are acceptable. In addition, in order to avoid any other diagnosis influencing the outcome, children who exhibit “co-morbidity,” i.e., they carry more than the diagnosis of ADHD, have to be ruled out as research subjects. This significantly limits the number of children used by this research design—as we mentioned above, the reality is that “isolated” instances of children with only ADHD are extremely hard, almost impossible, to find.
There are a number of obvious criticisms regarding these two requirements. First, the demand for protocol-based interventions reduces the teacher and/or trainer to a disciple who needs to “follow the book” so as to faithfully execute the instructions of the manual. However, experienced teachers know that in order for interventions to work they have to be tailor-made to the unique and particular needs of specific children and their parents. They are also aware that effectiveness scores and numbers cannot adequately represent their own personal and idiosyncratic contributions to the development of secure and trusting relationships with these children, which is crucial for their long-term improvement. This “instrumental” approach creates a structure of knowledge and a type of educational “reality,” which is fixed, rigid, and dogmatic. Any form of knowledge or imaginative speculation which is not based on “evidence”—or cannot be manualized and/or operationalized—is subject to be discarded as irrelevant or superfluous. This produces a form of thought censorship and intellectual inflexibility which is counterproductive to an educational environment, which ought to promote creativity and imagination.
Second, the transformation of educational processes in measurable forms makes them readily commodifiable and ready to go on the market in the form of “packaged behavioral treatments.” If the effectiveness of behavioral management and training scores is prioritized over individual contributions, then teachers and trainers become exchangeable—they can be easily replaced without any consequences and changes in the relationship with the child. This entails a “technological model of professional action” (Biesta, 2007). In such a model, the role of educationalists seems to be perceived as one that is simply responsive to given requirements and predetermined goals. Hence, teachers’ “‘professionalism’ inheres in the willingness and ability to adapt to the necessities and vicissitudes of policy” (Ball, 2004) but also to “experts.” For example, in Eiraldi et al. (2012) “Strategies for Implementing Evidence-Based Psychosocial Interventions for Children with Attention-Deficit/Hyperactivity Disorder,” there is a call for the use of “expert consultants,” namely child and adolescent psychiatrists, school and clinical psychologists, and other behavioral health professionals who “can assist school districts with the development of systems and mechanisms for the use of EBIs and provide training and support to behavioral health staff” (p. 3).
Third, the brevity of these interventions seems to satisfy not simply the demands of this particular research design but also the political demand for “cheap” inclusive education. Yet, experienced teachers and trainers know that, among other things, success-ful interventions take time. Short-term interventions might appear to provide the desired, immediate outcomes but, in order to avoid relapse, one needs a more sustained, long-term effort. It is worthy of note that there are not any studies at the moment, which investigate the long-term relapse rate of these psychosocial interventions.
Fourth, as mentioned above, this evidence-based methodology can only be applied to a limited number of interventions and, even then, for only a very limited number of ADHD children who do not simultaneously belong to other diagnostic categories. However, instead of concluding that this kind of evaluation of interventions is inadequate, the advocates of EBPs insist that “descriptions of interventions that are amenable to scientific study (i.e., specific, operationalized, and logically arrayed) are a concrete scientific advance” over all other, non-specific, and non-operationalizable ones (Walker and Bigelow, 2015, p. 26). Thus, the latter are excluded and the former are the only ones employed and funded by educational authorities.
Current researchers’ motivation to employ EBP and RCTs so as to influence educational policy and public opinion, and thus promote psychosocial interventions in the treatment of children with ADHD, was generated by the real threat that any form of psychological intervention would completely disappear from the sphere of inclusive education, if there was an over reluctance to provide appropriate evidence for the effectiveness of these interventions—leaving thus pharmacological treatments as the only viable, effective option.
The paradox is that, for the reasons stated above, we are gradually realizing these interventions do not work when applied to teaching in schools—teachers and trainers already know this and it is only a matter of time before research proves it too. This will lead, in its turn, to the false conclusion that all psychological interventions do not work, and the only treatment left which is effective is a pharmacological one, signifying thus a perverse return to an outcome that EBPTs were initially designed to avoid and replace.
AK and AG have made equal and direct contribution to the work and approved it for publication.
Conflict of Interest Statement
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
Antshel, K. M., Zhang-James, Y., Wagner, K. E., Ledesma, A., and Faraone, S. V. (2016). An update on the comorbidity of ADHD and ASD: a focus on clinical management. Expert Rev. Neurother. 16, 279–293. doi: 10.1586/14737175.2016.1146591
Bjornstad, G. J., and Montgomery, P. (2005). Family therapy for attention-deficit disorder or attention-deficit/hyperactivity disorder in children and adolescents. Cochrane Database Syst. Rev. 2, 1–26. doi:10.1002/14651858.CD005042.pub2
Chen, M. H., Lan, W. H., Bai, Y. M., Huang, K. L., Su, T. P., Tsai, S. J., et al. (2016). Influence of relative age on diagnosis and treatment of attention-deficit hyperactivity disorder in Taiwanese children. J. Pediatr. 172, 162–167. doi:10.1016/j.jpeds.2016.02.012
Eiraldi, R. B., Mautone, J. A., and Power, T. J. (2012). Strategies for implementing evidence-based psychosocial interventions for children with attention-deficit/hyperactivity disorder. Child Adolesc. Psychiatr. Clin. N. Am. 21, 145–159. doi:10.1016/j.chc.2011.08.012
Evans, S. W., Owens, J., and Bunford, M. N. (2014). Evidence-based psychosocial treatments for children and adolescents with attention-deficit/hyperactivity disorder. J. Clin. Child Adolesc. Psychol. 43, 527–551. doi:10.1080/15374416.2013.850700
Freedman, J. E. (2015). An analysis of the discourses on attention deficit hyper-activity disorder (ADHD) in US special education textbooks, with implications for inclusive education. Int. J. Inclusive Educ. 20, 32–51. doi:10.1080/13603116.2015.1073375
Gaastra, G. F., Groen, Y., Tucha, L., and Tucha, O. (2016). The effects of classroom interventions on off-task and disruptive classroom behavior in children with symptoms of attention-deficit/hyperactivity disorder: a meta-analytic review. PLoS ONE 11:e0148841. doi:10.1371/journal.pone.0148841
Gökçe, S., Yazgan, Y., Ayaz, A. B., Kayan, E., Yusufoğlu, C., Carkaxhiu Bulut, G., et al. (2017). Association between age of beginning primary school and attention deficit hyperactivity disorder. J. Dev. Behav. Pediatr. 38, 12–19. doi:10.1097/DBP.0000000000000370
Holmes, D., Murray, S. J., Perron, A., and Rail, G. (2006). Deconstructing the evidence-based discourse in health sciences: truth, power and fascism. Int. J. Evid. Based Healthc. 4, 180–186. doi:10.1111/j.1479-6988.2006.00041.x
Janesick, V. J. (1998). “The dance of qualitative research design: metaphor, methodolatry, and meaning,” in Strategies of Qualitative Inquiry, eds N. Denzin and Y. Lincoln (Thousand Oaks, CA: SAGE), 35–55.
Leuzinger-Bohleber, M., and Fischman, T. (2010). “Attention-deficit-hyperactivity disorder (AD/HD): a field for contemporary psychoanalysis?: Some clinical, conceptual and neurobiological considerations based on the Frankfurt prevention study,” in Assessing Change in Psychoanalytic Psychotherapies with Children and Adolescents, eds J. Tsiantis and J. Trowell (London: Karnac), 139–177.
Librero, J., Izquierdo-María, R., García-Gil, M., and Pieró, S. (2015). Edad relativa de los niños en clase y tratamiento farmacológico del trastorno por déficit de atención/hiperactividad. Estudio poblacional en un departamento de salud. Med. Clin. 145, 471–476. doi:10.1016/j.medcli.2015.02.022
Moncrieff, J., and Timimi, S. (2013). The social and cultural construction of psychiatric knowledge: an analysis of NICE guidelines on depression and ADHD. Anthropol. Med. 20, 59–71. doi:10.1080/13648470.2012.747591
Morrow, R. L., Garland, E. J., Wright, J. M., Maclure, M., Taylor, S., and Dormuth, C. R. (2012). Influence of relative age on diagnosis and treatment of attention-deficit/hyperactivity disorder in children. CMAJ 184, 755–762. doi:10.1503/cmaj.111619
Rafalovich, A. (2005). Exploring clinician uncertainty in the diagnosis and treatment of attention deficit hyperactivity disorder. Sociol. Health Illn. 27, 305–323. doi:10.1111/j.1467-9566.2005.00444.x
Schachar, R., Jadad, A. R., Gauld, M., Boyle, M., Booker, L., Snider, A., et al. (2002). Attention-deficit hyperactivity disorder: critical appraisal of extended treatment studies. Can. J. Psychiatry 47, 337–348. doi:10.1177/070674370204700404
Schwandt, H., and Wuppermann, A. (2016). The youngest get the pill: ADHD misdiagnosis in Germany, its regional correlates and international comparison. Labour Econ. 43, 72–86. doi:10.1016/j.labeco.2016.05.018
Solovieva, Y., and Rojas, L. Q. (2015). Qualitative syndrome analysis by neuropsychological assessment in preschoolers with attention deficit disorder with hyperactivity. Psychol. Russ. 3, 112–123. doi:10.11621/pir.2015.0309
Storebø, O. J., Rasmussen, P. D., and Simonsen, E. (2016). Association between insecure attachment and ADHD: environmental mediating factors. J. Atten. Disord. 20, 187–196. doi:10.1177/1087054713501079
Timimi, S., Moncrieff, J., Jureidini, J., Leo, J., Cohen, D., Whitfield, C., et al. (2004). A critique of the international consensus statement on ADHD. Clin. Child Fam. Psychol. Rev. 7, 59–63. doi:10.1023/B:CCFP.0000020192.49298.7a
Timimi, S., and Timimi, L. (2015). “The social construction of attention deficit hyperactivity disorder,” in The Palgrave Handbook of Child Mental Health: Discourse and Conversation Studies, eds M. O’Reilly and J. N. Lester (Basingstoke: Palgrave Macmillan), 139–157.
Walker, R. D., and Bigelow, D. A. (2015). “Evidence-informed, culture-based interventions and best practices in American Indian and Alaska native communities,” in A Guide to Treatments that Work, 4th Edn, eds P. E. Nathan and J. M. Gorman (New York, NY: Oxford University Press), 23–54.
Keywords: attention-deficit/hyperactivity disorder, psychosocial interventions, evidence-based practice, critique, inclusive education
Citation: Koutsoklenis A and Gaitanidis A (2017) Interrogating the Effectiveness of Educational Practices: A Critique of Evidence-Based Psychosocial Treatments for Children Diagnosed with Attention-Deficit/Hyperactivity Disorder. Front. Educ. 2:11. doi: 10.3389/feduc.2017.00011
Received: 13 February 2017; Accepted: 21 March 2017;
Published: 04 April 2017
Edited by:Brahm Norwich, University of Exeter, UK
Reviewed by:Garry Squires, University of Manchester, UK
Anna Logan, Dublin City University, Ireland
Copyright: © 2017 Koutsoklenis and Gaitanidis. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: Athanasios Koutsoklenis, email@example.com