It is widely recognized that depression is common in those aged 11–17, with up to 10% of this group meeting criteria for depression, with diagnostic rates possibly increasing (1–3). Those within this age range are referred to, depending upon their exact age, as children, “pre-teen” children, adolescents, or youth (particularly those aged at least 13). This age range (from 11 to 17) is also the time at which many psychiatric disorders first appear (4). Tragically, suicide rates are also higher in this group (5, 6), with two large studies suggesting that 4–7% of pre-teens, adolescents, and youth have made at least one suicide attempts in the previous 12 months (7, 8). Those who have the highest risk for subsequent suicide appear to be pre-teens, adolescents, and youth students aged 11–18 who have previously harmed themselves and/or who have depression (9).

Addressing this issue is therefore critical, and there are several approaches that may help based on cognitive behavioral therapy (CBT) interventions primarily, although other approaches have been found to be helpful as well (10–13). Nonetheless, the research evidence to date has not clarified if it is more effective to target entire populations (so-called “universal” interventions) (14–17), or whether it is more effective to identify and then intervene in a smaller “high-risk” group (18–21). Overall, research findings and reviews have been supportive of a variety of both high-risk and universal interventions potentially reducing suicide rates in those aged 11–18, without definitively determining if one approach has better outcomes (22–27). For this reason, some have suggested that the most effective method to address depression and suicide in pre-teens, adolescents, and youth is to combine both universal programs as well as screening for those at highest risk, followed by targeted interventions (5, 28–35). It is also accepted that schools are the most appropriate setting to screen and intervene for those aged 11–18, including to increase resiliency against both depression and suicidality (17, 36–39). Therefore, combining both universal and high-risk approaches in schools may offer the potential to offer the most positive outcomes (40, 41). Supporting such an approach, a recent review and meta-analysis concluded that future “refinement of school-based prevention programs has the potential to reduce mental health burden and advance public health outcomes” (28).

We worked with a school district in Alberta, Canada, to help them design a new program to be given during designated “health” classroom time, with the intention being to try and improve resiliency against depression and to reduce suicidal thinking in their students. This was of particular relevance to them since this school district had experienced a sharp increase in the number of youth suicides. We suggested a multimodal approach to potentially best address this issue, including regular assessments of students’ progress and thinking, including the potential for subsequent clinical interventions and referral (following informed consent). Studying complex interventions is problematic and the present study is neither randomized nor controlled, which would have been unethical when studying youth who are suicidal, but instead uses an interrupted time series design which may be appropriate for youth participation (40). This entire approach we termed Empowering a Multimodal Pathway Toward Healthy Youth (EMPATHY) (41). Previously, we have reported initial 3-month outcomes from the EMPATHY program, which found a highly significant short-term reductions in suicidality combined with improvements in both depression and anxiety across the entire school district population (41). Here, we report on the longer term outcomes in a larger group of students that were took part in this school-based EMPATHY program over a 15-month period, focusing on the key outcomes of suicidal thinking, depression, and outcome.

Assessment #1 (Baseline) occurred during February and March 2014 (n = 3,244); Assessment #2 was the 3-month follow-up screening which occurred during May and June 2014 (n = 3,229); Assessment #3 was the 7-month follow-up screening which occurred during September and October 2014 (n = 4,860); and Assessment #4, the final 15-month follow-up assessment, occurred during the period April to June 2015 (n = 4,496). The timing of the follow-up screenings were based around the school year, with Assessments #1 and #2 in school year 1, and Assessments #3 and #4 occurring in school year 2, with all schools being on vacation in July and August 2014. Note that only five schools took part in school year 1, the three middle schools and two high schools. Of these students, a total of 6,227 fully completed at least one assessment, 4,917 completed at least two assessments, 2,796 completed at least three assessments, while 1,884 completed all four ratings (Table 1; Figure 2). There were an additional 324 students who signed in but then did not complete any assessments and were not therefore included in the analysis. Thus, a total of 6,651 students potentially were involved, but we have data on only 6,227. Of the total group of 6,651 students, 2,121 identified as males (32%), 2,273 identified as females (34%), while 2,257 students (34%) declined to identify their gender. For this reason, we did not analyze compare male and female data.

Of the 1,884 students who completed all four assessments over both years, this group consisted only of students in Grades 6–11 from the first school year. This is because they could not be in either Grade 6 in school year 2 (as they would be new to the system) or Grade 12 in school year 1 (as they would have graduated before school year 2). The number of students who completed Depression ratings for each grade at each time-point is shown in Table 1, and it can be seen that there were more participants in every grade in the second year as more schools were involved.

The present results demonstrate that a complex multimodal intervention can impact entire school populations in a positive manner. The results build on the initial 3-month findings we have previously reported (41) and demonstrate that there can be significant longer term improvements in the rates of suicidality, depression, and anxiety across large school populations. Given that the majority of students did not have depression or anxiety, the ability to demonstrate such changes on a school-wide basis suggests that these are meaningful. Supporting this suggestion was the finding that the percentage of students who were actively suicidal decreased from 4.4% of the entire school population at baseline to 2.8% when the EMPATHY program ended. This positive improvement occurred despite the fact that most students received no direct interventions, i.e., those in Grades 8–12 did not receive resiliency training, and most students in all Grades did not have guided Internet CBT interventions.

It is possible that several components that may have contributed to these findings. First, since students received rapid feedback after competing questionnaires, with interviews in most cases being less than 48 h in most case, this may have been impactful. Anecdotal feedback from several students was that this was seen as very positive, as the rapid feedback made them feel that their issues were taken seriously, an idea which has previously been reported (63). Again, anecdotal feedback suggested that friends of the individuals identified also gained positively as they now were aware of additional support. Additionally, involvement of parents may also have been a contributing factor (64, 65). In this instance, there was extensive anecdotal feedback from parents who were extremely positive about how this process helped their understanding of the issues their pre-teen, child, or youth was experiencing. The impact of other, somewhat similar, programs on school cultures has been reported previously (66–68). However, determining if this occurred would have required specific qualitative study which was not carried out.

An additional possibility is that improvements were due, in part, to Resiliency Coaches spending a lot of recreational time in the same environment as the students and, while they were specifically trained to not act as “therapists,” they found that on many occasions students would spontaneously approach them and sometimes confide in them. This included disclosures regarding physical and/or sexual abuse they had experienced or witnessed. The impact of this is uncertain, but this is a different role to that held by specific school counselors, and the Resiliency Coaches were surprised by how frequently their availability allowed such spontaneous interactions to occur. It is recognized that friendship skills can help adolescent depressive symptoms in combination with other approaches (69), so it is possible this may also have played a role. Currently, however, these possibilities all remain speculative. In future studies, it will be important to examine them more specifically to determine what, if any, impact each of these may have.

In terms of the potential benefit for a universal CBT intervention (OVK), this remains uncertain. Given that in the present study, it was used in a different and truncated manner in the first year and administered by a different group of individuals than those who have previously been shown to deliver it effectively, it cannot be considered to have been adequately tested in the present study. That universal interventions may be more effective when delivered by psychologists has been suggested from other studies (70) means that we are uncertain what our negative findings may indicate. Determining the relative importance of this aspect of the program will be required in future studies, and most importantly, it is necessary to study the effectiveness of the full 16-session program. Nonetheless, it is well recognized that it can be difficult to detect the overall effect of such universal interventions, and this may explain the large number of studies that are unable to show a benefit. The potential benefit of universal approaches such as OVK, as well as those recently examined for suicide prevention (27, 33), needs to be evaluated as part of the approach of combination programs such as EMPATHY, but are unlikely to be measures that on their own significantly reduce youth depression and suicide rates.

In terms of the measurement scales used, it is important to recognize that simply using depression rating scales on their own may not be sufficient. This is because many pre-teens, children, and youth aged 11–18 have “sub-threshold” depression (i.e., depressive symptoms that do not meet diagnostic criteria) (71, 72), and many of these also have active suicidal thoughts (8). For these reasons, and the fact that scales for depression do not examine previous self-harm behaviors, most standard depression screening tools likely miss many youth who have active suicidal thoughts (73). Nonetheless, it does appear that the questions contained within the PHQ-9 may be very useful as part of a screening tool (74).

A further advantage of the present findings was that we could examine findings from two different groups: first those who completed assessments at least once (n = 6,227), giving cross-sectional data, and second the smaller group of 1,884 students who completed all 4 assessments and therefore provided longitudinal data. Utilizing findings from both groups increases our ability to have confidence in the findings since they both had similar outcomes.

An additional, anecdotal report was that many of the teachers and school Principals reported a marked decrease in the number of bullying cases brought to their attention. Although the schools did not formally collect specific data to measure this suggestion, these findings are compatible with our previous suggestions that the positive changes found initially at 3 months (41) (and in the present study) may have been due to a combination of several interventions, as well as non-specific interactions. Further research is needed to validate any such suggestions, and if so to more accurately determine whether any specific interventions, on their own, can have similar impacts across a school district.

When comparing the present findings to those from previous research it is important to note that we included all students within a complete school district. There was no exclusion of particular schools or groups of students, both of can conceivably bias any findings. This is also different in that here we have analyzed data collected as part of a multimodal program introduced on a district-wide basis to all schools, rather than a research study in which single, or combined, factors are compared. However, such an approach leads to the significant limitation that the lack of any control groups limits conclusions that may otherwise be made. Nonetheless, since one of the challenges facing those who wish to carry out research to improve the mental state of youth is the fact that students move between schools, only by examining depression rates or suicidal thinking across entire student populations can more accurate conclusions be made.

Another potential limitation is that the EMPATHY program took place during two different school years, and it was therefore conceivable that there might have been timing effects that impacted the outcomes. Thus, there could have been a major change only in school year 1 and not school year 2, in which case there should be no improvement in scores between Assessments #2 and Assessments #3. Or, alternatively, students may improved at Assessment #2 simply because of they were close to the summer vacation but then got worse when they returned (Assessment #3), only to improve again and have good results at the end of the second school year (Assessment #4). However, the results showed no such impacts, and in particular when we examined changes between the end of the first school year and the start of the second school year (Assessments #2 and #3, respectively), we found ongoing statistically significant improvements between these two visits for depression scores. This suggests that the EMPATHY program had long-lasting benefits which continued over time, and further might imply that the more interactions the students had, the greater the benefits. This was also suggested by the findings from the longitudinal group of 1,884 students. Both of these findings suggest no strong seasonal component underlay the current findings.

Other study limitations are that children, pre-teens, and youth all develop cognitively over time. However, we had no measurements either at baseline or at 15 months regarding how they may have possible changed during the 15-month follow-up period. It is conceivable that this may have, in some non-specified manner, altered the findings. We think this unlikely, given the consistency across grades, but we cannot be certain this had no impact. For example, all the 1,884 children and youth who were followed-up repeatedly were 15 months older than at baseline age; therefore, their ability to respond to questions and ease of handling computerized assessments most probably had improved. Similarly, we have no information regarding the general cognitive ability of study participants. In any classroom cohort, there are be adolescents with learning problems, or intellectual difficulties, but we have no information about individual students and whether this impacted outcomes, and no information about gender and age dispersion in classrooms or across grades, or cognitive ability at baseline regarding readability or understanding of questions.

Our inability to discuss the data in terms of possible gender differences is also a limitation. The large number of individuals who declined to identify their gender may have been, in part, a response to the fact that there were only two choices. Clearly, future youth surveys should give greater choice to individuals as to how they choose to identify their gender, and this may increase compliance rates.

In conclusion, the need for longer term, effective programs that can reduce youth suicidality and depression remains clear (5), and although a recent review showed that many programs do have some positive benefits (28), we believe the results from the present EMPATHY study program suggest that multimodal approaches may lead to significant mental health benefits over a longer time period. The present results from this program give support to the utility of such multimodal interventions, including combining both universal and targeted approaches, although controlled studies are needed before widespread implementation can be recommended. Initially, we had planned further studies, in part to determine reproducibility both in similar and different school populations. Unfortunately, although this was initially supported, with the unexpected ending of funding it was not possible to do this. However, we strongly encourage others to do this as implementation of any major programs in youth should only be instituted after rigorous research demonstrating reproducibility.

This program was approved by the Health Research Ethics Committee of the University of Alberta on December 5, 2013, ethics protocol number Pro00041063. Amendments to the original protocol and consent letters were approved in January of 2014, with all changes to the informed consent letter subsequently approved prior to the start of the program. The first student was screened in February 2014 and the follow-up screening was completed by June 2015. No additional student follow-up is possible, since program funding was terminated unexpectedly in June 2015 following a change in Provincial Government in Alberta in May 2015. This study is registered with http://ClinicalTrials.gov identifier: NCT02169960. Although an application for this was completed at the time of the ethics approval, due to an unfortunate administrative oversight, the actual submission to the registration database did not occur until July 2014. This omission was corrected as soon as it was recognized and was noted in our original publication (41). It should be noted upon consultation with the school district (and with the agreement of the Health Ethics Board of the University of Alberta, the supervisory ethics board), the screening process was deemed not part of the study since the school district was adamant that programs that they implement school district-wide did not need specific parental consent. This was based on the fact that education for any subject matter within these schools, even areas that may be considered controversial (such as discussions about sexual matters or sexual orientation, or ethical matters such as physician-assisted suicide) do not require parental consent. Thus, they did not believe that teaching students on a universal, district-wide basis about mental health issues required a different process than any other subject. Similarly, they asserted that assessing students on a range of topics is done frequently, again, on a universal, district-wide basis, and the assessments in the screening were not something for which specific consent was required. Indeed, they felt that requesting specific consent for what was taught or collected as part of class would be antithetical to their processes. Per their normal processes, they sent an information letter to all parents about the new program, thereby informing the parents/guardians that the school district was introducing a school district-wide “Resiliency Project” (they did not want to use the term “EMPATHY program” as they had recently introduced a different program that had a similar name). As noted, this approach was specifically agreed by the University of Alberta Health Ethics Review Board. The University of Alberta, as well as the Strategic Clinical Network for Addiction and Mental Health of Alberta Health Services, helped design the most appropriate screening tool and intervention program, train teachers about assessment, provide close liaison to clinical services in the region when required, and evaluate the effectiveness of the program. In discussions, the school set a threshold to trigger invitation to the targeted intervention, and this threshold varied by age and by school. This component included the option of taking part in a guided Internet-based CBT program. Since participation in this additional CBT program was not part of regular teaching, prior to receiving any such additional Internet-based CBT, the parents or guardians of the students provided written consent on behalf of the youth. Additionally, the student provided written assent, and all documents and the process were approved by the University of Alberta Health Ethics Review Board. The signed consent and assent forms were kept as part of the trial documentation, in a secure, private location. The data collected by the school as part of their regular classroom assessment were made available for analysis by the research team, but only on an anonymized basis. From a scientific viewpoint, the most rigorous findings would have come from a study in which the design included a group of students who were randomized to one group or another, one of which would have been to not have any intervention. Indeed, such an approach has been carried out in studies looking at depression rates in students and in theory randomization can be carried at the class level or school level. However, this was a school district-wide program that was specifically focused on decreasing the number of youth who were actively suicidal, with assessments of students to measure this. To have considered a program (or study) in which a group of students was assessed, found to be suicidal, but then not offered an intervention would clearly have been unethical. For this reason, we determined that the most appropriate methods to analyze the data collected by the school district was by examining data from both a cross-sectional analysis of all students who completed at least one assessment, as well as using a longitudinal analysis for those students who completed all four assessments (where we would be using each student as their own control). This approach to analysis of the data was approved by the Health Research Ethics Committee, with subsequent modification in April 2016 to approve secondary analysis of the information obtained from the EMPATHY program (presented here in the present publication) with the ethics protocol number Pro00061164.

PS, MB, VS, AA, JG, SH, CP, PL, KR, and RE were responsible for the design of the program. IC, SC, and CM were responsible for all statistical analysis, assisted by VS and KR.

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.