There is growing recognition of the importance of program theory in the development and evaluation of programs. However, reports of using theoretical models to develop programs in the social service sector are rare in the literature. The Logic Model offers an accessible framework that can be used to enhance capacity among frontline service workers with regard to the steps needed for program development and evaluation (1). This paper describes the development and evaluation of a train-the-trainer workshop (TTT) to develop (design and implement) community-based positive psychology family interventions. The workshop was delivered to social service workers (a combination of registered social workers and related-service workers) working in community agencies in Hong Kong.

Public health primary prevention requires effective, brief, and acceptable interventions to be delivered to large numbers of individuals in the community at low cost (2). Academic-community partnerships offer a synergy to develop large intervention programs (3). Academics bring the tools of science, whereas community service providers contribute insight regarding the needs of their constituents, and experience with the strategies that the community participants will find accessible and acceptable. As social service workers face constant changing needs, an even greater impact can be made if they can be provided with basic tools to adapt, develop, and evaluate new programs.

The Logic Model provides a simple road map for systematically developing interventions (4, 5). The model has been used as a planning and/or evaluation framework for a wide range of participants and settings, such as palliative care (6, 7); professional development programs for medical and health professionals (8, 9); programs for prevention of diabetes (10), HIV (11), breast cancer (12), and teenage pregnancy (13); research capacity in practice (14–16), primary care reform (17, 18); and community health promotions (19, 20). Although there has been increasing interest in the literature on planning and evaluation frameworks, the Logic Model has not been popularly used in the social service settings.

The train-the-trainer educational model has used experts to teach key stakeholders to deliver specific services (21) and has been broadly applied to workforce capacity building in health and social care settings (22–24). A TTT that teaches social service workers to use the Logic Model could more broadly enhance skills for the development and evaluation of programs. These skills could have considerable value in eventually maximizing the likelihood that programs being utilized in the community are based on local evidence. To our knowledge, there are no reports of TTTs designed to teach social service sector staff how to develop and evaluate simple programs to address needs in their community.

The FAMILY Project has been initiated and funded by The Hong Kong Jockey Club Charities Trust in collaboration with the School of Public Health of The University of Hong Kong (HKU-SPH). The aim of the FAMILY Project is to promote health, happiness, and harmony in Hong Kong (website: http://www.family.org.hk/) (25). In the first phase of the project, we conducted two TTTs for the Happy Family Kitchen Project (HFK) and the Enhancing Family Well-being Project (EFWB) to teach social service workers to develop community-based positive psychology family interventions (26, 27) leaving them with some freedom with regard to the specific content of their interventions. The second phase, the Happy Family Kitchen Project-II (HFK-II) aimed to increase public health impact by recruiting a larger sample of underprivileged families from different districts in Hong Kong and to respond to the expressed interest of frontline staff in the design and assessment of programs. The TTT presented in the current paper did not aim to teach a specific intervention, but rather to teach the general constructs of positive psychology in the context of family intervention development. The innovative component of our effort was that we provided frontline social service workers, who had little experience with research, with a framework for intervention development and evaluation in the form of the Logic Model.

Thus, in this paper, we examine the effectiveness of a training workshop for social service workers to develop community-based positive psychology family interventions. The primary outcomes were the changes in trainees’ knowledge, self-efficacy, attitude, and intention in the application of the general constructs of positive psychology and the Logic Model into the family interventions that they would develop after training. The secondary outcomes were trainees’ reactions to the training content, sharing of the knowledge, and benefits to their service organizations.

Fifty-six trainees, including 41 registered social workers, 13 related-service workers, and two teachers, attended the training workshops and were included in this study (Table 1). Four trainees attended the workshops, but left before completing the post-training assessment. At 6-month follow-up, four trainees did not respond because their corresponding service organizations had withdrawn from the HFK-II prior to the start of the intervention programs, and four trainees declined to complete the questionnaire. At 1-year follow-up, 10 trainees had resigned from their service organizations and three trainees could not be contacted. Fifty-six completed questionnaires at pre-training, 52 completed immediately after training, and 44 and 31 trainees completed at 6-month and 1-year follow-up, respectively, and all these questionnaires were thus collected (Figure 2). Thirteen trainees participated in the focus group interviews.

Table 2 shows the characteristics of trainees. The demographic characteristics of trainees who completed the 1-year questionnaire and those who did not and of trainees who joined the focus groups and those who did not were not different (Table S2 in Supplementary Material).

The trainees indicated a high level of satisfaction with the workshop. The TTT enhanced trainees’ competence (perceived knowledge and self-efficacy) and attitudes toward using positive psychology constructs and the Logic Model to develop family interventions. The findings ranged from small to large effect and were sustained to 1 year following the workshop. The current TTT adds to the literature as follows: (i) we provide a practical example of development (including design and implementation) and evaluation of a training workshop and (ii) we provide a template for applying the Logic Model in developing community-based family intervention programs.

Our findings are consistent with those from the TTTs of our previous projects (26, 27): both the HFK TTT and the EFWB TTT enhanced trainees’ competence and attitude in relation to using positive psychology to develop community-based family interventions. The effect size was the greatest in the current study (Cohen’s d ranged from 0.39 to 1.43), compared with those of the HFK TTT (Cohen’s d ranged from 0.37 to 1.22) and the EFWB TTT (Cohen’s d ranged from 0.24 to 1.06) (26, 27). The greater effect size was possibly because the current workshop was the third one we designed in the FAMILY Project to teach social service sector workers to apply positive psychology constructs in family interventions. The experience gained from the previous TTTs informed the selection of positive psychology themes and refinement of the training content, thereby enhancing the current TTT’s applicability and acceptability.

Implementation science is a rapidly growing discipline. The current TTT introduced the Logic Model, a simple tool with a clear structure and pictorial description, intended to be useful over the lifespan of a program from design to implementation and evaluation. The Logic Model helped the trainees to articulate each component of the entire program and consider the contextual factors of the participants. It facilitated the identification and collection of data for detecting behavior change and promoted trainees’ awareness regarding the importance of program fidelity. The goals of our training went beyond a typical program that targets only specific skills and concepts that are necessary to deliver a single intervention. We used the Logic Model as a systematic means to incorporate research and quality improvement in our interventions, which helped trainees to understand the process of “evidence-generating” and accomplished the mission of integrating clinical practice and research. We have previously attempted to teach some basic development and evaluation skills in our programs, but did so in a brief 1 h session, which elicited feedback that there was a need for more of this kind of training. The current workshop used a 3-h session to introduce the Logic Model instead of a 1-h session on program design and assessment in the EFWB TTT (27). We delivered detailed information on the Logic Model and included a group exercise and discussion about the application of the model. Such additional activities provided the trainees with more opportunity to develop confidence and skills to integrate the model into their individual practices.

Our program had several strengths. Our 2-day workshop was short, compared with most reports of TTTs in the literature, which ranged from 3 days to 10 days (24, 40–42). Brevity should make the training less costly and more widely accessible. We used experiential learning methods that have been shown to be effective in enhancing the professional development of social service workers (26, 43). We also provided post-training support by distributing training manuals, which helped promote trainees’ self-efficacy for designing and implementing future interventions (23). Finally, many TTTs in the literature have reported either qualitative evaluations such as narrative feedback and process evaluation (43, 44) or quantitative evaluations (24, 45), but not both. We used the Triangulation Research Method, which includes both quantitative and qualitative assessments to examine the effectiveness of the TTTs, and thus increased the validity of the results.

We acknowledge some limitations in our study. As validated questionnaires were not (and still are not) available in the literature, we used study-specific questionnaires for our targeted outcomes. We assessed only trainees’ perception of knowledge, but did not test their actual knowledge. The attrition rate was high, because our collaborating service organizations had a high staff turnover over the course of the 1-year study resulting in loss of follow-up in some of our trainees. We mitigated this concern somewhat by using the conservative intention-to-treat analysis. Our sample size was small, and the trainees who joined our study might not be representative of their colleagues within and beyond their own service organizations. We had no control group and could not assess the possibility that social desirability bias might have led to over-estimates of effect size. However, we did observe varying effect sizes for different outcomes, some of which diminished at 1-year follow-up, suggesting that primary social desirability effects would not be substantial. A study design with a control group for a TTT may provide stronger evidence, but is less practical and acceptable in community settings.

The need to increase capacity in the community has been well acknowledged. Our study offers a practical example of academic-community partnerships to transfer theory to practice for in-service training. We have provided an operational framework and process to educate frontline social service workers to design, implement, and evaluate their community-based interventions.

Ethical approval was granted by the Institutional Review Board (IRB) of The University of Hong Kong/Hospital Authority Hong Kong West Cluster (HKW IRB reference number: UW12-502) and was retrospectively registered at the National Institutes of Health (Identifier number: NCT01796275). Written informed consents were obtained from the trainees.

THL and SC conceived and designed the experiments. MM, AW, and CY contributed the tools and performed the experiments. AL performed the analysis. AL, SC, and T-hL wrote the manuscript. All authors approved the final version of the manuscript.

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.