Patient Uptake, Experiences, and Process Evaluation of a Randomized Controlled Trial of Internet-Based Cognitive Behavioral Therapy for Tinnitus in the United States

Introduction

Tinnitus is a chronic symptom, characterized by the perception of sounds in the ears or head of an individual without any external sound source, is a highly prevalent symptom affecting at least 10–15% of the adult population (1). Although not everyone is bothered by tinnitus, a proportion of those experiencing tinnitus finds it very distressing and it may affect many aspects of daily life, such as sleeping and concentrating. As experiencing tinnitus is associated with an increased risk of psychological difficulties, such as anxiety, depression, those distressed require interventions to help them cope with the tinnitus (2, 3). Managing tinnitus can be, notoriously challenging as there is often no medical cure (4). Management thus focuses on the address associated hearing loss, educating the patients, and provide tools and strategies to manage the tinnitus and associated problems. The intervention with the strongest research evidence according to tinnitus practice guidelines (5–7) and several systematic reviews (8, 9) is cognitive behavioral therapy (CBT).

A CBT is a psychological treatment addressing the unhelpful behaviors, thought patterns, and emotional reactions caused by tinnitus (10). To increase access to the CBT for tinnitus, an internet-based CBT for tinnitus (ICBT) was developed in Sweden (11) in a self-help format with psychological guidance. This program was later translated to German (12) and English (13). To further increase the accessibility, the ICBT for tinnitus is adapted to be delivered by the audiologists (14) with some training to handle the CBT elements without compromising the outcomes (15–18). To further increase the availability of CBT, the intervention was adapted for the population of the United States (19) and also translated into Spanish to reach the Spanish and English speakers (20). As a pilot study indicated the feasibility of the intervention (21, 22), a randomized clinical trial (RCT) was undertaken (21, 23). The studies in the United Kingdom were the first in which the ICBT was delivered by an audiologist (16–18). When delivered by an audiologist, this RCT in the United Kingdom indicated that ICBT led to a greater reduction in tinntius distress compared with weekly monitoring with an effective size of d = 0.46 [0.14–0.77]. The results were in line with the outcomes obtained in the studies in which psychologists had provided the guidance. In addition, there was a greater reduction in the negative tinnitus cognitions and insomnia. The results remained stable over the 2 month follow-up period. Although the favorable outcomes were obtained, there were some difficulties encountered during the running of the trial, largely surrounding low engagement, and the poor compliance rates for the questionnaire completion. To increase the applicability of the intervention and to prepare for later dissemination, an understanding is needed regarding the factors contributing to the outcomes, engagement, and compliance. Such understanding is best obtained by considering the perspectives and experiences of the participants of an intervention.

The process evaluations are a means of providing a framework for analyzing the key components in the healthcare interventions. Such evaluations are important as various external factors can affect the health conditions and intervention uptake (24–26). The different process evaluation models for healthcare interventions have thus been developed, such as the Reach, Effectiveness, Adoption, Implementation, and Maintenance framework [RE-AIM (27, 28)] and the components suggested by Baranowski and Stables (29) and Linnan and Steckler (30). Although each model is unique, the overlapping component includes investigating the recruitment procedures, the context of the research, the intervention delivery and how it was received, the outcomes obtained, and the implementation of the intervention. Despite the relevance of process evaluations, they are not widely used within audiology with only a few process evaluations related to hearing difficulties (31–33) and one related to the ICBT for the population of the United Kingdom (34).

The research objective of the present study was to identify the processes that could facilitate or hinder the clinical implementation of ICBT for tinnitus in the United State (23). This was done by considering the full trial implication from the recruitment to post-intervention follow-up. This was in view of gaining insights into the applicability of ICBT for the population in the United States and identify the factors that could help optimize dissemination. The specific objectives were to evaluate the processes regarding the enrollment, allocation, intervention delivery, the outcomes obtained, and the trial implementation.

Materials and Methods

Research Design

This study was a process evaluation of an RCT of ICBT for tinnitus conducted between March 2020 and July 2020. The process evaluation was conducted in parallel to a clinical trial that investigated the efficacy of ICBT for tinnitus in the United States (23). The participants were randomized with a 1:1 allocation ratio to the experimental group to receive ICBT for 8 weeks, or the control group who received the intervention after a delay of 8 weeks during which time they were monitored weekly. The outcome measures were completed at baseline, T1 (post-intervention for the experimental group), T2 (post-intervention for the control group), T3 at 2 month follow-up post-intervention for each group, T4 at 1 year follow-up.

The RCT and its protocol were pre-registered at the Clinical Trials.gov: NCT04004260 on July 2, 2019. Ethical approval was obtained from the Institutional Review Board at Lamar University, Beaumont, Texas, United States (IRB-FY17-209). The study was conducted and reported according to the Consolidated Standards of Reporting Trials (CONSORT) EHealth guidelines (35). An independent data monitoring committee monitored the running of the trial.

Participants

Target Recruitment

Following the sample size calculations, the goal was to enroll 152 participants. To ensure inclusivity, the aim was to recruit 48 Hispanic or Latino participants and 94 non-Hispanic or Latino. The racial categories targeted were American Indian/Alaskan Native (2), Asian (6), Black or African American (18), more than one race (20), and White (106).

Eligibility Criteria

Eligibility was determined in a two-stage process. The inclusion criteria were that the participants needed to be aged 18 years or over and living in Texas, United States. Computer and internet access were required. The participants had to have experienced tinnitus for a minimum duration of 3 months and have a score of 25 or above on the Tinnitus Functional Index (TFI), suggesting a need for tinnitus care (36). The exclusion criteria were indications of significant depression (≥15 scores) on the Patient Health Questionnaire [PHQ-9 (37)]. Other aspects that resulted in the exclusion are: indications of self-harm thoughts or intent (i.e., answering affirming on Question 10 of the PHQ-9 questionnaire), reporting any major medical, psychiatric, or mental disorder which may hamper commitment to the program or tinnitus as a consequence of a medical disorder still under investigation.

Eligibility Screening

Initially, the participants completed the baseline measurements online (T0). Following completion, a telephonic screening was arranged, to ensure participants fulfilled the study requirements. For any participant indicating possible self-harm thoughts or significant depression on the PHQ-9, a psychologist would phone them within 24 h. A clear protocol was set up for these participants. The scores were discussed, and the participants were questioned as to whether they had additional help and support for these problems. If there were any concerns, a stabilization plan was set up. If this was not possible, the crisis team would be contacted. The person would be kept on the phone until the crisis team arrived, although no such cases were reported in the current study. Everyone who called on the phone was provided with the emergency contact details. The psychology or other appointments were arranged as appropriate, or referrals made were indicated.

Intervention

The ICBT content was based on a Swedish CBT self-help program (38), transformed into an 8 week interactive e-learning version (39) and then, adapted linguistically, culturally, and functionally to ensure the suitability for the population of the United States (19, 20). The ICBT platform consisted of 22 modules with worksheets and quizzes (14). The participants were asked to read the modules weekly and ideally spend at least 10 min each day practicing the suggested strategies. The intervention specifically targeted reducing the activity limitations and participation restrictions and included applied relaxation due to the importance of this aspect in tinnitus managment (40). Both the groups received the same intervention, only the timings regarding receiving the intervention varied.

The guidance was provided to support the participants while undertaking the intervention. This included monitoring progress, monitoring the weekly scores, providing feedback on the worksheets completed, outlining the content of new modules, and answering questions. The participants who did not engage were contacted to support participation and to discuss the possible barriers. An encrypted 2-way messaging system within the ePlatform was used to communicate (39). The intervention was provided free of charge and the participants could continue to access it after the intervention was completed.

Parameters Used for the Process Evaluation of the Clinical Trial

The overlapping and relevant elements from the healthcare process evaluation models were used to identify the process to evaluate for this clinical trial from the RE-AIM model (27, 28), from Baranowski and Stables (29) and Linnan and Steckler (30). Five processes were selected, namely, enrollment, allocation, intervention delivery, the outcomes obtained, and the trial implementation as illustrated in Table 1. A demographic questionnaire was used to establish the health-related and tinnitus-specific information at baseline (T0). The standardized outcome measures were completed at baseline (T0), after the experimental group completed the intervention (T1) after the control group completed the intervention (T2), at 2 month follow-up (T3), and 1 year follow-up (T4). The primary outcome measure was tinnitus severity as measured by the TFI (36). The secondary outcomes were:

▪ The Generalized Anxiety Disorder−7 [GAD-7 (41)] to assess the symptoms of generalized anxiety disorder.

▪ The PHQ-9 (37) (38) indicated the symptoms of depression.

▪ The Insomnia Severity Index [ISI (42)] assessed the presence of insomnia.

▪ The Tinnitus Cognitions Questionnaire [TCQ (43)] was used to measure the negative tinnitus cognitions.

▪ The EQ-5D-5L (44) measured general health-related quality of life.

▪ The Tinnitus and Hearing Survey [THS (45)] was used as a short measure to identify the tinnitus severity, hearing disability, and hyperacusis of the participants.

▪ A short questionnaire was administered to try to determine the effect of coronavirus disease 2019 (COVID-19) on the study asking whether the participants had COVID-19 and how this was affecting them. This questionnaire was added during the middle of the study due to the study being administered during the height of the first wave of the pandemic.

▪ A satisfaction questionnaire was designed to assess the suitability, content, usability, presentation, and exercises from the intervention consisting of 15 five-point Likert-type scaled questions (39).

▪ The open-ended questions to find out more about the experiences from the intervention such as which modules were helpful, what barriers were found, and suggestions for improvements. This was done by both asking open-ended questions in a survey as well during the phone calls made to the participants post-intervention.

TABLE 1

Table 1. The processes in the clinical trial.

Data Analysis

Data analysis incorporated a mixed approach, including both quantitative and qualitative analyses. The Statistical Package for Social Sciences (IMB SPSS for Windows V.26.0, NY, USA) was used for the statistical analyses (46).

Descriptive statistics were used to describe the sample characteristics. The continuous variables were summarized with means and SDs. The categorical variables were described using frequencies and percentages. The effect sizes were used to determine the outcome effectiveness. The outcomes related to the satisfaction of the intervention and the specific components were rated on a 5-point Likert scale and were analyzed using descriptive statistics.

The open-ended questions were analyzed using a qualitative content analysis described by Graneheim and Lundman (47). The content analysis enables the systematic interpretation of the participant statements to identify the central aspects (a set of condensed categories) that emerge from careful examination of the raw data using a bottom-up approach. Various steps were involved in the process. Initially, the responses were read repeatedly and coded for “meaning units,” which are statements that relate to the same central category. These meaning units formed the units of analysis for coding. The next process was identifying the categories that were repeatedly mentioned. The responses that related to the same category were grouped together. The repeated patterns were further grouped until the clear condensed categories and subcategories were identified. The codes were then gradually merged into the broader categories and subcategories by grouping thematically similar codes together. The categories were subsequently condensed by combining the categories with similarities, ensuring that the categories were mutually exclusive. The category labels were assigned. After selecting the codes and categories, the original responses were checked to ensure they were in line with the assigned categories and to identify if any additional categories emerged. The dataset was rechecked for consistency. The data coding was performed independently by the two researchers. The coding was compared and in case of inconsistencies, these were discussed.

Results

Processes Related to the Enrollment

The recruitment and participant screening processes are discussed in this section.

Processes Related to Recruitment

The multidimensional recruitment strategies were employed to obtain a varied sample. Table 2 outlines the different recruitment strategies used and how successful each was. A comprehensive study website was designed to provide information for those who were interested in the study. This included how to register, the aims of the program, the time commitment, and the nature of the intervention. All the recruitment strategies guided individuals to the study website (www.tacklingtinnitus.org). Google analytics indicated that 3,720 users viewed the website as outlined in Table 2. This indicates that the recruitment strategies drew sufficient interest to the website.

TABLE 2

Table 2. The various recruitment strategies used.

All the recruitment materials were translated from English into Spanish, adding an extra layer of complexity. Both the English and Spanish team members were required for this study. Recruiting the Spanish participants required additional thought and it was difficult to target these participants. Direct contact was included during the recruitment, using a public patient initiative (PPI).

Processes Related to Participant Screening

There were 157 participants who registered on the study website and showed interest prior to the recruitment opening. Further recruitment means drew a total of 315 participants who showed interest in the study and were screened (263 English and 52 Spanish). Of these, 158 were eligible (as shown in Figure 1). The exclusion reasons included having high depression scores or a positive answer regarding self-harm intent (46 English and 3 Spanish), low tinnitus severity (36 English and 1 Spanish), living outside the recruitment area in the State of Texas, United States (5 English and 40 Spanish). According to the protocol set for this trial, a psychologist was required in the team to make phone calls to the 49 participants (three using translation) who had high depression scores. Having a psychologist was helpful, as tinnitus is best approached from a multidisciplinary perspective (4–10) but involving more experts may be an expense, not all the teams can accommodate. As all the participants contacted were those with known depression that was being treated and there were no cases that raised concern, this particular study did not specifically require the expertise of the psychologist to deal with any serious depression or self-harm intent.

FIGURE 1

Figure 1. Flowchart of enrollment, allocation, intervention delivery, and outcome assessment completion. T1 is after the experimental group received the treatment, T2 after the control group received the treatment, T3 is at 2 month follow-up, and T4 is at 1 year follow-up.

Processes Related to Allocation

Reach

The target numbers according to sample size calculations were n = 152. A total of 158 participants were enrolled as shown in Figure 1, which indicated that the required participants were reached. There were fewer Spanish speaking participants than aimed for.

Context

The social, demographical, and socio-economic characteristics of the participants were identified. Equal gender ratios were recruited with n = 80 (51%) being female and n = 78 (49%) being male. A wide age range was represented (19–84 years) with a mean of 57 (SD: 12) years, which correspond to the expected range due to the incidence of tinnitus being most prevalent in the 40–70 years age range (1). A wide range of tinnitus duration was found (3 months−70 years) with the average tinnitus duration being 14 years (SD: 14).

The majority had obtained a university degree (n = 84; 53%) or other training vocationally or from a college (n = 53; 34%). Only a minority had only a high school qualification (n = 21; 13%). The majority were skilled workers or professionals (n = 95, 61%) with only 10 (6%) not working, and 52 (33%) being retired. To ensure the participants were representative of those living in the United States, different ethnic categories were targeted. In addition, the planned ethnic enrollment was less than expected as shown in Table 3. Most of the participants indicated that they were frequent computer and internet users (n = 144, 91%) with only 9% (n = 14) having only basic computer skills.

TABLE 3

Table 3. The ethnic and racial characteristics of the participants.

The clinical presentation of the participants indicated tinnitus severity at a level requiring the need for a tinnitus intervention with a mean TFI score of 53.98 (SD: 17.54). The mean anxiety score on the GAD-7 was 5.6/21 (SD: 4.26) and the mean PHQ-9 was 5.48/27 (4.12) indicating mild anxiety and depression. This reflects the inclusion criteria requiring no participants with significant levels of depression (15 or more on the PHQ-9). The ISI indicated that this group had subthreshold insomnia with a score of 10.05 (SD: 5.84). The context of the research thus showed that the participants with troublesome tinnitus and a wide range of demographic backgrounds were drawn to the study.

Randomization

As an unbiased randomization process is required in a clinical trial, the randomization process was considered. Randomization was not done by the team directly involved with the participants to avoid any possible bias. The team statistician provided computer-generated randomization scheduled and an independent research assistant randomized the participants in a 1:1 allocation in the blocks of varying sizes after the participants were pre-stratified for language (English and Spanish). Following randomization, no group differences were evident as there was no estimated difference in the baseline tinnitus severity between the groups (p = 0.92). The demographic profiles of the groups were similar in the terms of variables, such as gender and age. The participants and investigators could not be blinded to the group allocation due to the nature of the intervention. To minimize bias, the participants were informed when the intervention would commence but not explicitly to which group they were assigned.

Processes Involved in Intervention Delivery

Dose Delivered

The intervention materials were released weekly over and 8 week period. Each week, the participants received 2–3 modules, a practice diary, and videos of the techniques as shown in Table 4. The dose was delivered as planned and according to the protocol. It outlines that a comprehensive intervention was delivered consisting of 22 modules and a variety of other elements. This included videos in most of the modules. In addition, the participants received weekly guidance in the form of messages to provide feedback on the work done and to try to encourage the participants who were not engaging during the intervention.

TABLE 4

Table 4. Dose delivered and received for 118 participants undertaking the internet-based cognitive behavioral therapy (ICBT) intervention.

Dose Received

Only 54% of the participants were able to complete the 8 week CBT course as 10 withdrew and 38 never accessed the intervention materials. Table 4 shows the extent to which participants actively engaged and interacted with the resources provided. Table 5 shows how many users opened each module. There was a steady decline from 104 opening the initial module to 30 doing the final module. The engagement for the optional modules was also low ranging between 55 and 19 openings of each module. The number of worksheets completed was reduced from 86 for the initial worksheet to 14 for the later worksheets.

TABLE 5

Table 5. The engagement and satisfaction with the intervention.

To identify whether the engagement was related to satisfaction with the modules, the satisfaction for each module is shown in Table 5. The overall ratings were high for being able to understand the modules at 9.4/10 (SD: 1.2), the usefulness of the information 9.1/10 (SD: 1.6), and applicability of the information was 9.0/10 (SD: 1.6). These ratings are high, indicating those that did the modules found them helpful and usable as indicated by the example of the open-ended responses about what they gained from the information.

Processes Involved in the Outcomes Obtained

Adherence

Overall, the compliance for completing the outcome measures was low as shown in Figure 1. The completion rates at T1 were 72%. This decreased to 57% at T2, 42% at T3, and 35% at T4.

Primary Outcome Results

The main outcome was a reduction in tinnitus distress. This was achieved as indicated by an effect size of d = 0.46 (CI: 0.14–0.77) after the experimental group received the treatment (23). After the control group received the treatment, their tinnitus severity reduced. These improvements were maintained during the 2 and 12 month follow-up periods. These results were clinically significant for 51% of the participants from both the groups after completing the intervention (n = 75/148) indicating that their tinnitus severity reduced by more than 22.74 points. Hence, although the engagement was not optimal, the improvements in tinnitus distress were evident.

Secondary Outcome Results

Furthermore, the intervention led to the experimental group having a significantly greater reduction in insomnia, negative tinnitus cognitions, and hearing disability. Significant differences were not found for anxiety, depression, and quality of life, although the reductions were maintained during the follow-up periods (23).

The study reported minimal or no adverse effects. During the intervention period, only 1 (0.6%) participant had an increase of more than 10 points on the THI-S questionnaire. On finding out more, this was related to a particularly stressful deadline for work under difficult circumstances during the COVID-19 pandemic. There was only 1 (0.6%) participant who reported an adverse effect on the outcome questionnaire, explaining that initially, their tinnitus was more bothersome due to all the focus on tinnitus at the start of the intervention. There were no serious adverse events such as privacy breaches or major technical problems.

The involvement of a data monitoring committee added transparency and accountability to the results. The quarterly reports were prepared for the committee to monitor the enrollment, recruitment, results, adverse effects, and trial running.

Processes Involved in the Trial Implementation

Implementation Fidelity

Various protocols were set up before commencing and a pilot trial was initially run to identify the shortcomings to aid the effective implementation of the clinical trial (21, 22). The materials were adapted to ensure they were accessible without high linguistic demands (20) and the platform was functionally acceptable (19) before running the clinical trial. The intervention was delivered between the end of March 2020 and July 2020. This was during the peak of the first wave of the COVID-19 pandemic. The intervention ran as planned, although it was started 2 weeks earlier than planned when it became apparent that the pandemic was causing disruptions to everyday life.

A questionnaire was administered to try to determine the effect of COVID-19 on the study. Only a few responses were received. Of those responses, 5/43 (12%) said that they had had the COVID-19 virus. Of those answering, 12/43 (28%) reported that the situation was affecting their tinnitus. The reasons provided included increased anxiety, stress, being more depressed, and social isolation. Almost half (21/43; 49%) indicated that the pandemic had negatively impacted their emotional state and 12/ 43 (28%) felt lonely due to the social restrictions. During the intervention, some people became ill and could not complete the program. Others were given additional time as they did not have enough energy to complete the program after recovering. Thus, the pandemic did influence the intervention fidelity for some participants.

Barriers to Implementation

To identify the barriers to intervention usage, an intervention satisfaction questionnaire (39) was completed to identify how satisfied participants were with the intervention. The mean overall score for the satisfaction questionnaire was 46/75 (61% satisfaction) which was lower than expected due to higher satisfaction during the feasibility and pilot phase (19–21). To further investigate this, the ratings for the individual questions were investigated as shown in Figure 2. The highest rating was for the readability of the materials that the navigation was clear and it was straightforward to use. The lowest ratings were for having the motivation to complete the program, the worksheets, and how interesting the information was. These intervention aspects were the barriers to the intervention engagement.

FIGURE 2

Figure 2. The ratings of different aspects of the intervention on a scale of 1–5. The error bars represent SEM.

The open-ended questions were furthermore analyzed to identify the additional barriers to implementation. These included both the personal and intervention factors as shown in Table 6. The personal factors identified were time barriers, the impact of the COVID-19 pandemic, lack of self-discipline, and other health problems. The intervention factors making completion difficult included the length and number of tasks on the intervention, that tinnitus was heightened due to the focus on tinnitus during the intervention, and that some people sought a cure and not strategies to help them cope with the tinnitus.

TABLE 6

Table 6. The barriers to the intervention implementation.

Facilitation of Effectiveness

Prior to starting the intervention, the participants were asked to commit to the intervention and indicate the level of commitment on a 1–10 points scale. Those committing indicated this by a score of 10. Some found they were not able to commit as intended although this commitment motivated others as indicated by the statements, such as “I have a family and full-time job. I couldn't keep up at some point. But I made a huge commitment to the first modules (because I felt the improvement on my tinnitus) so I did my sessions every day.” This commitment by some contributed to them noticing the improvements in their tinnitus.

The open-ended responses were analyzed to identify the facilitators of effectiveness. The facilitators identified were that the intervention was empowering, accessible, well-structured, and they were adequately supported while undertaking it, as shown in Table 7.

TABLE 7

Table 7. The facilitators of effectiveness to the intervention.

Discussion

This process evaluation was undertaken to determine which aspects of the implementation of a clinical trial delivering ICBT to the population of the United States hampered and facilitated the outcomes obtained. The process explored included the enrollment of participants, the intervention delivery, the outcomes obtained, and trial implementation as explored in this discussion.

Processes Involved in the Enrollment

A lot of preparation and planning was involved to ensure a range of recruitment strategies was incorporated. As this was the first tinnitus ICBT trial with Spanish participants, much research was done to investigate how to improve reaching this population [e.g., (48–52)]. The Spanish speakers were furthermore involved in the research team and during the intervention adaptation (20). Despite costly and varied recruitment strategies, it was very difficult to recruit Spanish speakers for this trial. A subsequent pilot trial with a wider recruitment area indicated that there is interest from the Spanish speakers, but ways of reaching and encouraging them to participate are still difficult (22). Moreover, although the trial targeted different ethnic and racial groups, this was not achieved and the strategies to reach a greater variety of ethnic and racial groups need to be sought.

This process evaluation highlighted various factors that could help with future trials to aid recruitment and enrollment. More hands-on involvement from a public patient group involving the individuals with bothersome tinnitus would be helpful to reach those with tinnitus (53). Such a group would advise on the strategies that the research team may not consider. Although the group members were involved in generating the recruitment ideas, directly contacting those with tinnitus at support groups, more involvement in future trials is encouraged. Hearing about the intervention effects from those with tinnitus may carry more weight than the professional contact. It was identified that a better understanding of the current public views on tinnitus and tinnitus interventions is required. Having a clear picture of what is being said in social media, public statements, on websites, on social media, blogs, and forums, advertising, policy documents, or reports provides a starting point regarding what perceptions need to be managed. Many people with tinnitus desire a treatment to completely cure tinnitus (54). Although explicitly stated that this intervention involved tinnitus management, some people still expected a cure and hence were disappointed.

It was evident that careful thought needs to be given to the inclusion criteria in the clinical trials. Excluding those with mild depression made the screening process very complicated as a psychologist had to be involved in the trial and screening process which increase the resources required. The subsequent trials indicated that including the participants with depression did not hamper the trial outcomes and their tinnitus severity decreased more than those without significant depression (23). Narrowing the recruitment to only the State of Texas was a further barrier. As this was an internet trial, using a wider pool across the country may be more helpful to reach the targeted numbers. The participants reached were those with higher socioeconomic status due to the higher levels of education. This may reflect the recruitment strategies used. An alternative way of reaching the different socio-economic groups needs to be sought which is likely to involve the alternative treatment approaches, such as less intense versions of this intervention.

Processes Involved in the Allocation

The aim of the screening process prior to the participant allocation was to ensure that those involved were suitable for the trial, motivated to complete the intervention, and committed to completing the outcome measures for the trial. Although the participants confirmed this in the online and telephone screening, many never started the intervention. A clear need was identified to have better means of identifying who may be more engaged and motivated to do the intervention. To try to identify if this intervention is more suitable for certain tinnitus subgroups, a further trial was undertaken, dividing the participants into subgroups based on the level of their tinnitus severity (23). This indicated that the effectiveness of the intervention increased with the greater initial levels of tinnitus distress a baseline. The reductions in tinnitus distress were greater for those with significant levels of depression at the bassline. Rodrigo et al. (55) identified that the greater baseline tinnitus severity and those with greater educational levels were more likely to have a greater reduction in tinnitus distress after undertaking an ICBT intervention. The participants in this current trial represented those with higher levels of education as the majority had a university degree, college, or vocational training. When subgrouping those with tinnitus, Beukes et al. (56) suggested that the unique management pathways may be more suited for some tinnitus subgroups. Further work is required to identify which individuals with tinnitus are more suited for ICBT.

Processes Involved in Intervention Delivery

All the participants who were assigned to the treatment were provided with access to the treatment program. However, several did not take the opportunity to engage with the material as 10 participants withdrew, and 30 participants never logged into the platform to access the intervention. Although attrition is similar to that of prior ICBT studies [mean of 14% (57)], engagement is lower than that previously reported in the trials in the United Kingdom [e.g., (16, 17)].

In addition, the initial modules were opened more than the final modules. The worksheet completion decreased during the later weeks of the intervention. Some participants indicated that they thought the intervention was too long which could be a contributing factor. Other participants found it helpful to have a comprehensive intervention. The intervention length and range of materials may, however, be a barrier for some. For those reading the modules, they rated the intervention highly in the terms of usefulness, applicability, and being able to understand the modules. The intervention dose was similar in the terms of guidance and delivery. The US intervention, however, had one additional module and more worksheets. Despite modifying it for ease of reading, the modules were opened by fewer participants compared with the participants of the United Kingdom who opened 74% of the recommended modules and 50% of the optional modules (34). When comparing these results with engagement by the population of the United Kingdom (16), stark differences are found. This earlier clinical trial indicated that the participants logged into the program on average 27 times compared with 8 times for the participants of the United States (23).

Processes Involved in the Outcomes Obtained

Undertaking ICBT led to a significant reduction in tinnitus distress which was the primary aim of the intervention. The overall reduction with an effect size of d = 0.46 (CI: 0.14–0.77) was slightly lower than that compared with the pooled result of previous European ICBT trials of d = 0.50 (CI: 0.37–0.63) in the recent systematic review (57). These studies found a medium effect for ICBT reducing insomnia and a small effect for reducing anxiety and depression. The present study results varied as significant reductions being evident for the secondary outcomes for insomnia, tinnitus cognitions, and hearing disability but not for anxiety and depression. This may be related to those with significant levels of depression being excluded.

The compliance for completing the outcome measures was low, with 72% completion at the first time point, and dropping to 35% at the 1 year follow-up. This is lower than the previous ICBT in Europe, for example, the completing levels of 92 and 78% at post-intervention and 2 month follow-up for the participants from the United Kingdom (18). It may indicate that the population of the United States has other intervention needs or require additional motivation or incentives to complete the outcome measures. Satisfaction was lowest for motivation to complete, doing the worksheets, and how interesting the information was. Interestingly, the participants from the United Kingdom also rated these aspects the lowest (16, 34). The ways of increasing the motivation to do the intervention and worksheets are required. Overall satisfaction was lower than the ratings from the population of the United Kingdom where the majority of the scores were above 3/5 (16, 34). This may indicate the cultural differences or expectations from the interventions that may differ.

The facilitators identified were that the intervention was empowering, accessible, well-structured, and they were adequately supported while undertaking. Those thus undertaking the intervention found it very helpful and ways of getting more people to undertake the intervention are required.

Processes Involved in the Trial Implementation

Adequate trial preparations were undertaken, such as assessing the intervention materials (20), the functionality of the platform (19), and doing a pilot study before commencing (21, 22). Although the implementation fidelity was high, the trial was run during the first wave of the COVID-19 pandemic. Due to the intervention being online the trial could, however, continue. It was apparent that some of the participants were unwell with COVID-19 and thus unable to engage as planned. Even after recovering, they found it difficult to do the program due to less energy. They were given more time to complete the program which impacted the intervention fidelity. Both the COVID-19 pandemic and virus have been shown to impact the tinnitus severity for some individuals (58). A subsection of participants (12%) in this study indicated that they had COVID-19 and 49% reported that the pandemic had negatively impacted their emotional state. It is likely that the pandemic and COVID-19 had a negative impact on the engagement in the intervention, but the extent of the impact is difficult to untangle. The participants in the control group had a weekly questionnaire to complete during the active intervention period without receiving the intervention. Some participants expressed a dislike of these questionnaires which may have impacted their subsequent engagement in the trial.

This process evaluation provides an opportunity for the participants to highlight the factors that made undertaking the intervention difficult. Personal parries, such as time barriers, the impact of the COVID-19 pandemic, lack of self-discipline, and other health problems were identified as barriers. The ways of increasing support to do the intervention should be sought. One idea may be involving significant others in the intervention process (59). This support may be motivational and help the intervention seem less burdensome. The intervention factors making the completion difficult included the length and number of tasks on the intervention, that tinnitus was heightened due to the focus on tinnitus during the intervention, and that some people sought a cure and not strategies to help them cope with the tinnitus. Such barriers can be reduced by ensuring the potential participants have a good understanding of exactly what the intervention entails. Modifying the intervention to ensure it is less time consuming but still comprehensive is required. The facilitators to the intervention's effectiveness were that it is empowering, accessible, and well-structured. The participants greatly valued the support they received from the guidance provided.

Study Limitations and Future Directions

This evaluation was based on the barriers and facilitators identified by the participants completing the outcome measures. Although those not engaging were contacted by email, text, and phone, it was not always possible to reach them. This process evaluation would have benefited from including the views of those who did not engage or complete the outcome measures to truly reflect the barriers to participation. More effective ways of measuring engagement are required. Although it is possible to see if someone has opened a module, it is not possible to determine how much they have read, or how long they spent on the chapter. The outcomes measures used were all based on the clinical outcomes. For tinnitus, there may be more important or relevant outcomes not included that could have provided more insights. Future studies should investigate these, such as the intervention effectiveness on participation in the activities, impact on work, and relationships.

Conclusions

This process evaluation has provided a broader understanding of the factors affecting recruitment and the research context. The impact of factors, such as social and family support should be considered (24–26). The aspects that contributed to the effectiveness of the intervention, such as the participants finding it empowering, accessible, and well-structured were identified. The barriers restricting engagement, such as the intervention length, time limitations, and low self-discipline levels need addressing. The results of this process evaluation should be implemented into further clinical trials to improve the reach, engagement, and outcomes obtained.

Data Availability Statement

The datasets presented in this study can be found in online repositories. The names of the repository/repositories and accession number(s) can be found at: http://doi.org/10.6084/m9.figshare.13646012.

Ethics Statement

Ethical approval was obtained from the Institutional Review Board at Lamar University, Beaumont, Texas, US (IRB-FY17-209). The patients/participants provided their written informed consent online to participate in this study.

Author Contributions

The study was conceived by VM, GA, and EB. The study platform was provided by GA. The data collection and analysis were done by EB. EB drafted the manuscript. All the authors critically analyzed the full manuscript and approved the final version.

Funding

This work was funded by the National Institute on Deafness and Communication Disorders (NIDCD) of the National Institute of Health (NIH) under the award number R21DC017214.

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher's Note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Acknowledgments

A special thank are extended to all the participants who participated in the Tackling Tinnitus study.

References

1. McCormack A, Edmondson-Jones M, Somerset S, Hall D. A systematic review of the reporting of tinnitus prevalence and severity. Hear Res. (2016) 337:70–9. doi: 10.1016/j.heares.2016.05.009

PubMed Abstract | CrossRef Full Text | Google Scholar

2. Trevis KJ, McLachlan NM, Wilson SJ. A systematic review and meta-analysis of psychological functioning in chronic tinnitus. Clin Psychol Rev. (2018) 60:62–86. doi: 10.1016/j.cpr.2017.12.006

PubMed Abstract | CrossRef Full Text | Google Scholar

3. Salazar JW, Meisel K, Smith ER, Quiggle A, McCoy DB, Amans MR. Depression in patients with tinnitus: a systematic review. Otolaryngol Head Neck Surg. (2019) 161:28–35. doi: 10.1177/0194599819835178

PubMed Abstract | CrossRef Full Text | Google Scholar

4. Zenner HP, Delb W, Kröner-Herwig B, Jäger B, Peroz I, Hesse G, et al. A multidisciplinary systematic review of the treatment for chronic idiopathic tinnitus. Eur Arch Otorhinol. (2017) 274:2079–91. doi: 10.1007/s00405-016-4401-y

PubMed Abstract | CrossRef Full Text | Google Scholar

5. Cima RF, Mazurek B, Haider H, Kikidis D, Lapira A, Noreña A, et al. A multidisciplinary European guideline for tinnitus: diagnostics, assessment, and treatment. HNO. (2019) 67:10–42. doi: 10.1007/s00106-019-0633-7

PubMed Abstract | CrossRef Full Text | Google Scholar

6. Fuller TE, Haider HF, Kikidis D, Lapira A, Mazurek B, Norena A, et al. Different teams, same conclusions? A systematic review of existing clinical guidelines for the assessment and treatment of tinnitus in adults. Front Psychol. (2017) 8:206. doi: 10.3389/fpsyg.2017.00206

PubMed Abstract | CrossRef Full Text | Google Scholar

7. Tunkel DE, Bauer CA, Sun GH, Rosenfeld RM, Chandrasekhar SS, Cunningham ER Jr, et al. Clinical practice guideline: tinnitus. JAMA Otolaryngol Head Neck Surg. (2014) 15:S1–40. doi: 10.1177/0194599814545325

PubMed Abstract | CrossRef Full Text | Google Scholar

8. Fuller TE, Cima R, Langguth B, Mazurek B, Vlaeyen JW, Hoare DJ. Cognitive behavioural therapy for tinnitus. Cochrane Database Syst Rev. (2020) 1:CD012614. doi: 10.1002/14651858.CD012614.pub2

PubMed Abstract | CrossRef Full Text | Google Scholar

9. Landry EC, Sandoval XC, Simeone CN, Tidball G, Lea J, Westerberg BD. Systematic review and network meta-analysis of cognitive and/or behavioral therapies (CBT) for tinnitus. Otol Neurotol. (2020) 41:153–66. doi: 10.1097/MAO.0000000000002472

PubMed Abstract | CrossRef Full Text | Google Scholar

10. Andersson G. Psychological aspects of tinnitus and the application of cognitive–behavioral therapy. Clin Psychol Rev. (2002) 22:977–90. doi: 10.1016/S0272-7358(01)00124-6

PubMed Abstract | CrossRef Full Text | Google Scholar

11. Andersson G, Strömgren T, Ström L, Lyttkens L. Randomized controlled trial of internet-based cognitive behavior therapy for distress associated with tinnitus. Psychosom Med. (2002) 64:810–6. doi: 10.1097/00006842-200209000-00014

PubMed Abstract | CrossRef Full Text | Google Scholar

12. Jasper K, Weise C, Conrad I, Andersson G, Hiller W, Kleinstaeuber M. Internet-based guided self-help versus group cognitive behavioural therapy for chronic tinnitus: a randomized controlled trial. Psychother Psychosom. (2014) 83:234–46. doi: 10.1159/000360705

PubMed Abstract | CrossRef Full Text | Google Scholar

13. Abbott JA, Kaldo V, Klein B, Austin D, Hamilton C, Piterman L, et al. A cluster randomized trial of an internet-based intervention program for tinnitus distress in an industrial setting. Cogn Behav Ther. (2009) 38:162–73. doi: 10.1080/16506070902763174

PubMed Abstract | CrossRef Full Text | Google Scholar

14. Beukes EW, Andersson G, Manchaiah V, Kaldo V. Cognitive Behavioral Therapy for Tinnitus. San Diego, CA: Plural Publishing (2021).

Google Scholar

15. Beukes EW, Allen PM, Manchaiah V, Baguley DM, Andersson G. Internet-based intervention for tinnitus: outcome of a single-group open trial. J AM Acad Audiol. (2017) 28:340–51. doi: 10.3766/jaaa.16055

PubMed Abstract | CrossRef Full Text | Google Scholar

16. Beukes EW, Baguley DM, Allen PM, Manchaiah V, Andersson G. Audiologist-guided Internet-based cognitive behaviour therapy for adults with tinnitus in the United Kingdom: a randomized controlled trial. Ear Hear. (2018) 39:423–33. doi: 10.1097/AUD.0000000000000505

PubMed Abstract | CrossRef Full Text | Google Scholar

17. Beukes EW, Andersson G, Allen PM, Manchaiah V, Baguley DM. Effectiveness of guided internet-based cognitive behavioural therapy vs face-to-face clinical care for treatment of tinnitus: a randomized clinical trial. JAMA Otolaryngol Head Neck Surg. (2018) 144:1126–33. doi: 10.1001/jamaoto.2018.2238

PubMed Abstract | CrossRef Full Text | Google Scholar

18. Beukes EW, Allen PM, Baguley DM, Manchaiah V, Andersson G. Long-term efficacy of audiologist-guided Internet-based cognitive behaviour therapy for tinnitus. Am J Audiol. (2018) 27:431–47. doi: 10.1044/2018_AJA-IMIA3-18-0004

PubMed Abstract | CrossRef Full Text | Google Scholar

19. Manchaiah V, Vlaescu G, Varadaraj S, Aronson EP, Fagelson MA, Munoz MF, et al. Features, functionality, and acceptability of internet-based cognitive behavioral therapy for tinnitus in the United States. Am J Audiol. (2020) 29:476–90. doi: 10.1044/2020_AJA-20-00002

PubMed Abstract | CrossRef Full Text | Google Scholar

20. Beukes EW, Fagelson M, Aronson EP, Munoz MF, Andersson G, Manchaiah V. Readability following cultural and linguistic adaptations of an Internet-based intervention for tinnitus for use in the United States. Am J Audiol. (2020) 29:97–109. doi: 10.1044/2019_AJA-19-00014

PubMed Abstract | CrossRef Full Text | Google Scholar

21. Beukes EW, Andersson G, Fagelson M, Manchaiah V. Audiologist-supported Internet-based cognitive behavioural therapy for tinnitus in the United States: a pilot trial. Am J Audiol. (2021) 30: 717–29. doi: 10.1044/2021_AJA-20-00222

PubMed Abstract | CrossRef Full Text | Google Scholar

22. Beukes EW, Munoz MF, Andersson G, Manchaiah V. Internet-based cognitive behavioral therapy for tinnitus in Spanish: a global feasibility trial. Int J Audiol. (2021). doi: 10.1080/14992027.2021.1971780

PubMed Abstract | CrossRef Full Text | Google Scholar

23. Beukes E, Andersson G, Fagelson M, Manchaiah V. Internet-based audiologist-guided cognitive behavioral therapy for tinnitus in the United States: A randomized controlled trial. J Med Int Res. (2021)

PubMed Abstract | Google Scholar

24. Guilaran J, de Terte I, Kaniasty K, Stephens C. Psychological outcomes in disaster responders: a systematic review and meta-analysis on the effect of social support. Inter J Disaster Risk Sci. (2018) 9:344–58. doi: 10.1007/s13753-018-0184-7

CrossRef Full Text | Google Scholar

25. Hou D, Chen YX, He D, Zhang X, Fan X. The relationship between caregiver reactions and psychological distress in family caregivers of patients with heart failure. J Cardiovasc Nurs. (2020) 35:234–42. doi: 10.1097/JCN.0000000000000636

PubMed Abstract | CrossRef Full Text | Google Scholar

26. Wang J, Mann F, Lloyd-Evans B, Ma R, Johnson S. Associations between loneliness and perceived social support and outcomes of mental health problems: a systematic review. BMC Psychiatry. (2018) 18:156. doi: 10.1186/s12888-018-1736-5

PubMed Abstract | CrossRef Full Text | Google Scholar

27. Glasgow RE, Vogt TM, Boles SM. Evaluating the public health impact of health promotion interventions: the RE-AIM framework. Am J Public Health. (1999) 89:1322–7. doi: 10.2105/AJPH.89.9.1322

PubMed Abstract | CrossRef Full Text | Google Scholar

28. Dzewaltowski DA, Glasgow RE, Kelsges LM, Estabrooks PA, Brock E. RE-AIM: evidence-based standards and a Web resource to improve translation of research into practice. Ann Behav Med. (2004) 28:75–80. doi: 10.1207/s15324796abm2802_1

PubMed Abstract | CrossRef Full Text | Google Scholar

29. Baranowski T, Stables G. Process evaluations of the 5-a-day projects. Health Educ Behav. (2000) 27:157–66. doi: 10.1177/109019810002700202

PubMed Abstract | CrossRef Full Text | Google Scholar

30. Linnan L, Steckler A., editors. Process Evaluation for Public Health Interventions and Research. San Francisco, CA: Jossey-Bass/Wiley (2002).

Google Scholar

31. Gussenhoven AH, Singh AS, Goverts ST, van Til M, Anema JR, Kramer SE. A process evaluation of implementing a vocational enablement protocol for employees with hearing difficulties in clinical practice. Int J Audiol. (2015) 54:507–17. doi: 10.3109/14992027.2015.1009642

PubMed Abstract | CrossRef Full Text | Google Scholar

32. Meijerink JFJ, Pronk M, Kramer SE. Experiences with and lessons learned from developing, implementing, and evaluating a support program for older hearing aid users and their communication partners in the hearing aid dispensing setting. Am J Audiol. (2020) 29:638–47. doi: 10.1044/2020_AJA-19-00072

PubMed Abstract | CrossRef Full Text | Google Scholar

33. Ratanjee-Vanmali H, Swanepoel DW, Laplante-Lévesque A. Patient uptake, experience, and satisfaction using web-based and face-to-face hearing health services: process evaluation study. J Med Internet Res. (2020) 22:e15875. doi: 10.2196/15875

PubMed Abstract | CrossRef Full Text | Google Scholar

34. Beukes EW, Manchaiah V, Baguley DM, Allen PM, Andersson G. Process evaluation of Internet-based cognitive behavioural therapy for adults with tinnitus in the context of a randomised control trial. Int J Audiol. (2018) 57:98–09. doi: 10.1080/14992027.2017.1384858

PubMed Abstract | CrossRef Full Text | Google Scholar

35. Eysenbach G, Consort-EHEALTH Group. CONSORT-EHEALTH: improving and standardizing evaluation reports of Web-based and mobile health interventions. J Med Internet Res. (2011) 13:e126. doi: 10.2196/jmir.1923

PubMed Abstract | CrossRef Full Text | Google Scholar

36. Meikle MB, Henry JA, Griest SE, Stewart BJ, Abrams HB, McArdle R, et al. The tinnitus functional index: development of a new clinical measure for chronic, intrusive tinnitus. Ear Hear. (2012) 33:153–76. doi: 10.1097/AUD.0b013e31822f67c0

PubMed Abstract | CrossRef Full Text | Google Scholar

37. Spitzer RL, Kroenke K, Williams JB. Validation and utility of a self-report version of PRIME-MD: the PHQ primary care study. JAMA. (1999) 282:1737–44. doi: 10.1001/jama.282.18.1737

PubMed Abstract | CrossRef Full Text | Google Scholar

38. Andersson G, Kaldo V. Internet-based cognitive behavioral therapy for tinnitus. J Clin Psychol. (2004) 60:171–8. doi: 10.1002/jclp.10243

PubMed Abstract | CrossRef Full Text | Google Scholar

39. Beukes EW, Vlaescu G., Manchaiah V, Baguley, DM, Allen PM, et al. Development and technical functionality of an Internet-based intervention for tinnitus in the UK. Internet Interv. (2016) 6:6–15. doi: 10.1016/j.invent.2016.08.002

PubMed Abstract | CrossRef Full Text | Google Scholar

40. Beukes E, Andersson G, Fagelson M, Manchaiah V. Dismantling internet based cognitive behavioral therapy for tinnitus. The contribution of applied relaxation: a randomized controlled trial. Internet Interv. (2021) 25:100402. doi: 10.1016/j.invent.2021.100402

PubMed Abstract | CrossRef Full Text | Google Scholar

41. Spitzer RL, Kroenke K, Williams JB, Löwe B. A brief measure for assessing generalized anxiety disorder: the GAD-7. Arch Intern Med. (2006) 166:1092–7. doi: 10.1001/archinte.166.10.1092

PubMed Abstract | CrossRef Full Text | Google Scholar

42. Bastien CH, Vallières A, Morin CM. Validation of the insomnia severity index as an outcome measure for insomnia research. Sleep Med. (2001) 2:297–307. doi: 10.1016/S1389-9457(00)00065-4

PubMed Abstract | CrossRef Full Text | Google Scholar

43. Wilson P, Henry J. Tinnitus cognitions questionnaire: development and psychometric properties of a measure of dysfunctional cognitions associated with tinnitus. Int Tinnitus J. (1998) 4:23–30.

PubMed Abstract | Google Scholar

44. Herdman M, Gudex C, Lloyd A, Janssen M, Kind P, Parkin D, et al. Development and preliminary testing of the new five-level version of EQ-5D (EQ-5D-5L). Qual Life Res. (2011) 22:1727–36. doi: 10.1007/s11136-011-9903-x

PubMed Abstract | CrossRef Full Text | Google Scholar

45. Henry J, Griest S, Zaugg TL, Thielman E, Kaelin C, Galvez G, et al. Tinnitus and hearing survey: a screening tool to differentiate bothersome tinnitus from hearing difficulties. Am J Audiol. (2015) 24:66–77. doi: 10.1044/2014_AJA-14-0042

PubMed Abstract | CrossRef Full Text | Google Scholar

46. IBM Corp. IBM SPSS Statistics for Windows, Version 26.0. Armonk, NY: IBM Corp (2019).

Google Scholar

47. Graneheim UH, Lundman B. Qualitative content analysis in nursing research: concepts, procedures and measures to achieve trustworthiness. Nurse Educ Today. (2004) 24:105–12. doi: 10.1016/j.nedt.2003.10.001

PubMed Abstract | CrossRef Full Text | Google Scholar

48. Barrera AZ, Kelman AR, Muñoz RF. Keywords to recruit Spanish-and English-speaking participants: evidence from an online postpartum depression randomized controlled trial. J Med Internet Res. (2014) 16:e6. doi: 10.2196/jmir.2999

PubMed Abstract | CrossRef Full Text | Google Scholar

49. Bunge EL, Taylor LA, Bond M, Stephens TN, Nishimuta K, Barrera AZ, et al. Facebook for recruiting Spanish- and English-speaking smokers. Internet Interv. (2019) 17:100238. doi: 10.1016/j.invent.2019.02.002

PubMed Abstract | CrossRef Full Text | Google Scholar

50. García AA, Zuñiga JA, Lagon C. A personal touch: the most important strategy for recruiting Latino research participants. J Transcult Nurs. (2017) 28:342–7. doi: 10.1177/1043659616644958

PubMed Abstract | CrossRef Full Text | Google Scholar

51. Hammer CS. Strategies for recruiting African American and Hispanic women as participants in research. Contemp Issues Commun Sci Dis. (2000) 27:127–34. doi: 10.1044/cicsd_27_F_127

CrossRef Full Text | Google Scholar

52. Sha M, McAvinchey G, Quiroz R, Moncada J. Successful techniques to recruit Hispanic and Latino research participants. Survey Pract. (2017) 10:1–9. doi: 10.29115/SP-2017-0014

CrossRef Full Text | Google Scholar

53. Grier S, Evans D, Gibson A, Chin TL, Stoddart M, Kok M, et al. Finding and engaging patients and the public to work collaboratively on an acute infection microbiology research public panel. Res Involv Engagem. (2018) 4:1–9. doi: 10.1186/s40900-018-0083-5

PubMed Abstract | CrossRef Full Text | Google Scholar

54. Beukes EW, Onozuka J, Brazell TP, Manchaiah V. Coping with tinnitus during the COVID-19 pandemic. Am J Audiol. (2021) 30:385–93. doi: 10.1044/2021_AJA-20-00188

PubMed Abstract | CrossRef Full Text | Google Scholar

55. Rodrigo H, Beukes EW, Andersson G, Manchaiah V. Internet-based cognitive-behavioural therapy for tinnitus: secondary analysis to examine predictors of outcomes. BMJ Open. (2021) 11:e049384. doi: 10.1136/bmjopen-2021-049384

PubMed Abstract | CrossRef Full Text | Google Scholar

56. Beukes EW, Baguley DM, Manchaiah V, Andersson G, Allen PM, Kaldo V, et al. Investigating tinnitus subgroups based on hearing-related difficulties. Int J Clin Pract. (2021) 75:e14684. doi: 10.1111/ijcp.14684

PubMed Abstract | CrossRef Full Text | Google Scholar

57. Beukes EW, Manchaiah V, Allen PM, Baguley DM, Andersson G. Internet-based interventions for adults with hearing loss, tinnitus, and vestibular disorders: a systematic review and meta-analysis. Trends Hear. (2019) 23:233121651985174. doi: 10.1177/2331216519851749

PubMed Abstract | CrossRef Full Text | Google Scholar

58. Beukes E, Ulep AJ, Eubank T, Manchaiah V. The Impact of COVID-19 and the pandemic on tinnitus: a systematic review. J Clin Med. (2021) 10:2763. doi: 10.3390/jcm10132763

PubMed Abstract | CrossRef Full Text | Google Scholar

59. Beukes EW, Andersson G, Manchaiah V. Third party disability for significant others of individuals with tinnitus: A cross-sectional survey design. Authorea. (2021). doi: 10.22541/au.163045338.85276524/v1

CrossRef Full Text | Google Scholar