Abstract
Background:
This pilot study aims to test the usability of the iACTwithPain platform, an online ACT-based intervention for people with chronic pain, to obtain information on which intervention and usability aspects need improvement and on expected retention rates.
Methods:
Seventy-three Portuguese women with chronic pain were invited to complete the first three sessions of the iACTwithPain intervention assess their quality, usefulness and the platform’s usability. Twenty-one accepted the invitation. Additionally, eight healthcare professionals working with chronic medical conditions assessed the platform and the intervention from a practitioner’s point of view.
Results:
This study presented a considerable attrition rate (71.43%) among chronic pain participants, with six completers. There were no significant differences in demographic or clinical variables between dropouts and completers except for completed education (participants who dropped out presented less education than completers). Reasons for dropout were related to difficult personal events occurring during the time of the intervention, lack of time, or having forgotten. There seemed to be an overall satisfaction with both the intervention, its contents and form of presentation of information, and the platform, concerning its design, appearance, and usability. Real image videos were preferred over animations or audio by chronic pain participants. Healthcare professionals emphasized the appealing and dynamic aspects of the animation format.
Conclusion:
This study informs the ongoing improvement of the iACTwithPain platform and provides valuable information on aspects researchers should consider while developing online psychological interventions for chronic pain. Further implications are discussed.
Introduction
Chronic pain can be defined as persistent pain lasting more than 3 months, has a prevalence of approximately 20% in adults (), can have a detrimental impact in mobility and quality of life (), and productivity (). Chronic pain is associated with high economic burden, and is considered as one of the most expensive long-term health conditions in industrialized countries (; ; ).
Acceptance of pain is considered an important factor for a successful adaptation to chronic pain, linked to less depression, pain-related anxiety, and disability (). These findings have supported the application of acceptance-based therapies, such as Acceptance and Commitment Therapy [ACT; ], to this population. ACT is an empirically validated psychological approach for chronic pain () that promotes acceptance and engagement with values-guided behavior despite chronic pain symptoms (). The efficacy of ACT for chronic pain has been demonstrated in a meta-analysis by , and in a broader review including both acceptance and mindfulness-based interventions (). At the same time, the pertinence of promoting self-compassion [i.e., the ability to be sensitive to personal suffering and being motivated to kindly alleviate it; (; )] in chronic pain has been highlighted due to its protective role against depressive symptomatology () and the positive effects compassion-focused interventions have presented in this population (). The combination of ACT and compassion for chronic pain was recently implemented in a pilot test with promising results ().
In recent years, online-delivered interventions have generated increased interest due to their accessibility, flexibility and cost-effectiveness (). In particular, online-based ACT has been proved to be efficacious for chronic pain in a recent meta-analysis. Online-ACT was greater than control conditions in reducing pain interference, pain intensity, depression, and anxiety, and in increasing mindfulness, and psychological flexibility (). For these reasons, and additionally considering the advantages of online interventions to improve health-related outcomes (), and for chronic pain in particular (), the iACTwithPain intervention platform was developed by the authors of this paper (iACTwithPain research team). iACTwithPain is an ACT- and Compassion-based intervention tailored explicitly to chronic pain. It comprises eight sessions to be completed on an online platform throughout 8 weeks (1 session per week). The efficacy of the iACTwithPain intervention in improving chronic pain impact and related health and quality of life markers will be tested in full in a randomized controlled trial (). It is the aim of the current study to pilot test the usability of the platform by examining the feedback of clinicians and chronic pain patients of first three sessions in order to obtain information on which intervention and usability aspects need improvement, and on expected retention rates. Given our team’s combined expertise in psychology, design and engineering, we hypothesize qualitative feedback from participants reflecting pertinent content for patients with chronic pain, and an online platform that is intuitive, engaging and esthetically pleasing. We additionally expect high usability scores, as measured by the System Usability Scale [SUS; ], relating to participants’ use of the platform.
Methods
Ethical Approval and Data Safety
This study was approved by Ethical Committee of the Faculty of Psychology and Educational Sciences of the University of Coimbra (on 28/11/2019), and was conducted in accordance with the ethical standards in the 1964 Declaration of Helsinki and its later amendments. All collected data will be stored (for 5 years) using high standard security mechanisms, and thus ensuring confidentiality. Data will be in anonymized and can only be assessed by the research team.
Sample Size Calculation
According to , on average, a sample of five participants can detect 85% of the usability problems. Therefore, we aim to have a sample of at least 5 in each group (participants with chronic pain and health professionals). Assuming a conservative dropout rate of 83% () for the clinical sample, at least 30 participants should be invited to enroll in the study.
Procedures
The study’s chronological order was as follows: (1) participants recruitment (presentation of the study and informed consent); (2) during the following 2 weeks participants tested the iACTwithPain platform; (3) in the third week the usability and quality assessment questionnaire was administered; (4) participants who dropped out from the study were contacted to fill in a questionnaire (reasons for dropping out).
Seventy-three Portuguese women with chronic pain, enrolled in a different study who had demonstrated interest in taking part in the current one, were invited to complete the first three sessions of the iACTwithPain intervention and assess their quality and usefulness, as well as to assess the platform’s usability. Inclusion criteria were: age between 18 and 65 years; diagnosis of chronic pain; internet access; and proficiency in Portuguese. Exclusion criteria were: not providing informed consent; or pain due to malignancy, trauma, or surgery. Twenty-one accepted the invitation to participate, signed an informed consent, and were enrolled in the study. Participants with chronic pain that did not complete the three sessions were asked to fill out a questionnaire on the reasons for attrition.
Additionally, nine healthcare professionals working with chronic medical conditions (four psychologists, four physicians, and one nurse) were also invited to assess the platform and the intervention from a practitioner’s point of view. Of these, eight signed an informed consent before the start of the study and were enrolled in the study.
The iACTwithPain Platform
The iACTwithPain intervention was designed based on the psychological flexibility model (; ) and compassion-focused interventions applied to chronic pain (). The psychologists’ members of the iACTwithPain team have expertise in developing and efficacy testing ACT and compassion-based interventions for chronic conditions such as chronic pain (), cancer (), inflammatory bowel disease (), and psychiatric disorders (). Moreover, the team’s knowledge of the psychological impact of chronic pain and the underlying psychological processes (, , ) were taken into consideration. During iACTwithPain development, the principles of ACT and compassion-based interventions were strictly followed to ensure pertinent and rigorous therapeutic sessions. As described in the RCT protocol () treatment integrity guidelines for ACT () were followed and included: (a) training in ACT and compassion focused therapy models, ensuring that the therapists fully grasp the concepts and principles of the interventions and have previous competence/experience in their application; (b) the content of the sessions are ACT-consistent (for example focus on the function rather than the content); and the known processes of change of ACT and compassion were followed during the development of the intervention.
Participants will be randomly assigned to two experimental arms: and ACT-only intervention or an ACT and compassion-focused intervention. All sessions will be the same except for sessions five and six. The ACT and compassion-focused group will be exposed to compassion themes and exercises (e.g., what is compassion, developing compassion toward the self and others, obstacles to compassion). The ACT-only group will reinforce previous topics (e.g., willingness, acceptance, defusion, and observing self) without introducing new information or practices. The ACT intervention will include the following core themes: (a) Awareness of internal experiences (mindfulness; self-as-context); (b) Openness to experience difficult experiences (acceptance of pain: willingness toward thoughts, emotions and physical sensations; and cognitive defusion); (c) Engagement with valued action (values clarification and commitment to meaningful actions); (d) Self-Compassion (promoting self-kindness in times of difficulty; for the ACT and compassion-focused group only). Sessions will comprise video-animations (with videoscribes and therapists’ avatars), real-image videos, texts, and audios guiding meditative practices. iACTwithPain is a self-paced intervention and therefore the participant can decide when to login and complete each session. Nevertheless, the intervention was designed so that sessions are completed weekly (one session per week, over 8 weeks). Each new session will become available in the platform every Monday. Emails notifying the participants that a new session is available will be sent weekly.
Session 1 is completed immediately after session 0, and the following sessions are then completed once per week. A brief introductory session (session 0) will welcome participants to the intervention and introduce the platform. Participants are asked to practice between-session mindfulness and/or compassion-based meditative exercises as often as possible. In this usability study, participants were asked to complete the introductory session (session zero), session one and two. Session one focuses on psychoeducation about chronic pain, promotion of creative hopelessness, introduction to mindfulness practice, and mindfulness of breathing practice. Session two focuses on the usefulness of mindfulness to manage suffering, promotion of mindfulness practice, and body scan practice (see Table 1 for more detailed information). Mindfulness exercises throughout the intervention focus on a non-judgmental, open, and accepting attitude toward the present moment. Visual examples from these sessions are presented in Figure 1.
TABLE 1
| Topic | Content/Exercise | Format |
| Session 0 – Introduction to the intervention and the platform | ||
| Welcome video | Information about the nature and structure of the program, and its objectives. | Real-image video with a therapist |
| Navigation of the platform | Presentation of the platform and information regarding its navigation (including instructions and steps to move through the program) | Real-image video |
| Motivation and intentions clarification | Contemplative exercise “Exploring my motivations to do this intervention” | Real-image video with a therapist |
| Session 1 – Psychoeducation and creative hopelessness | ||
| Check-in | Brief exercise to focus and anchor on the present moment (a soft landing exercise). | Video-animation with a therapist avatar |
| Psycho-education about chronic pain | Multidimensional phenomenon of pain: video about the function of pain, how it manifests itself in the body, and its various components. | Real-image video with a therapist |
| The problem with our problem-solving minds (controlling is the problem) | Video about how the human mind works and how it attempts to control unpleasant internal experiences, and consequently generating suffering and exacerbating our problems. | Videoscribe animated video |
| Promotion of creative hopelessness | Video about exploring alternative ways of relating to our sensations, thoughts, memories and emotions. One of the core competencies that will be developed in the iACTwithPain program is the ability to be in the present moment. This competence will be developed through mindfulness training. | Real-image video with a therapist |
| Introduction to mindfulness practice | Instructions on proper posture to perform contemplative practices (appropriate body postures for the contemplative practices are exemplified). | Real-image video with a therapist |
| Mindfulness of breathing practice. | Audio | |
| Between-session assignment | Mindfulness of breathing practice. | Audio |
| Session 2 – The body as an anchor on the present moment | ||
| Check-in | Brief exercise to focus and anchor on the present moment (a soft landing exercise). | Video-animation with a therapist avatar |
| Mindfulness as a key aspect to manage suffering | Video presenting what Mindfulness is and its benefits. | Real-image video with a therapist |
| Therapists’ personal experience with mindfulness: Tips for maintaining regular practice | In this video, therapists share their personal experience with mindfulness. Not only the difficulties experienced, but also the benefits obtained, resulting from a frequent and committed practice. | Real-image video with a therapist |
| The body as an anchor to the present moment | In our body everything is integrated. All systems, organs, muscles and nerve endings are interconnected and communicate with each other. Our emotions also manifest in the body (for example when we are anxious, we may feel tension in the muscles or the heart beating faster). Video about the complex entity that is the body, and exploration of usual and alternative ways of relating to our emotions, thoughts, and feelings, as inhabitants of the body. | Videoscribe animated video |
| Between-session assignment | Mindful movement (Qigong) | Real-image video with a therapist |
| Body scan meditation | Audio | |
Structure and contents of session 0, 1, and 2.
Information partly retrieved from .
FIGURE 1
Measures
Primary Outcome
Both groups of participants (patients and professionals) were asked to complete the SUS [(); original validation (); portuguese validation ()]. This scale is robust and widely used to evaluate the usability of products and user interfaces (). The European Portuguese validation is equivalent to the original version in terms of semantics and content, and presented good psychometric properties e.g., high convergent validity and satisfactory inter-rater percentage of agreement (). It provides a single reference score for participants’ view of the usability of a product or service. It comprises ten items rated on a 5-point Likert scale (1: Strongly Disagree; 5: Strongly Agree) and has been shown to present adequate psychometric properties (). The SUS items were coded by subtracting 1 from the odd items’ score, subtracting even items’ score from 5 (correction for the reverse scored items), and multiplying the re-coded values by 2,5 (). This results in new scores ranging from 0 to 100, where higher scores indicate better usability. According to the adjective ratings proposed by , SUS scores from 0 to 25 are “Worst imaginable,” 25–39 “Poor,” 39–52 “Ok,” 52–73 “Good,” 73–85 “Excellent,” and 85–100 are “Best imaginable” ().
Secondary Outcomes
Participants with chronic pain and healthcare professionals were asked to complete a set of self-report questions regarding the quality of the intervention (e.g., content clarity, pertinence of the intervention themes, platform design and organization, individual session quality), session duration, and preferences on type of presentation format (e.g., video, audio). These questions were developed by the research team to tailor to the specific platform characteristics that this study aims to assess. The platform kept track of participant engagement concerning the number of logins in the platform and engagement with mindfulness practice audios or videos.
Statistical Analyses
All analysis were performed on SPSS (Statistical Package for the Social Sciences), version 25 (IBM Corp., Armonk, NY, United States). Descriptive and frequency analyses were conducted to analyze participants’ evaluations of the platform/intervention, as well as engagement with the platform. Mann–Whitney U and Fisher’s exact test were conducted to analyze differences between groups of participants (e.g., completers, dropouts). The retention rate was computed as the proportion of participants that completed the three sessions.
Results
In this study we aimed to pilot test the usability of the iACTwithPain platform, in what concerns our primary outcome – usability scores; and secondary outcomes – qualitative assessment, engagement, and retention rate.
Primary Outcome – Usability
Sample of Participants With Chronic Pain
Regarding usability scores (SUS), on average, iACTwithPain’s platform was rated as excellent (N = 5, M = 76.50, SD = 16.83). Two participants rated it as good (M = 58.75, SD = 1.77), one rated it as excellent (82.5), and two rated it as best imaginable (M = 91.25, SD = 5.30).
Sample of Healthcare Professionals
On average, the iACTwithPain’s platform usability (SUS) was rated by healthcare professionals as excellent (N = 5, M = 84.50, SD = 7.79). Two participants rated it as excellent (M = 80.00; SD = 2.50), and three as best imaginable (M = 91.25; SD = 8.84).
Secondary Outcomes – Qualitative Assessment, Engagement, and Retention Rate
Qualitative Assessment of the iACTwithPain Platform
Sample of Participants With Chronic Pain
Of the six completers, three preferred real image video format, two preferred animations, and one preferred sole audios. Table 2 presents participants’ feedback on each of these formats. Written feedback highlighted advantages to the real image video format, which was overall described as attractive and motivating to practice. One comment indicated that more investment in the background of the real-time videos should be made and that these videos could include animations. Described advantages for the audio format were accessibility and having less distractions. Animation videos were described as interesting and having an attractive and simple design.
TABLE 2
| Questions | Participants endorsing each response category, n | M (SD) | ||||
| 1. Strongly disagree | 2. Slightly disagree | 3. Neither disagree nor agree | 4. Slightly agree | 5. Strongly agree | ||
| Audio | 34.16 | |||||
| Easy to use | 0 | 0 | 0 | 2 | 4 | 4.67 (0.52) |
| I got easily distracted | 1 | 1 | 0 | 3 | 1 | 3.33 (1.52) |
| Interesting | 0 | 0 | 0 | 4 | 2 | 4.33 (0.52) |
| I would use this format again | 0 | 0 | 0 | 4 | 2 | 4.33 (0.52) |
| Appropriate lenght | 0 | 1 | 0 | 3 | 2 | 4.00 (1.10) |
| Has technical quality | 0 | 2 | 0 | 2 | 2 | 3.67 (1.37) |
| Boring and uninteresting | 4 | 1 | 0 | 1 | 0 | 1.67 (1.21) |
| Message is clear and easy to understand (*) | 0 | 0 | 0 | 2 | 3 | 4.60 (0.55) |
| Appealing and increases motivation | 0 | 0 | 0 | 4 | 2 | 4.33 (0.52) |
| Animation | 35.60 | |||||
| Easy to use (*) | 0 | 0 | 0 | 3 | 2 | 4.40 (0.55) |
| I got easily distracted (*) | 2 | 2 | 1 | 0 | 0 | 1.80 (0.84) |
| Interesting (*) | 0 | 0 | 0 | 2 | 3 | 4.60 (0.55) |
| I would use this format again (*) | 0 | 0 | 0 | 2 | 3 | 4.60 (0.55) |
| Appropriate lenght (*) | 0 | 0 | 0 | 3 | 2 | 4.40 (0.55) |
| Has technical quality (*) | 0 | 0 | 0 | 1 | 4 | 4.80 (0.45) |
| Boring and uninteresting (*) | 4 | 0 | 0 | 1 | 0 | 1.60 (1.34) |
| Message is clear and easy to understand (*) | 0 | 0 | 0 | 1 | 4 | 4.80 (0.45) |
| Appealing and increases motivation (*) | 0 | 0 | 0 | 2 | 3 | 4.60 (0.55) |
| Real image video | 36.66 | |||||
| Easy to use | 0 | 0 | 0 | 1 | 5 | 4.83 (0.41) |
| I got easily distracted | 2 | 1 | 1 | 2 | 0 | 2.50 (1.38) |
| Interesting | 0 | 0 | 0 | 2 | 4 | 4.67 (0.52) |
| I would use this format again | 0 | 0 | 0 | 2 | 4 | 4.67 (0.52) |
| Appropriate lenght | 0 | 0 | 0 | 2 | 4 | 4.67 (0.52) |
| Has technical quality | 0 | 0 | 1 | 1 | 4 | 4.50 (0.84) |
| Boring and uninteresting | 4 | 1 | 0 | 1 | 0 | 1.67 (1.21) |
| Message is clear and easy to understand | 0 | 0 | 0 | 2 | 4 | 4.67 (0.52) |
| Appealing and increases motivation | 0 | 0 | 1 | 1 | 4 | 4.50 (0.84) |
Participants’ feedback on the different kinds of content format (n = 6).
(*) one participant did not respond to these topics.
Concerning other aspects of the intervention and platform, participants generally rated positively their satisfaction (on a scale from 0, no satisfaction, and 10, extremely satisfied) with the platform’s design (M = 7.50; SD = 2.01), color pattern (M = 8.33; SD = 1.37), and attractiveness (M = 7.67; SD = 1.75).
The topics covered by the two assessed sessions were very positively evaluated (M = 8.67; SD = 1.03). The topics found most useful to participants (n = 4) were psychoeducation about pain (n = 1), mindfulness of breathing (n = 1), and body scan practice (n = 2). Participants also provided positive answers (on a scale from 0, no satisfaction, and 10, extremely satisfied) regarding their interest in continuing the intervention (M = 7.33; SD = 3.20). Furthermore, four participants provided feedback on what they believe could be changed to improve these three sessions. One participant indicated that the sessions seemed to cover all important aspects. Another referred to longer intervals without instructions in the mindfulness practices, and two stated that they felt that strategies to reduce pain should be included in the intervention.
Sample of Healthcare Professionals
The healthcare professionals group seemed to prefer the animation (n = 3) and real image video (n = 2) formats. Real image videos (M = 34.20; SD = 2.68) and animations (M = 33.60; SD = 1.67) were overall more positively evaluated than audios (M = 27.80; SD = 3.49). Overall, written feedback about the audios indicated that this format is easy to use and accessible, although not particularly appealing or stimulating. Animations were described as attractive, and appealing, but impersonal. Real image videos were described as providing a connection with the therapeutic team and the opportunity to get to know the therapists better. Some technical issues were also identified regarding this format (e.g., indications to improve the flow of the video cuts; audio and video synchronization).
Healthcare professionals presented favorable ratings on other aspects of the platform (on a scale from 0, no satisfaction, and 10, extremely satisfied): design (M = 8.40; SD = 1.52), color pattern (M = 8.80; SD = 1.30), and attractiveness (M = 8.40; SD = 1.34).
Concerning the contents of the intervention, positive ratings were also provided (0, no satisfaction – 10, extremely satisfied) for all assessed items: content quality (M = 9.60; SD = 0.55), pertinence of the topics (M = 9.40; SD = 0.89), topics sequencing (M = 8.60; SD = 0.55), and appropriateness of therapists’ posture (M = 9.20; SD = 0.84).
Participant Engagement and Retention
The completers’ sample logged into the platform on average 16.33 times (SD = 12.91), and evaluated the first session on average as 4 (SD = 0.63), and the second session as 3.83 (SD = 0.75), both on a scale of 1–5. In what concerns home practice, completers practiced on average 2.67 times (SD = 3.20) the practice from session one (mindfulness of breathing), and 2.20 times (SD = 2.68) the practice from session 2 (body scan).
The dropout sample logged into the platform on average 3.33 times (SD = 3.90). Four of these participants completed the first session, assessed it as 4.5 on average (from 1 to 5; SD = 0.58), and did the practice from session one on average 1.75 times (SD = 0.96). No participant from the dropout group assessed the second session nor engaged in the second practice.
Reasons for Dropout
Six of the 12 participants that dropped out from the study completed the attrition questionnaire. Three participants indicated a difficult personal situation after the beginning of the study as the reason for dropout, two referred to lack of time, and one referred to having forgotten to complete the intervention (although participants were reminded once a week to complete the sessions). No participant referred to any reason associated with the platform or the intervention. Two participants provided additional feedback, indicating that: (1) “it looked very well structured, simple, and potentially very useful”; and (2) “I really liked the intervention’s contents and how the platform was structured. I had a problem with the login once, but it was quickly fixed.”
Descriptive Statistics
Sample of Participants With Chronic Pain
The demographic and clinical characteristics of the participants that accepted to participate, the ones who dropped out, and the completers are presented in Table 3.
TABLE 3
| Accepted to participate (N = 21) | Dropped-out (N = 12) | Completers (N = 6) | Test of differences | ||||
| Mann Whitney U | Fisher’s exact test | p | |||||
| Age, M (SD) | 45.35 (6.70) | 45.09 (4.01) | 46.67 (11.55) | −32.50 | – | 0.960 | |
| Middle school | 1 (4.80) | 1 (8.30) | – | 10.32 | 0.007 | ||
| High school | 8 (38.10) | 4 (41.70) | – | ||||
| Education, n (%) | BSc degree | 6 (28.60) | 2 (16.70) | 4 (66.70) | – | ||
| Post-graduation | 4 (19.00) | 4 (33.3) | – | ||||
| MSc degree | 2 (9.50) | – | 2 (33.3) | ||||
| 1–5 years | 5 (23.80) | 1 (8.30) | 3 (50) | – | 3.65 | 0.177 | |
| Time since diagnosis, n (%) | 5–10 years | 4 (19.00) | 3 (25.00) | 1 (16.70) | |||
| 10+ years | 12 (57.10) | 8 (66.70) | 2 (33.30) | ||||
| Fibromyalgia | 16 (76.20) | 9 (75.00) | 4 (66.70) | – | 3.42 | 0.769 | |
| Chronic pain diagnosis, n (%) | Rheumatoid arthritis | 2 (9.50) | 1 (8.30) | 1 (16.70) | |||
| Sjogren Syndrome | 1 (4.80) | 1 (8.30) | 0 (0) | ||||
| Low back pain | 1 (4.80) | 0 (0) | 1 (16.70) | ||||
| Scleroderma | 1 (4.80) | 1 (8.30) | 0 (0) | ||||
| Comorbid medical condition diagnosis, n (%) | Yes | 11 (52.40) | 5 (41.70) | 4 (66.70) | – | 0.620 | |
| No | 10 (47.60) | 7 (58.30) | 2 (33.30) | ||||
Descriptives and frequencies of the demographic and clinical variables in study across groups, and tests of differences between participants who dropped out and completers.
M, mean; SD, standard deviation.
There were no statistically significant differences between completers and participants who dropped out regarding age, time since chronic pain diagnosis, chronic pain diagnosis, and diagnosis of comorbid medical conditions. However, there were significant differences regarding education - participants that dropped out presented less education. The final sample of completers was composed of women with chronic pain with ages between 35 and 65 years.
Sample of Healthcare Professionals
Regarding the group of healthcare professionals, out of the eight professionals that signed an informed consent, five (four psychologists and a physician) provided feedback. These five professionals, four women and a man, had worked in chronic pain contexts between 5 and 20 years (M = 10.40; SD = 6.19) in hospitals in the North and Center regions of Portugal.
Discussion
Overall, the pilot usability study of the iACTwithPain platform (and first three sessions: 0, 1, and 2) presented promising results in terms of usability scores (high for both clinical and healthcare professionals samples), quality assessment of the different content delivery methods, and engagement metrics. Qualitative feedback from participants and health professionals will be taken into consideration for the development of the full iACT intervention, and measures to avoid dropout will be adopted.
Of the six chronic pain participants that completed this pilot test, and the five healthcare professionals that provided feedback, there seemed to be an overall satisfaction with both the intervention, regarding its contents and form of presentation of information, and the platform, concerning its design, appearance, and usability. Real image videos, mainly used to introduce new topics, provide rationales, or exemplify the possible meditation positions, appeared to be more preferred than animations or audios by chronic pain participants. This was possibly due to a perception of “being closer” to the therapists, provided by the real image videos, which may help participants feel understood and find the motivation to practice. It may also be hypothesized that real image videos are more effective in tapping into tacit cognitive-emotional factors of efficacy in psychotherapy, such as an empathic and compassionate therapeutic relationship, which is more difficult to convey through non-human avatars. This was highlighted by healthcare professionals’ feedback, who nonetheless also emphasized the appealing and dynamic aspects of the animation format. However, for meditative practices, the audio format should have preferred use over real image videos or animations to avoid unnecessary distractions during practice. Finally, two participants with chronic pain indicated that they believed that strategies to reduce pain should be included in the intervention. Even though the first session of the iACTwithPain intervention states the rationale for accepting pain rather than attempting to reduce or control it (), and the negative effects of doing the latter (), it seems that these participants still held on to the idea that pain must be avoided to lead a satisfactory life. Still, participants had only completed the two first sessions of the intervention, where ACT’s acceptance and values topics are not clearly focused on (these topics are presented more in-depth in later sessions). Therefore, it might be beneficial for the future RCTs to increase the focus on acceptance earlier in the intervention, by including, for example, experiential acceptance exercises or metaphors.
The study attrition rate was of 71.43%. Out of the 21 participants that signed the informed consent, 12 (57.14%) accessed the intervention, and 6 (28.57%) completed the study. No significant differences in demographic or clinical variables were found between dropouts and completers except for completed education, with participants who dropped out presenting less education than completers. This result is in line with findings from previous research, where lower educational level was associated with higher risk if dropping out (). Motives for dropout, offered by 6 of the 12 participants who did not finish the intervention, were unrelated to the intervention or platform. There were reasons related to difficult personal events occurring during the intervention, lack of time, or having forgotten. Similar results were found in a review about reasons for dropping out in ACT interventions (). The iACTwithPain platform will thus include mechanisms to identify when participants have not logged in to the platform for 3 days in a row, so the team can send engagement reminders to these participants. This strategy is aligned with previous studies showing that contingent reminders can improve motivation and boost behavior change (). If emails are not enough to engage non-responsive participants, phone calls will be implemented, since previous online-based studies have suggested that closer contact with the therapist is associated with lower dropout rates (; ). Still, given that only half of the participants who dropped out provided reasons for having stopped participating, it is difficult to know whether the platform/intervention did not influence some participants’ dropout. Other factors that we did not account for might be influencing dropout, namely comorbid depression or anxiety, relationship status, or chronic pain severity, according to a review by . The attrition rate presented by this study falls within the range of 2–83%, presented by a review on dropout rates from online treatments for psychological disorders (). It should nonetheless be noted that the current intervention is directed at people with chronic pain, and most importantly, that this is a usability study, in which participants may present less motivation to participate in comparison with a trial of a full intervention.
This usability study will improve the iACTwithPain platform by informing which type of format is preferred and which one works best for each type of content. Considering feedback from the participants, the real image videos with therapists were overall preferred and viewed as engaging and motivating, so we will mostly select this format in the future intervention. Nevertheless, all types of exercises will be maintained (animations, videoscribes, real image videos, and audios), since some participants showed preference for these formats and since this will allow for a more varied and stimulating user experience. The audio format will be selected for guided exercises (e.g., 80th Birthday Party) or meditations (e.g., body scan), since participants reported that this format helped them maintain focus on the instructions of the exercises. Participants did not provide negative feedback regarding the attractiveness, navigation or esthetic elements (e.g., design and color pattern) of the platform. Therefore, no changes will be made to these elements. Finally, weekly automatic and contingent reminders (when a participant does not login in for three consecutive days) will be implemented to reduce the likelihood of participants dropping out from study.
This study has some limitations that are worth discussing. We did not assess previous experience with acceptance or compassion-based psychological interventions, so it was not possible to control the possible confounding effect of previous knowledge/experience on these approaches. The small sample size might have influenced the results obtained and might not generalize to broader samples. Only women enrolled in this study (although this was not a criterion for this study), which limits generalization of these results to male chronic pain samples. However, this usability study allowed the initial test of the iACTwithPain platform and provided valuable insights for developing the full intervention. A future feasibility study informed by the current one is being conducted by the iACTwithPain team and will assess several recruitment pathways/strategies, as well as the full intervention (both quantitatively and qualitatively).
The present study has informed the improvement of the iACTwithPain platform before its final testing in a larger clinical trial. In addition, this study provides useful information on some aspects researchers should consider while developing online psychological interventions for chronic pain.
Publisher’s Note
All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.
Statements
Data availability statement
The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.
Ethics statement
The studies involving human participants were reviewed and approved by the Faculty of Psychology and Educational Sciences, University of Coimbra, Portugal. The patients/participants provided their written informed consent to participate in this study.
Author contributions
IT: conceptualization, data curation, formal analysis, and writing—original draft, review and editing. RG: data curation, formal analysis, and writing—original draft. SC, PM, MN, TL, JD, and JP-G: writing—review and editing. BP: data curation and writing–review and editing. PC: funding acquisition, conceptualization, supervision, and writing—review and editing. All authors read and approved the final manuscript.
Funding
This project (REF: POCI-01-0145-FEDER-028829 | PTDC/PSI GER/28829/2017) was funded by the Fundo Europeu de Desenvolvimento Regional (FEDER) of the European Union, through the Programa Operacional Regional do Centro (CENTRO 2020) of Portugal-2020 and by the Fundação para a Ciência e Tecnologia I.P./MCTES through national funds (PIDDAC). RG was supported by a Ph.D. Grant (SFRH/BD/5099/2020), sponsored by FCT (Portuguese Foundation for Science and Technology).
Conflict of interest
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
Abbreviations
- ACT
Acceptance and Commitment Therapy
- SUS
System Usability Scale
- M
mean
- SD
standard deviation.
References
1
APA Presidential Task Force on Evidence-Based Practice (2006). Evidence-based practice in psychology.Am. Psychol.61271–285. 10.1037/0003-066X.61.4.271
2
BangorA.KortumP. T.MillerJ. T. (2008). An empirical evaluation of the system usability scale.Int. J. Hum. Comput. Interact.24574–594. 10.1080/10447310802205776
3
BangorA.KortumP.MillerJ. (2009). Determining what individual SUS scores mean: adding an adjective rating scale.J. Usabil. Stud.4114–123.
4
BergmoT. S. (2015). How to measure costs and benefits of eHealth interventions: an overview of methods and frameworks.J. Med. Internet Res.17:e254. 10.2196/jmir.4521
5
BernfortL.GerdleB.RahmqvistM.HusbergM.LevinL. -Å (2015). Severity of chronic pain in an elderly population in Sweden—impact on costs and quality of life.Pain156521–527. 10.1097/01.j.pain.0000460336.31600.01
6
BrookeJ. (1996). SUS -A quick and dirty usability scale Usability and context.Usabil. Eval. Ind.1891–7. 10.1201/9781498710411-35
7
CarvalhoS. A.GillandersD.PalmeiraL.Pinto-GouveiaJ.CastilhoP. (2018). Mindfulness, selfcompassion, and depressive symptoms in chronic pain: the role of pain acceptance.J. Clin. Psychol.742094–2106. 10.1002/jclp.22689
8
CarvalhoS. A.TrindadeI. A.DuarteJ.MenezesP.PatrãoB.NogueiraM. R.et al (2021b). Efficacy of an ACT and compassion-based eHealth program for self-management of chronic pain (iACTwithPain): study protocol for a randomized controlled trial.Front. Psychol.12:630766. 10.3389/fpsyg.2021.630766
9
CarvalhoS. A.GillandersD.ForteT.TrindadeI. A.Pinto-GouveiaJ.LapaT.et al (2021a). Self-compassion in acceptance and commitment therapy for chronic pain: a pilot study.Scand. J. PainOnline ahead of print, 10.1515/sjpain-2021-0214
10
CarvalhoS. A.TrindadeI. A.GillandersD.Pinto-GouveiaJ.CastilhoP. (2019). Cognitive fusion and depressive symptoms in women with chronic pain: a longitudinal growth curve modelling study over 12 months.Clin. Psychol. Psychother.26616–625. 10.1002/cpp.2386
11
CarvalhoS. A.XavierA.GillandersD.Pinto-GouveiaJ.CastilhoP. (2021c). Rumination and valued living in women with chronic pain: how they relate to the link between mindfulness and depressive symptoms.Curr. Psychol.401411–1419. 10.1007/s12144-018-0052-z
12
CuijpersP.van StratenA.AnderssonG. (2008). Internet-administered cognitive behavior therapy for health problems: a systematic review.J. Behav. Med.31169–177. 10.1007/s10865-007-9144-1
13
DornerT. E.AlexandersonK.SvedbergP.TinghögP.RopponenA.Mittendorfer-RutzE. (2016). Synergistic effect between back pain and common mental disorders and the risk of future disability pension: a nationwide study from Sweden.Psychol. Med.46425–436. 10.1017/S003329171500197X
14
DuarteC.Pinto-GouveiaJ.StubbsR. J. (2017). Compassionate attention and regulation of eating behaviour: a pilot study of a brief low-intensity intervention for binge eating.Clin. Psychol. Psychother.24O1437–O1447. 10.1002/cpp.2094
15
ElliottA. M.SmithB. H.PennyK. I.Cairns SmithW.Alastair ChambersW. (1999). The epidemiology of chronic pain in the community.Lancet3541248–1252. 10.1016/S0140-6736(99)03057-3
16
FaulknerL. (2003). Beyond the five-user assumption: benefits of increased sample sizes in usability testing.Behav. Res. Methods Instrum. Comput.35379–383. 10.3758/bf03195514
17
GilbertP. (2009). Introducing compassion-focused therapy.Adv. Psychiatr. Treat.15199–208. 10.1192/apt.bp.107.005264
18
GoodingH.StedmonJ.CrixD. (2020). All these things don’t take the pain away but they do help you to accept it’: making the case for compassion-focused therapy in the management of persistent pain.Br. J. Pain1431–41. 10.1177/2049463719857099
19
GroenewaldC. B.PalermoT. M. (2015). The price of pain: the economics of chronic adolescent pain.Pain Manag.561–64. 10.2217/pmt.14.52
20
HayesS. C.StrosahlK. D.WilsonK. G. (2012). Acceptance and Commitment Therapy: The Process and Practice of Mindful Change, 2nd Edn. New York, NY: Guilford Press.
21
KareklaM.ConstantinouP.IoannouM.GlosterA. T.KareklasI. (2019). The phenomenon of treatment dropout, reasons and moderators in acceptance and commitment therapy and other active treatments: a meta-analytic review.Clin. Psychol. Eur.1:e33058. 10.32872/cpe.v1i3.33058
22
KaryotakiE.KleiboerA.SmitF.TurnerD. T.PastorA. M.AnderssonG.et al (2015). Predictors of treatment dropout in self-guided web-based interventions for depression: an ‘individual patient data’ meta-analysis.Psychol. Med.452717–2726. 10.1017/S0033291715000665
23
MaceaD. D.GajosK.Daglia CalilY. A.FregniF. (2010). The efficacy of web-based cognitive behavioral interventions for chronic pain: a systematic review and meta-analysis.J. Pain11917–929. 10.1016/j.jpain.2010.06.005
24
MartinsA. I.RosaA. F.QueirósA.SilvaA.RochaN. P. (2015). European Portuguese validation of the System Usability Scale (SUS).Procedia Comput. Sci.67293–300. 10.1016/j.procs.2015.09.273
25
MayerS.SpickschenJ.SteinK. V.CrevennaR.DornerT. E.SimonJ. (2019). The societal costs of chronic pain and its determinants: the case of Austria.PLoS One14:e0213889. 10.1371/journal.pone.0213889
26
McCrackenL. M.EcclestonC. (2003). Coping or acceptance: what to do about chronic pain?Pain105197–204. 10.1016/S0304-3959(03)00202-1
27
McCrackenL. M.VowlesK. E.Gauntlett-GilbertJ. (2007). A prospective investigation of acceptance and control-oriented coping with chronic pain.J. Behav. Med.30339–349. 10.1007/s10865-007-9104-9
28
MelvilleK. M.CaseyL. M.KavanaghD. J. (2010). Dropout from internet-based treatment for psychological disorders.Br. J. Clin. Psychol.49455–471. 10.1348/014466509X472138
29
NeffK. D. (2003). The development and validation of a scale to measure self-compassion.Self Identity2223–250. 10.1080/15298860309027
30
PlumbJ. C.VilardagaR. (2010). Assessing treatment integrity in acceptance and commitment therapy: strategies and suggestions.Int. J. Behav. Consultation Ther.6263–295. 10.1037/h0100912
31
SmithB. H.ElliottA. M.ChambersW. A.SmithW. C.HannafordP. C.PennyK. (2001). The impact of chronic pain in the community.Fam. Pract.18292–299. 10.1093/fampra/18.3.292
32
TrindadeI. A.FerreiraC.Pinto-GouveiaJ. (2020). Acceptability and preliminary test of efficacy of the mind programme in women with breast cancer: an acceptance, mindfulness, and compassion-based intervention.J. Context. Behav. Sci.15162–171. 10.1016/j.jcbs.2019.12.005
33
TrindadeI. A.GuiomarR.CarvalhoS. A.DuarteJ.LapaT.MenezesP.et al (2021a). Efficacy of online-based acceptance and commitment therapy for chronic pain: a systematic review and meta-analysis.J. Pain221328–1342. 10.1016/j.jpain.2021.04.003
34
TrindadeI. A.PereiraJ.GalhardoA.FerreiraN. B.Lucena-SantosP.CarvalhoS. A.et al (2021b). The LIFEwithIBD intervention: study protocol for a randomized controlled trial of a face-to-face acceptance and commitment therapy and compassion-based intervention tailored to people with inflammatory bowel disease.Front. Psychiatry12:699367. 10.3389/fpsyt.2021.699367
35
VeehofM. M.OskamM.-J.SchreursK. M. G.BohlmeijerE. T. (2011). Acceptance-based interventions for the treatment of chronic pain: a systematic review and meta-analysis.Pain152533–542. 10.1016/j.pain.2010.11.002
36
VeehofM. M.TrompetterH. R.BohlmeijerE. T.SchreursK. M. G. (2016). Acceptance- and mindfulness-based interventions for the treatment of chronic pain: a meta-analytic review.Cogn. Behav. Ther.455–31. 10.1080/16506073.2015.1098724
37
VowlesK. E.McCrackenL. M. (2008). Acceptance and values-based action in chronic pain: a study of treatment effectiveness and process.J. Consult. Clin. Psychol.76397–407. 10.1037/0022-006X.76.3.397
38
WebbT. L.JosephJ.YardleyL.MichieS. (2010). Using the internet to promote health behavior change: a systematic review and meta-analysis of the impact of theoretical basis, use of behavior change techniques, and mode of delivery on efficacy.J. Medic. Internet Res.12:e4. 10.2196/jmir.1376
Summary
Keywords
acceptance and commitment therapy, chronic pain, online intervention, usability study, IT
Citation
Guiomar R, Trindade IA, Carvalho SA, Menezes P, Patrão B, Nogueira MR, Lapa T, Duarte J, Pinto-Gouveia J and Castilho P (2022) Usability Study of the iACTwithPain Platform: An Online Acceptance and Commitment Therapy and Compassion-Based Intervention for Chronic Pain. Front. Psychol. 13:848590. doi: 10.3389/fpsyg.2022.848590
Received
04 January 2022
Accepted
24 June 2022
Published
22 July 2022
Volume
13 - 2022
Edited by
Fanbo Meng, Jiangnan University, China
Reviewed by
Michael Bordieri, Murray State University, United States; Francisco Javier Cano-García, Seville University, Spain
Updates
Copyright
© 2022 Guiomar, Trindade, Carvalho, Menezes, Patrão, Nogueira, Lapa, Duarte, Pinto-Gouveia and Castilho.
This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: Raquel Guiomar, raquelguiomar18@gmail.com
This article was submitted to Health Psychology, a section of the journal Frontiers in Psychology
Disclaimer
All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article or claim that may be made by its manufacturer is not guaranteed or endorsed by the publisher.