Feasibility of Recovery Assessment Scale – Domains and Stages (RAS-DS) for everyday mental health practice

Introduction Routine use of self-rated measures of mental health recovery can support recovery-oriented practice. However, to be widely adopted, outcome measures must be feasible. This study examined the feasibility of Recovery Assessment Scale – Domains and Stages (RAS-DS) from the perspectives of mental health workers. Method Mental health workers who had previously sought permission to use RAS-DS (n=58) completed an online survey that explored three aspects of feasibility: practicality, acceptability and applicability. Results The highest-rated feasibility items related to applicability, or usefulness in practice, with over 90% of participants reporting that RAS-DS helps “promote discussion” and covers areas that are “meaningful to consumers”. Acceptability items indicated that the purpose of RAS-DS is clear but length was an issue for some participants. At a practical level, RAS-DS was seen as easy to access but training was seen by many as necessary to ensure optimal use. Conclusion Results suggest potential usefulness of RAS-DS as a routine outcome measure and identify aspects that can be addressed to further enhance feasibility including provision of training materials and opportunities, wide-reaching promotion of its use as a collaborative tool, and further investigation of issues around instrument length.


Introduction
Over the last three decades, the notion of recovery and adoption of recovery-oriented practice has gained prominence in mental health services (1,2).Shifting from a dominant focus on symptom amelioration (3), recovery refers to a "deeply personal, unique process of changing one's attitudes, values, feelings, goals, skills and/or roles -a way of living a satisfying, hopeful and contributing life, even with limitations caused by illness" (4, p. 527).Recoveryoriented practice is grounded in the belief that individuals experiencing mental illness can recover.Practice should embody a collaborative and person-centred approach that fosters selfdetermination, choice, and hope (5).
Concurrent with shifts towards recovery-oriented practice, mental health services are increasingly expected to evaluate service delivery through use of routine outcome measures (6,7).Routine outcome measurement was also introduced to support mental health workers in clinical decision-making and engaging consumers in careplanning (6,8).Current routine outcome measures mandated in countries such as Australia include a mixture of clinician-rated (e.g., Health of the Nation Outcome Scales ( 9)) and self-rated (e.g., Behaviour and Symptom Identification Scale-32 (10)) measures that focus on symptomology, hospitalisation rates and level of functioning (7).However, in the context of moving towards recovery-oriented practice, Lakeman (11) and Happell (12) argued that mandating routine use of such symptom-focused measures does little to inform recovery-oriented approaches and is not aligned with what is meaningful to consumers in their recovery.Indeed, this may create barriers to recovery-oriented practice (13).
The ability to assess and measure recovery-focused outcomes from the consumer's perspective supports the adoption of recoveryoriented practice (14,15).Using self-rated recovery measures presents opportunities for workers to focus on what is important to individual consumers, and actively engage consumers through collaborative goal-setting and care-planning, all fundamental to recovery-oriented practice (16,17).While mandating completion of a recovery measure by consumers would be antithetical to the choice and autonomy inherent in the concept of recovery, routinely offering it and explaining the benefits is valuable in developing recovery-oriented services.
Consequently, efforts have been made to establish psychometrically sound recovery measures that can be used for routine outcome measurement (18-20).However, none of these measures have been comprehensively examined for their feasibility, with typically only brevity explored.To support the uptake of selfrated recovery measures in routine practice, it is important that measures are not only psychometrically sound but also feasible.For example, low or variable completion rates of routine outcome measures are common (6,7,21) and these are often related to feasibility issues such as completion times, scoring complexity, unmet training needs, and, in particular, limited valuing of the measures by workers (22)(23)(24)(25)(26).
RAS-DS is a 38-item self-rated recovery measure that encompasses four recovery domains: "Doing things I value" (functional aspects), "Looking forward" (psychological aspects), "Mastering my illness" (symptom management), and "Connecting and belonging" (social aspects).Users rate items on a four point scale (1 untrue, 2 a bit true, 3 mostly true, 4 completely true).It was developed from the Recovery Assessment Scale (RAS) (27), with revised structure and items to address identified problems with RAS measurement properties (28).Formal training is not required to administer RAS-DS and a user-manual can be readily accessed online to guide workers (29).Items are all positively phrased so that a total recovery score can be obtained by simply adding all items and domain scores are obtained by averaging item scores in each domain.RAS-DS has demonstrated good reliability, validity, and sensitivity to change (30,31).Beyond measuring outcomes, however, RAS-DS was designed as a tool to guide workers' collaborative practice with consumers (29).It has been translated into 18 languages and uptake has been widespread across 26 countries.While these features make it a promising candidate for a routine outcome measure, little work has been done to establish its feasibility.Previous studies have suggested that it was relatively quick and easy to use and contributed towards goal-setting and the therapeutic alliance (31,32).However these studies touched on feasibility issues in the context of broader analysis of measurement properties.Given the barriers to implementing routine outcome measures, a more comprehensive assessment of the feasibility of RAS-DS is required.The aim of this study was to explore the feasibility of RAS-DS in detail from the perspectives of mental health workers.

Study design
This descriptive study used surveys with RAS-DS users to collect both quantitative and qualitative data to gain a nuanced understanding of participants' perspectives of the feasibility of RAS-DS (33).Ethical approval was obtained from The University of Sydney Human Research Ethics Committee (approval number 2019/921).

Measuring feasibility
One of the most influential feasibility frameworks used in mental health services was developed by Andrews et al. (34).This framework proposed that a "feasible" measure is practical, acceptable, and applicable to consumers and workers.A practical measure imposes minimal cost.Further, administering, scoring, and interpreting results should be simple, with instructions provided and little training needed.An acceptable measure is brief and userfriendly, for example, the format and language should be easy for consumers to understand.Lastly, an applicable measure addresses outcomes aligned with consumer priorities and facilitates appropriate treatment decisions.In recovery-based practice, facilitating appropriate treatment decisions requires consumer involvement and understanding between worker and consumer.To be applicable, a measure should also be multi-dimensional to enable specific domains to be illuminated, and able to be meaningfully aggregated for management requirements.This framework has been utilised in previous studies (35,36) to evaluate the feasibility of potential routine outcome measures.In addition to aspects covered within the Andrews et al. (34) framework, some additional feasibility considerations have also been suggested.These include cultural appropriateness, capacity to promote discussions between consumers and workers (36), flexibility in administration options, and ability to provide information otherwise unavailable, contributing towards workers' understanding of consumers (37).Therefore, for this study, the Andrews et al. (34) framework was expanded upon to include these additional aspects.The updated feasibility framework is summarised in Table 1.

Participants
Eligible participants were workers currently or previously using RAS-DS.Given the absence of a comprehensive sampling frame, convenience sampling was used.RAS-DS information specifies that, while the tool is freely available, the author's permission is required for organisations to use it.The author keeps a database of people who have requested this permission and have agreed to receive information pertaining to it.Invitations were sent to the 183 individuals on this database.In recognition that that many people and organisations do not seek permission and that other individuals at each organisation will use RAS-DS, the invitation encouraged potential participants to distribute the invitation to other colleagues who had used RAS-DS.The email included a Participant Information Statement and the link to the survey.

Data collection
The survey instrument was developed for this study and pilottested with two RAS-DS users from Australia and Canada.It collected information on: a) participants' demographics and practice context, b) current or previous use of RAS-DS; and c) feasibility aspects of RAS-DS using items developed from the updated feasibility framework (Table 1).No items were included around RAS-DS being low cost, multidimensional and able to be aggregated given that these are objective features of RAS-DS (which is cost-free, is arranged around four domains and has demonstrated measurement properties that support aggregation) (31).
Throughout the survey, participants were given the option to explain or comment on their responses, and they provided free-text responses to questions about most and least useful aspects of RAS-DS.Current RAS-DS users were also asked further questions regarding their RAS-DS use (e.g., why and how RAS-DS was used, frequency of use and whether RAS-DS helped them be more recovery-oriented in their practice).
Study data were collected and managed using the REDCap electronic data capture tool hosted at The University of Sydney (38).

Data analysis
Frequencies, medians and interquartile range values were calculated for fixed choice items.Qualitative data from openended questions ranged from short phrases to paragraphs.Constant comparative analysis was used to thematically analyse this data (39).Due to conceptual overlap in responses, data from different questions were analysed together to compile and quantify the overall frequency of themes.Initial coding began with reading each response and identifying and labelling underlying concepts (39).With each response, data were compared with existing codes to check if they conveyed similar meanings.New concepts were labelled as new codes.Progressively, codes were compared to one another and grouped into higher-level categories if conceptual similarity existed (39).To enhance trustworthiness, the authors SR and NH independently coded the full data set and subsequently discussed their interpretations and categorisations before reaching consensus.Counts were made of participants reporting each thematic category.

Participants
The survey was commenced by 79 participants, however 20 responses were excluded as they had insufficient usable data or the participant (n=1) did not meet the inclusion criteria as they did not provide consent for participation.The final sample size was 58.All participants provided informed consent.Demographic characteristics of participants are summarised in Table 2 and their described use of RAS-DS are presented in Table 3.

Practicality
Summary results for practicality questions are presented in Table 4. Responses to the practicality questions indicate that, while most people found it practical, a significant minority indicated that training and support was, or would be beneficial, especially around interpretation.A large majority of participants saw RAS-DS as easy to access and score (86.6% and 81.5% respectively).While still a majority, fewer agreed that the results were easy to interpret (66.7%) and that minimal training was required (62.3%).
Participants' qualitative comments, both in relation to their ratings for these items and in response to other open-ended questions, provide additional detail about their perceptions of its practicality.In terms of ease of access, positive comments often referenced RAS-DS being freely available online, however some participants seemed unaware of this and mentioned having accessed RAS-DS through the author or their workplace.The one participant who disagreed that RAS-DS was easy to access described "having to print it out on paper is a pain" (P44) and three others noted that "an app would be immensely useful" (P34).A number of people who found the RAS-DS very easy to score indicated that they had used automatic scoring provided by the authors in Excel or provided by their employers, while some who gave a low rating for ease of scoring described manual scoring as time consuming.Of the 18 people who commented about their rating, the most common comments (n=7), made by people with quite different ratings, centred around the meaning of scores being highly individualised and the need to interpret RAS-DS in conversation or "consultation with the consumer" (P13) themselves.
Seventeen people (29%) commented on the need for training.Four (7%) characterised RAS-DS as "self-explanatory".However, thirteen (22%), with varying ratings of agreement on this question, believed that some training was required, especially for workers to meaningfully integrate RAS-DS and its results into practice and for  Five participants (9%) noted that practical issues were sometimes related, not to RAS-DS per-se, but to their own services, for example the time they had available, lack of platform for documentation of results and lack of awareness of the tool amongst different teams.(e.g., "It depends on the time provided in the clinic.When I have a lot of patients, there were limited time to use the RAS-DS" (P75)).

Acceptability
Summary results for acceptability questions are presented in Table 5.A large majority of people agreed that the RAS-DS had a clear and relevant purpose, that items were easy to understand and that consumers could choose method of completion.The lowest rated items in any domain were those relating to the length of RAS-DS.
A total of 32 participants (55%) commented about the length of RAS-DS.While six (11%) characterised is as "short" (P27) or "brief" (P19), 12 participants (22%) indicated that RAS-DS was "a bit long" (P32), or had too many items.For example, one explained that "our patients complete the tool but will do better with a shorter version" (P71).In contrast, 5 participants (9%) explained that the length was less important than the potential benefits of RAS-DS or that the items were necessary to explore recovery holistically.P1, for example thought that RAS-DS "needs to have the level of detail as it gets clients thinking of many aspects of recovery", while P34 stated that "there are a few questions, but the fact that they help encourage conversation is important".Five participants (9%) explained that the appropriateness of the length depended on the consumer, for example the acuteness of their symptoms, their cognition or their literacy.However, four others (7%) described using strategies to address these issues, such as using RAS-DS "one section at a time" (P41) and "assist(ing) (consumers) in completing the form" (P54).
Twenty-six people commented about other acceptability issues in their discussion of most and least helpful features of the RAS-DS or explanation of their usefulness rating.Seventeen of these were positive about the use of language, with 9 (16%) commenting that it was easy for consumers to understand the purpose and "the language is simple to understand" (P2).Nine (16%) appreciated the use of positive language, "a more positively worded set of questions" (P28) and that "it doesn't feel like a pass/fail thing" (P58).However, seven people (13%) felt that the wording could be abstract or confusing for some consumers, especially those experiencing acute symptoms or cognitive limitations.When discussing least helpful aspects, a small number of participants also brought up issues with specific items (n=4; 7%) and lack of cultural appropriateness (n=2; 4%) (e.g., "It does not include element of recovery specifically related to family, which is important in an Asian context."(P10)).

Applicability
Summary results for applicability questions are presented in Table 6.Applicability was the highest ranked domain of feasibility overall.Of the 45 participants currently using RAS-DS who reported the overall usefulness of RAS-DS, more than 90% reported finding it moderately useful (n=10), quite useful (n=15) or very useful (n=17) overall.Two of those who reported RAS-DS only slightly useful or not useful explained that this was because they had limited understanding/education on how to use RAS-DS.Another said usefulness was limited as few workers in their service were recovery-oriented.
Of the more specific applicability items, the highest-rated items were "The RAS-DS helps promote discussion" and "The areas covered by the RAS-DS are meaningful and important to the people I work with".
Most participants who responded to the question about whether RAS-DS helped them to take a recovery-oriented approach in practice (40 out of 45, 89%) agreed that it did either to a large (n=17), moderate (n=19) or small (n=4) degree.Three participants were unsure and the two participants who reported "No" explained that they had "always been recovery-oriented" (P56).
Free text responses provided further information about the applicability of RAS-DS.Of the 49 current users, 47 responded to the question asking how they used the results.Five (10%) reported not using them significantly beyond as mandated documentation.For example, P6 stated "I score the RAS-DS, put it in my case notes and don't look at it again".Others, however, reported a variety of uses, which are detailed and exemplified in Table 7. Forty-six participants commented on applicability aspects of RAS-DS when asked about most useful features, with some providing additional related comments in response to other questions.Because of overlap of these themes with reported use, frequency and examples are also included in Table 7.
However, nine participants noted, when asked about least useful features, that the usefulness of RAS-DS was dependent on how the consumer engaged with it, which could be influenced by a variety of features such as consumer's understanding of their illness, how they were feeling on that day or how much information they wanted to

Discussion
The purpose of this study was to examine the feasibility of RAS-DS in detail from workers' perspectives.Overall, results support previous findings that have suggested that RAS-DS is easy to use and valued by consumers and workers for its ability to facilitate goal-setting, prompt discussions about recovery, and track recovery progress (31,32).This study provides additional information and detail, however, especially about aspects of feasibility that most support routine use and aspects that warrant further consideration.
Applicability was the highest-rated feasibility domain of RAS-DS and a large proportion of participants described applicability aspects as what they found most useful about RAS-DS.This is promising as research has repeatedly demonstrated that workers are more willing to adopt routine measures that contribute to practice (25,40) and promote discussions that inform care-planning (41)(42)(43).Free-text responses also show how RAS-DS can promote recovery-oriented practice.For example, understanding consumer perspectives, stimulating reflections on recovery and actively engaging consumers in goal-setting are seen as critical to ensuring that interventions are person-centred and meaningful to consumers (44).Some participants, however, indicated using RAS-DS in a more service-oriented way, for example, for staff to monitor individual progress or the overall impact of services.While a number of barriers have been found to workers implementing RAS-DS as a recovery planning tool rather than just an outcome measure (45), this study demonstrates its potential for use for many purposes.However, it is clear from the minority of participants who appeared to make little use of the results, that mandating RAS-DS is of little use if workers do not have the time, understanding or the will to use it as designed.
Despite many positive responses, some participants raised concerns regarding the acceptability of RAS-DS.First some   participants found it was too long overall, or too long for certain consumers.Although RAS-DS takes most consumers 15 minutes or less to complete (31), this is worth considering given that workers are less likely to use tools they deem too long (22,40).However, the same studies have concurrently evidenced that workers are more likely to use a tool they believe informs practice (22,40).In the current study, some participants suggested that the length of RAS-DS was of lesser importance compared with its benefits and that the length was necessary to explore recovery holistically.A shorter measure, although more practical, may be less applicable.Crawford et al. (46) found that compared to shorter tools, consumers valued in-depth tools that examined different aspects of life and facilitated discussions.Therefore, this prompts the question: how brief should a measure be to be practical, whilst still applicable and useful to workers and consumers?Promotion of the benefits of RAS-DS for use as a recovery planning tool rather than just an outcome measure (45) and education around different ways to complete the RAS-DS, such as across multiple sessions, may help to alleviate this perception.However, further exploration of worker and consumer perspectives on how to achieve this balance is warranted.Related to practicality, many participants believed additional support or training was needed to optimally use RAS-DS and its results to guide practice.The type of training provided on the use of a routine measure can be pivotal in determining whether it is used solely as an administrative tool or as a tool to guide and inform practice (47,48).For self-rated measures such as RAS-DS, training focused on how they can be used collaboratively with consumers can increase workers' perceptions of their value and facilitate uptake (22,49).Although developing a tool that is easy to use with minimal training is critical for uptake (50), further consideration is needed to ensure that RAS-DS is used optimally and as intended.The authors have conducted additional research into the facilitators and barriers to using RAS-DS as more than an outcome measure (45) and, based on these findings, are currently developing an app that will incorporate guidance for workers on the implementation of RAS-DS and interpreting the results as well as support its use for recovery focused conversations and person-centred  goal setting.This app will also address other practicality issues by eliminating the need for paper copies and by calculating scores, thus making the RAS-DS easier to use.Some participants believed the usefulness of RAS-DS was dependent on consumer factors such as their understanding of their illness, mood, or how much they wanted to disclose.Similar criticisms have been reported previously with clinicians expressing concerns that consumers may over or under-represent their experiences and doubting the validity and results of self-rated measures if they did not align with their clinical judgement (51-53).However, considering the deeply personal and unique nature of recovery (4), it is argued that no one apart from consumers themselves can measure their recovery (54).Perhaps, workers should instead view differences in opinions as opportunities to further explore consumer perspectives and to develop shared understandings of consumers' experiences.Further, consideration should be given to relational issues.Consumers may respond "strategically" if there was mistrust or fear about how their selfratings would be interpreted and whether it would affect services they received (55, 56).Developing a strong and trusting therapeutic alliance with consumers is key in creating a safe environment for consumers to respond honestly when using self-rated measures (56).

Limitations
This study gathered worker perspectives of RAS-DS.While workers are the "gatekeepers" of the tool and determine whether and how it is used, understanding consumer perspectives about feasibility is essential (13).While previously examined to some degree (31,32), consumer perspectives should be addressed in more detail in future studies.
A major limitation of this study relates to the use of convenience sampling to recruit participants.It is likely that people who contacted the author for permission to use RAS-DS, were using it at the time of the study (as 85% of respondents were), and chose to respond to a survey about it, would value its use to a greater extent than those who did not.Therefore, results of the study may well be positively skewed and thus not necessarily generalisable to the wider population of mental health workers.Nevertheless, the study suggests aspects of RAS-DS that make it viable as a routine outcome measure and those requiring further consideration.

Conclusion
Notwithstanding the limitations, the findings from this study indicate a potential for RAS-DS to be used as a routine outcome measure of recovery.They also suggest that the RAS-DS has great potential to promote meaningful recovery-focussed discussions with consumers, facilitate collaborative care-planning, monitor consumer recovery progress and support recovery-oriented practice.Although further consideration of length may be warranted, initiatives in progress to address training needs and promote the use of RAS-DS as more than an outcome measure will further enhance its feasibility.

TABLE 1
Updated feasibility framework adopted in this study.Purpose is clear and relevant to consumers a Wording is easy to understand for consumers a Culturally appropriate b Flexible administration (i.e., consumers are able to choose whether to complete the tool on their own or with workers) c (37)sy to score (i.e.simple and quick calculation of totals) a Easy to interpret results (i.e. the meaning of item and total scores is clear) a Easy to access a Low cost a Acceptability Brief (i.e., not too many questions and items) a a Able to be aggregated (i.e., meaningful totals can be calculated) a a Adapted from Andrews, Peters(34).bAdapted from Siggins Miller Consultants (36).cAdapted from Slade, Thornicroft(37).

TABLE 2 Continued
people with less clinical training and experience (e.g., "people need to know how to use it properly as a tool to engage conversations with people, instead of just get them to tick boxes" [P44]).

TABLE 3
Use of RAS-DS (n = 49).with worker.For example, P6 stated that "It is very dependent on how the person is feeling on the day and as to what they actually want to disclose".
a The remainder of the table only includes the data from participants who were currently using RAS-DS (n = 49).disclose

TABLE 4
Practicality ratings and scores.
a 'I don't know' responses treated as missing data.b Combined "Agree" and "Strongly agree" responses.c Combined "Disagree" and "Strongly disagree" responses.d IQR, Interquartile range.

TABLE 5
Acceptability ratings and scores.IQR, Interquartile range.e For simplicity of comparison, negatively-worded items are re-phrased and reverse scored for the purpose of table.
a 54 participants completed this section.Differences in 'n' due to 'I don't know' responses being treated as missing data.bCombined "Agree" and "Strongly agree" responses.cCombined "Disagree"and "Strongly disagree" responses.d

TABLE 6
Applicability ratings and scores.IQR, Interquartile range.e For simplicity of comparison, negatively-worded items are re-phrased and reverse scored for the purpose of table.
a 54 participants completed this section.Differences in 'n' due to 'I don't know' responses being treated as missing data.b Combined "Agree" and "Strongly agree" responses.c Combined "Disagree" and "Strongly disagree" responses.d

TABLE 7
Frequency and examples of how RAS-DS results are used and how RAS-DS is regarded as useful.