The interRAI Suite of Mental Health Assessment Instruments: An Integrated System for the Continuum of Care
- 1School of Public Health and Health Systems, University of Waterloo, Waterloo, ON, Canada
- 2Psychiatry and Neuropsychology Department, Maastricht University, Maastricht, Netherlands
- 3Centre for Health Services Research, Faculty of Medicine, University of Queensland, Brisbane, QLD, Australia
- 4Psychiatric Department, Rio Hortega University Hospital, Zamora, Spain
- 5Institute of Gerontology, University of Michigan, Ann Arbor, MI, United States
- 6Division of Psychiatry, Turku University Hospital, Turku, Finland
- 7Graduate Program in Health Promotion, Human Development and Society, Lutheran University of Brazil, Canoas, Brazil
- 8School of Criminology and Criminal Justice, Nipissing University, North Bay, ON, Canada
- 9Department of Health Sciences for Lynn Martin, Lakehead University, Thunder Bay, ON, Canada
- 10Departments of Psychiatry and Family Medicine Larner College of Medicine, University of Vermont, Burlington, VT, United States
- 11Faculty of Education, Althouse College, Western University, London, ON, Canada
- 12LUCAS Center for Care Research and Consultancy & Academic Center for General Practice in the Department of Public Health and Primary Care, KU Leuven University, Leuven, Belgium
The lives of persons living with mental illness are affected by psychological, biological, social, economic, and environmental factors over the life course. It is therefore unlikely that simple preventive strategies, clinical treatments, therapeutic interventions, or policy options will succeed as singular solutions for the challenges of mental illness. Persons living with mental illness receive services and supports in multiple settings across the health care continuum that are often fragmented, uncoordinated, and inadequately responsive. Appropriate assessment is an important tool that health systems must deploy to respond to the strengths, preferences, and needs of persons with mental illness. However, standard approaches are often focused on measurement of psychiatric symptoms without taking a broader perspective to address issues like growth, development, and aging; physical health and disability; social relationships; economic resources; housing; substance use; involvement with criminal justice; stigma; and recovery. Using conglomerations of instruments to cover more domains is impractical, inconsistent, and incomplete while posing considerable assessment burden. interRAI mental health instruments were developed by a network of over 100 researchers, clinicians, and policy experts from over 35 nations. This includes assessment systems for adults in inpatient psychiatry, community mental health, emergency departments, mobile crisis teams, and long-term care settings, as well as a screening system for police officers. A similar set of instruments is available for child/youth mental health. The instruments form an integrated mental health information system because they share a common assessment language, conceptual basis, clinical emphasis, data collection approach, data elements, and care planning protocols. The key applications of these instruments include care planning, outcome measurement, quality improvement, and resource allocation. The composition of these instruments and psychometric properties are reviewed, and examples related to homeless are used to illustrate the various applications of these assessment systems.
The lives of persons living with mental illness are affected by the interplay of a broad range intrinsic and extrinsic factors emerging over the life course. From the earliest stages of life to the person's final moments, these factors can influence opportunities for growth and development, access to resources, engagement in interpersonal relationships, participation in community, and achieving an overall sense of well-being.
Mental health concerns are both pervasive and heterogeneous. Mental illness and substance use disorders are the world's leading cause of disability (1) accounting for up to 32% of years lived with disability and 13% of disability-adjusted life years (2). Unipolar depression was the fourth leading cause of total disease burden in 2000, after perinatal conditions, lower respiratory infections, and HIV/AIDS (3). With many underlying psychological, biological, social, economic, and environmental causes, it is unlikely that preventive strategies, clinical treatments, therapeutic interventions, or policy options will succeed as singular solutions for the challenges of mental illness.
Persons living with mental illness tend to receive services and supports in multiple settings across the health care continuum that are often fragmented, uncoordinated, and inadequately responsive. Needs may remain unaddressed (4) as the person navigates a path to many providers that function in narrowly defined siloes. Patient flow through episodic service environments (e.g., hospitals) may be constrained (5) and access to appropriate services may be delayed due to lengthy waiting lists (6).
To be effective, any health system's approach must include strategies for identifying and responding to mental illness and related dimensions of health and well-being throughout the life course and in all parts of the health care continuum. As such, the ability for different providers, organizations and sectors to communicate with one another is crucial.
This paper provides an overview of the interRAI suite of mental health instruments, which is designed to function as an integrated assessment and screening system to provide a holistic view of the person's strengths, preferences, and needs. It begins with an examination of the range of factors that must be considered beyond psychiatric symptoms in order to support a person-centered approach to shared decision-making. Next, it describes the design, psychometric properties, and applications of interRAI mental health assessments using examples related to homelessness and trans-institutionalization.
Assessing the Bigger Picture: The Need to Look Beyond Psychiatric Symptoms
System integration requires the use of a common language for describing needs, monitoring service use, and tracking outcomes over time. Even within a specific care setting, unstructured narrative charts have little value when they are simply “electronic paper records” (7). Medical charts are incomplete, cumbersome, and overly narrow in their focus (8–10). While natural language processing might reduce the burden of reading volumes of narrative notes (11, 12), unstructured charts often have too many information gaps to have value (13). Standardized clinical assessment data can be more useful if they cover the relevant domains, use psychometrically sound data elements, and can be accessed and interpreted for immediate use.
Some clinicians are reluctant to use standardized assessment tools (14) even though their role in evidence-informed practice is recognized (15). Their reasons might include a perceived lack of benefit over clinical judgment, concerns over psychometric properties, and practicality (14). Even if these issues are addressed to one clinician's satisfaction, inconsistency across settings in the choices of measures prevents communication between organizations serving the same individual.
Given the complexity of mental illness over the life course and the likelihood of engaging with diverse service providers, the data requirements for assessment are not rudimentary. Obviously, psychiatric symptoms, cognitive function, diagnosis, and behavioral issues will be highly relevant in most clinical contexts, particularly during acute episodes or mental health crises. However, if the view is widened to the person's broader experience, ability to function, and overall well-being, more domains must be considered as part of a comprehensive assessment of the “whole person”.
Life Course Perspective: Growth, Development, and Aging
About 20% of the Canadian population experiences problems with mental health or addictions annually and about half will face mental health problems by age 40 (16). Estimates from New Zealand's Dunedin cohort studies suggest that only 17% of the population will have enduring lifetime freedom from mental illness (17). This is explained, in part, by factors at both ends of life.
About 70% of mental health problems are reported to begin in childhood or adolescence (18), many of which persist over the lifetime (19, 20). Early childhood development sets the stage for social and emotional functioning, academic achievement, and interpersonal relationships (21, 22). As an individual's identity develops in adolescence (23), social networks and peer pressure (24), substance abuse (25), parenting styles (26), and school performance come into play (27). Age-based service restrictions mean the transition from youth to adulthood is often met with fewer mental health resources (28).
Aging is associated with the onset of a number of conditions that can affect mental health, with dementia as the most obvious example (29, 30). Mental illness may accelerate the aging process resulting in shorter life expectancy and greater years of life lost compared with the general population (31–33). Some, but not all, of this is attributable to deaths by suicide (34). Other causes include higher smoking rates leading to increased cancer and cardiovascular disease (35, 36) and higher rates of diabetes arising from poor diet, physical inactivity, and drug related side effects (37). Persons with psychiatric diagnoses who do survive to later life have greater odds of being the most frail (38).
Physical Health and Disability
Physical comorbidities are often neglected needs in persons living with mental illness (39, 40). Health care providers are commonly divided according to whether they provide physical health or mental health services. This bifurcation occurs to the detriment of persons living with mental illness who tend to receive lower quality medical care for physical health problems (33, 41). For example, mental health problems are often overlooked or stigmatized in acute hospital settings (33), cancer screening tends to be inadequate (42), and effective therapeutic regimens are less likely to be received by patients with mental illness who have acute myocardial infarction (43). In mental health settings, treatment of physical health problems is often delayed or inadequate (44, 45), and health promotion may be regarded as a low priority (46).
Mental health issues may be both consequences and causes of physical disability. A four-nation study showed that impaired physical functioning and dual sensory loss were associated with greater odds of depressive symptoms in home care clients (47). Suicide attempts may cause disability among survivors, but also there is increased risk of suicide among persons living with physical disabilities (48–51) for reasons that include pain and a sense of burden to others (52, 53).
Social isolation and loneliness are increasingly recognized as important risk factors for the physical and mental health of persons of all ages (54–56). For example, social support networks may play an important role in facilitating recovery from mental health problems (57) and they may provide instrumental support with tasks such as child care, transportation, or medical management (58). On the other hand, mental health problems may pose barriers to forming or maintaining close personal relationships over the life course (59, 60). In addition, social relationships may be a source of stress, trauma, or abuse (61–63) that can have immediate and long-term consequences for mental health. Hence, interventions must consider both the social resources that may be beneficial and relationships that may be detrimental to the person's well-being.
The pathways between poverty, deprivation and mental illness are multifold. Low income households are associated with higher rates of depression and antisocial behavior in children, but the impact may lessen if household resources improve (64). Job loss may pose tremendous stress on individuals and families resulting in worsened physical and mental health (65). However, for persons with pre-existing mental illness, unemployment and poverty are pervasive (66) resulting in reduced access to adequate food, shelter, clothing, and medical treatment. The stigma associated with poverty and mental illness (67, 68) may be further magnified in low resource nations (69).
Homelessness and Housing
Stable housing is an important prerequisite for recovery. Persons who lack stable housing, particularly the homeless, commonly experience higher rates of physical and mental illness, poor quality of life and high mortality (70–74). At the core of homelessness are the processes of marginalization and exclusion, with multiple risk factors driving these experiences (75–78). For instance, persons with childhood trauma, mental illness, and substance use are at greater risk for housing instability and homelessness (79). Homelessness arises from an interplay of individual vulnerabilities, interpersonal, structural, and systemic factors. Individual factors consist of psychosocial vulnerabilities connected to family background, health, and trauma. Interpersonal factors are linked to behaviors like substance use. Poverty, unemployment, and lack of affordable housing are examples of the structural factors; lack of social security of the systemic ones.
The relationship between substance use and mental illness is multifaceted. Based on a recent meta-analysis, more than half of persons with mental illness have a comorbid substance use disorder (80). This comorbidity leads to poorer clinical outcomes, worsened physical and mental health; higher levels of disability; increased risk of suicidal behavior (81), homelessness, and psychiatric readmissions (82); and greater difficulties in interpersonal and family relationships (80). Symptoms of mental illness may interfere with substance use treatment and the reverse may also be true (83).
Proposed mechanisms for this comorbidity include self-medication where people use substances to treat their own mental health symptoms; the reverse-causal pathway where substance use causes or worsens the symptoms of mental illness or side effects of substances produce symptoms similar to mental illness; and a shared environmental or genetic vulnerability where factors like poverty or childhood trauma increase risk for both (84). Considering the complexity of these relationships, comprehensive assessment is needed to understand the context of substance use in relation to symptoms of mental illness.
Trans-Institutionalization and the Criminal Justice System
Deinstitutionalization of mental health services began in Canada and other countries in the 1960s–70s with the rapid closure of psychiatric hospitals followed by further reductions in general hospital admissions for mental health in the 1990s (85). In most countries, the motivations for this change included breakthroughs in pharmaceutical treatments (86) combined with an emphasis on human rights and social inclusion of persons with mental illness (87), and the expectation of better outcomes in the community (88). However, legal concerns and fiscal decisions aimed at reducing health care expenditures (85, 89) were also important drivers. Globally, there is considerable variability in rates of deinstitutionalization (90, 91) despite broad based philosophical support (92, 93).
Deinstitutionalization is associated with improved life expectancy for persons with mental disorders (94), disability reduction (90, 91), better quality of life, and greater autonomy (95). Adverse consequences include a reported rise in trans-institutionalization, homelessness, and criminalization of persons with mental illness (96–99). While there is debate about the extent to which deinstitutionalization has resulted in negative outcomes (100, 101), contact with the criminal justice system is now a common experience for persons with mental illness.
Police have become the first line of contact for many persons living with mental illness (102, 103), with substance use and impulse control issues as common causes (104). In some jurisdictions, the large-scale expansion of inpatient forensic mental health services has been driven by increased admissions of younger individuals from ethno-racial minorities with low levels of violence and substance use offences (105). Specialized mental health courts provide alternative mechanisms for dealing with offenders with mental illness (106), and more advanced analytics of data based on high quality assessments will be essential for the improvement of forensic mental health services (107).
The shift to community-based mental health services has been accompanied by a transition from a biomedical care model to one that emphasizes personal recovery (108–110). While there is no common global definition of recovery (111), there are agreed-upon common core principles. For example, that mental health is viewed in terms of a continuum of functioning; recovery may be an iterative or non-linear process; and belonging, community engagement, hope, connectedness, identity, freedom from stigma, meaning in life, and empowerment are key markers to consider (108). Providing a supportive environment that builds on strengths and is calibrated to needs is a priority, rather than stopping services (112). Treatment of symptoms is important, but “clinical recovery” is either not enough or not fully possible in some cases. Therefore, assessment must consider the person in a broader social context to reach beyond symptoms alone.
Stigma associated with mental health problems often hinders full citizenship and inclusion in society (113, 114). At the societal level, public stigma includes labeling persons with mental illness as dangerous or less competent than the general population. At the individual level, self-stigma occurs when persons apply psychiatric labels to their identity, often with negative consequences. Those who experience perceived stigma or discrimination may apply stereotypes to themselves resulting in underestimation of their own potential and a sense of helplessness (115). This may lead to avoidance of necessary care, increasing problems in multiple domains, and worsened prognosis (116, 117).
Conglomeration Versus Integration of Assessment
If we accept that standardized assessment has value, we must consider how to define that standard. In many health care settings, usual practice has been to create a conglomerated information system made up of home-grown intake forms combined with a collection of clinical scales used to measure singular issues (e.g., cognition, depression) on an “as-needed” basis.
Data from these conglomerated systems are typically not automated and forms completed separately are not linked. For the service recipient, this can lead to repetitive measurement of the same issues and limited access to clinically relevant information for team members in the circle of care. Longitudinal case histories cannot be constructed easily, linkages between clinical issues tracked on different forms may go unrecognized, and person-level analytics cannot be readily applied to the clinical record. Data compiled this way have little utility for population level analyses because converting them into a complete, electronic database is a daunting task. Handwritten notes on incomplete forms stored in locked filing cabinets serve little purpose beyond single-case after-the-fact chart audits.
Even if data are automated, when the choice of which scale to use for a particular concern is left to individual clinicians, there will almost certainly be no measurement consistency at the organization or system levels. The Handbook of Psychiatric Measures (118) contains 26 chapters with over 275 distinct psychiatric measures, and that list was almost surely incomplete at the time of publication. Assuming one could resolve the controversy over whether the Montreal Cognitive Assessment should be used in place of the Mini Mental State Examination (119), what is the likelihood of a consensus on which of the nine depression measures listed in the Handbook should be used with all patients, in all settings, during all episodes of care? At the population level, inconsistent data have diminished value. Imagine the response by an organization ranked negatively in an outcome-based quality indicator where different providers use different tools. The fact that an indicator was “not measured the same way” across settings will be the immediate defense used to negate the validity of any differences in performance suggested by the indicator.
Using specific scales on an “as-needed” basis is also a flawed strategy. We do not always know who has a given problem without asking. Indeed, one of the values of comprehensive assessments is that they force the clinicians to ask about underlying issues that go beyond the immediate, presumed problem. Persons with substance use problems are not only found in substance use programs; depression affects people other than those in a mood disorder program; cognitive impairment spans the barriers of a dementia unit. Under-detection of issues like pain, depression and dyspnea is an important problem throughout the health care system (120–122).
If we succeeded in achieving a consensus on a set of tools to measure psychiatric symptoms using a collection of single-focus measures, the data volume and time to complete assessment could still be unwieldy. For example, a set of 15 well-known mental health instruments would measure cognition, (123), delirium (124), depression (125), anxiety (126), psychosis (127), mania (128), trauma (129), pain (130), sleep (131), behavior (132), substance use (133), instrumental activities of daily living (134), functional status (135), suicidality (136), and aggression (137). Despite comprising more than 400 different items, this set does not address many of the key issues highlighted previously.
Further, intellectual property and licensing requirements may be difficult to resolve. Within a single clinical setting the task of securing permissions for 15 or more different instruments would be a challenge, but to do so on a national level is impractical. This would be further complicated by electronic health record vendors licensing requirements, as well as the challenge of training clinical staff on coding conventions that likely differ dramatically among instruments. Finally, though the conglomerated set of assessments may measure the severity of an issue at a given time and can be used on a repeated basis to monitor changes, these measures generally do not invoke a clinical response in reaction to their numeric scores.
The alternative to a conglomeration of stand-alone tools is to use a single, integrated assessment system that can be applied across care settings to persons of any age. The system should serve multiple applications for multiple audiences, including triggering a clinical response leading to shared decision-making in support of key recovery goals.
The interRAI Suite of Mental Health Instruments: An Integrated Assessment System
interRAI (www.interRAI.org) is a not-for-profit network of over 100 researchers, clinicians and policy experts from over 35 countries in North and South America, Europe, Asia, Africa and Oceania (138–140). It was founded with an initial focus on geriatric research in the early 1990s, but its scope broadened to include vulnerable persons of all ages. interRAI's multinational program of research aims to develop and implement comprehensive assessment and screening systems to support improved quality of care and quality of life among persons of all ages with complex needs across the continuum of health and social service settings. This includes the creation of psychometrically sound measurement systems, the application of data from those assessments to support multiple uses by multiple audiences, and completion of multinational comparative analyses and “natural policy experiments.”
The initial innovation behind interRAI assessments is that they were designed not only to describe status or severity at a given time, but also to invoke a clinical response through the use of embedded triggering algorithms and care planning guidelines (141).
The first interRAI instruments were designed to be used in a single sector at a time (142, 143), and the release of interRAI's mental health instrument for in-patient psychiatry (144) represented its first effort to target the general adult population. In 2000, interRAI launched a major effort to redesign all of its assessment instruments to function as a fully integrated suite of measures (145–148). The most recent developments by interRAI include the creation of a parallel suite of instruments for children and youth (149, 150), screening systems for use by non-health professionals (151), and a set of self-report tools to measure patient experience (152–154) and patient reported outcome measures(155, 156).
The interRAI suite of assessment instruments includes over 20 comprehensive assessments, supplementary assessments, and screeners. All of these systems include indicators related to mental health (particularly cognition and depressive symptoms). However, the focus in the following section will be on the adult versions of the mental health instruments using homelessness to illustrate how they can provide insights about strengths, preferences, and needs.
Assessment and Screening Instruments for Adults
Mental Health Settings
The interRAI Mental Health (MH) assessment (157, 158) supports care plan development in 20 domains, and is used in in-patient mental health settings at admission, discharge (if more than 7 days after admission), every 90 days (for long-stay patients), and when there is a clinically significant change in the person's status that is not a self-limiting and could require modifications to the care plan. The instrument is available in English, Canadian and Belgian French, Flemish, Icelandic, and Finnish.
The first version of the instrument was released in 1999, with a major update in 2002. The most recent version 9.1.2 was published in 2012 (157) with revisions designed to make it fully compatible with the interRAI suite (147). The MH was pilot tested in Nordic countries and the US, but the primary implementation has been in two Canadian provinces (Ontario, Newfoundland and Labrador). Local implementations have occurred in two other provinces (Quebec, Manitoba). To date over 1.4 million assessments have been completed on over 320,000 unique individuals in Canada.
The target audience is adults aged 18 years and older, including acute, long-stay, forensic and geriatric patients. Canadian provinces have also implemented the MH in adult-designated beds, even if those are occupied by persons under 18 years of age. This proved to be a helpful source of data for publications that pre-dated the completion of the child-youth suite of mental health instruments (159).
The assessment is comprised of 396 items in several domains (see Table 1). It is mainly completed by mental health professionals (typically nurses or social workers), and includes a limited number of self-report items. Most items employ a standard 3-day look-back period, although service use and therapies use a standard 7-day window. Other items use 30-day, 90-day or lifetime estimates depending on the nature of the issue. The average time to complete the assessment is 1 hour, and it can be used as a replacement for a standard nursing intake assessment. The MH instrument deals with items that usually would be addressed in mental health assessments, but also expands the scope to look at broader issues within a time frame for completion that is consistent with conventional practice. It contains about the same number of items as the hypothetical conglomerate of stand-alone tools described earlier, but it substantially increases the scope of issues addressed.
Table 1 Item counts by domain area and interRAI mental health system for adult populations (age 18+ years).
Community Mental Health
The interRAI Community Mental Health (CMH) assessment (160) supports care plan development in 20 domains. It is designed for community agencies employing mental health clinicians, including those with case managed mental health services and assertive community treatment programs. The assessment can be used at intake, discharge, every 6 months depending on length of stay, and after a change in the person's status that requires care plan modification. If the person is admitted to the community agency from an inpatient setting using the MH, the discharge MH assessment is shared to support continuity of care (161) to allow additional time for community staff to establish a clinical relationship with the person. The target audience for the CMH is also adults aged 18 years and older, including a broad range of service recipients. Although community mental health agencies are the typical service provider, it has been used in consultation liaison services for long-term care to deal with psychiatric and behavioral issues outside the usual scope of practice for clinicians in those settings as well as a Dutch study of homeless services recipients (162). The original version of the CMH released in 2007 was pilot tested in Ontario, Canada, New York State, Finland, Belgium, Chile and Hong Kong. The most current version 9.2 published in 2012 is compatible with the interRAI suite. Newfoundland and Labrador have begun provincial implementation of the system, as well as regions in Ontario and Quebec. The Swiss Home Care association has announced plans for national implementation. The instrument is available in English, French (Canadian, Belgian, Swiss), Swiss German, Swiss Italian, Flemish, Finnish, and Chinese. For this paper, 12,862 assessments from pilot studies are available for analyses. Because of constraints related to European data protection laws, it was not possible to complete pooled analyses with those data (162).
The CMH includes 405 items dealing with comparable domains and look-back periods as used in the MH. The two instruments share 330 common items, but the CMH has 75 items not present in the MH and the MH has 66 items not present in the CMH. The main differences relate to issues that are encountered in one, but not the other, service environment. For example, the CMH includes an expanded range of items on social relationships and supports in the community, and on the home environment. The MH includes items on control interventions (e.g., restraints) not used in community settings.
The average time to complete the CMH is also an hour, but completion time may be affected by the lack of access to another informant for persons who have difficulty responding. This issue poses a greater challenge than noted in the interRAI Home Care assessment (163) where caregivers are routinely available as additional sources of information.
Emergency and Crisis Services
The interRAI Emergency Screener for Psychiatry (ESP) (164) is designed for general emergency departments, psychiatric emergency departments, and mobile crisis teams. Like the MH and CMH, the ESP is typically completed by nurses, social workers, or clinicians other than psychiatrists. It is used at the time of crisis/emergency, with the expectation that a follow-up assessment for persons remaining on service would revert to the MH or CMH once the person is stabilized.
The first version of the ESP was pilot tested in Ontario in 2004, and the interRAI suite version 9.1 was published in 2011. The ESP's 141 items are a subset of those in the MH and CMH. The instrument has been pilot tested in Ontario and Quebec, Canada and Finland. Two regional mental health services in Canada have begun implementation of the ESP and a child-youth variant of the instrument. The current data holdings available at the time of writing comprised 5,249 ESPs completed in Canadian organizations.
The target audience for the ESP is also adults aged 18 years and older; however, the clinical focus of the ESP is narrower. Whereas the MH and CMH focus on care plan development in 20 areas, the emphasis of the ESP is on patient safety issues and acute symptoms. Thus, the ESP has a 24-hour look back period to focus on immediate presenting concerns. It triggers care plans in three areas of safety (harm to self, harm to others, inability to care for self), and in substance related withdrawal symptoms.
The average time to complete the ESP is 30 min; however, the acute nature of illness may pose barriers to continuous completion of the assessment if other informants are not available.
Police, Emergency Medical Services, and Other Non-Mental Health Settings
The interRAI Brief Mental Health Screener (BMHS) is a short screening tool intended to be used by non-mental health professionals who may be the first line of contact for persons with mental health needs (165). The initial target audience for the BMHS is police services (151) with the aim of facilitating improved communications among polices officers, emergency mental health services, and community mental health agencies (166). The BMHS is a 46-item subset of the ESP, but the training manual includes additional materials for training non-mental health staff in the use of appropriate terminology to describe the person's presenting symptoms. It is designed to be used at a single time point, and it employs a 24-hour look back period.
The first draft version of the BMHS was developed using analyses of 41,019 MH admission assessments to determine the minimum subset of items needed to identify persons who would be admitted to inpatient psychiatry due a combination of disordered thought and danger to public safety (167). The draft BMHS was tested with two police services and five hospitals in Ontario in 2011. The finalized version 9.3 of the BMHS was published in English and French in 2015, and the screener is now in extensive use by over 40 police agencies (local, provincial, and federal) in four Canadian provinces. A pilot study of the BMHS in one US region is expected to launch in 2020. For the purposes of the present paper, 70,005 Canadian BMHS assessments were used.
Unlike the previous assessments, the BMHS is not a care planning tool. It is designed to provide a systematic means of summarizing the observations of police officers (and other non-mental health staff) using the same items that are employed in the interRAI mental health suite. It can also be considered a mental health training intervention, because the screener provides guidance to those using the instrument on how to identify and describe mental health symptoms. The BMHS includes three patient safety algorithms related to harm to self, harm to others, and inability to care for self based on machine learning analyses of MH and BMHS data (168). These algorithms help police and others to communicate acuity to hospitals and community mental health agencies in real time. The Ontario Human Services and Justice Coordinating Committee recommended the province-wide use of the BMHS in 2019 in order to facilitate more timely and appropriate transfers of the care of persons with mental illness from police control to mental health agencies (169).
There is considerable potential to employ the BMHS in other settings. Emergency Medical Services are also in frequent contact with persons with mental health crises in the community and face similar decision-making challenges. In addition, a non-police version of the BMHS is available that could be used with peer-led agencies, shelters, food banks, or other settings for persons with mental illness.
Forensic and Addictions Services
The interRAI MH and CMH instruments both address a broad range of adult service recipients, including persons with addictions and those in contact with the criminal justice system (170–172). However, in forensic and addictions programs, there may be value in obtaining additional information about the severity of the problem (e.g., criminal convictions) or items that would not be asked routinely for all service recipients (e.g., readiness for change, health symptoms associated with substance abuse, dynamic and static forensic risk factors).
To that end, interRAI has two new supplements for the MH and CMH to expand depth in criminal justice and addictions with 33- and 35-items, respectively. These items provide additional severity measures, risk algorithms and specialized care planning triggers for these clinical subpopulations only. At the time of writing, both supplements are in beta versions, with expected publication in 2020.
Non-Mental Health Settings for Adults
All interRAI assessment and screening systems, including those for non-mental health settings, have at least a core set of mental health items. A brief description of some of the main adult instruments follows with examples of mental health research done in those settings.
The interRAI Intellectual Disability (ID) assessment (173, 174) is used with adults aged 18 years and older with intellectual and developmental disabilities (e.g., Down syndrome, autism) in community and residential services. It comprises 320 items, including 188 in common with the interRAI MH, and it is usually completed by developmental services workers. The ID includes support planning protocols for abuse by others, communication, continence, injurious behavior, meaningful activities, mental illness, and social relationships (175). The mental health content of the ID includes measures of psychological well-being, stressful life events and trauma, cognition, psychosis, extrapyramidal symptoms, mood, anxiety, negative symptoms, sleep disturbance, behaviors, violence, and previous psychiatric hospitalizations. These indicators have been the focus of a number of studies in ID settings (176–180) as well as cross-national comparison of persons with ID (181).
The interRAI ID has been implemented in the state of New York (USA) and in Prince Edward Island (Canada). It was used in Ontario, Canada to support the closure of the province's last three large institutions for persons with intellectual disabilities (179, 180, 182–184). Several other jurisdictions in the USA and Canada have announced plans to implement the ID. ID supplements to the MH and CMH instruments are currently being pilot tested in Ontario and in Belgium.
All interRAI mental health assessments include items on intellectual disability, since persons with dual psychiatric and ID diagnoses are an important subpopulation in mental health settings. Several interRAI papers have examined persons with intellectual disabilities in trans-institutional settings (179, 185–189).
Home Care and Nursing Homes
interRAI's Long Term Care Facility (LTCF) and Home Care (HC) instruments (142, 143, 163, 190, 191) were first developed more than 25 year ago. By the end of 2018, over 9 million interRAI home care and nursing home assessments had been completed in Canada alone. In the US, since 1990, over 100 million interRAI nursing home assessments have been completed. The use of LTCF and HC is worldwide; other nations with large scale implementations of one or both of these instruments include Iceland, Finland, Belgium, France, Switzerland, Italy, Hong Kong, and New Zealand.
The LTCF and HC contain 322 and 318 items, respectively, and are completed by health professionals (typically nurses or social workers). The mental health items assessed in both instruments include cognition, delirium, mood, behaviors, psychosocial well-being, psychosis, alcohol and psychotropic medications. The 27 care planning protocols triggered by the HC and LTCF include ones dealing with cognitive loss, delirium, mood, behavior, abusive relationships, tobacco and alcohol use, and appropriate medications (192).
The mental health research in long-term care with the LTCF and HC includes depression (193), bipolar disorder (194, 195), suicidal behaviors (196, 197), traumatic brain injury (198–200), delirium(201, 202), aggressive behavior (203–205), and cognitive impairment (206–210).
Two quality of care issues of considerable interest are use of physical restraints (211–218) and potentially inappropriate use of antipsychotics (211–214, 219–225). The Canadian Institute for Health Information (CIHI) now reports interRAI's risk adjusted quality indicators for restraint and antipsychotic use in nursing homes in seven provinces/territories. This will expand to all other jurisdictions except Quebec once current implementations are complete.
Instruments for Children and Youth
The newest instruments in the interRAI mental health suite are those designed for children and youth. Implementation of the interRAI MH in all adult inpatient beds in Ontario was mandated in 2005 in response to recommendations from the province's Joint Policy and Planning Committee (JPPC) of the Ministry of Health and Long Term Care and the Ontario Hospital Association. The JPPC also called for development and implementation of a compatible system for children and youth. In response, interRAI researchers developed the interRAI Child/Youth Mental Health (ChYMH) assessment system for children aged 4–18 in mental health settings (226) as well as a shorter screener (227).
The ChYMH has 382 items and 31 summary scales (e.g., internalizing, externalizing, distractibility and hyperactivity, aggression, anxiety, social disengagement, depression severity index), and risk assessments to inform the intensity and nature of the child or youth's service needs (149, 159, 228–236). Additionally, the ChYMH has 30 care planning protocols and a preliminary system to describe resource intensity (237). An Adolescent Supplement covers items not addressed with younger children (e.g., substance use) and a Youth Justice Supplement is for youth in correctional settings. A variant of the ChYMH is also available for children/youth with intellectual and developmental disabilities (238). Finally, a new instrument for newborns to three-year olds is nearing completion (239).
The interRAI ChYMH has already been adopted by 90 children's agencies in Ontario, and three other provinces have expressed interests in adopting the system. In addition, there were 16,588 MH assessments on children aged 13–17 in adult mental health beds in Ontario by December 2018.
The availability of an integrated suite of mental health assessment systems that span the life course from newborns to centenarians provides enormous potential to improve care with various transitions that occur over the life span. In addition, at the person-level these compatible assessment systems can provide a rich continuous clinical picture of the person's growth, development and aging from a comprehensive perspective. At the population level, the large-scale implementations of these instruments portend the emergence of a new, detailed longitudinal database of large cohorts of individuals living with mental illness in the earliest stages of their lives until the latest stages and end of life.
What Makes interRAI an Integrated Mental Health Information System?
The following factors make interRAI's mental health assessments and screeners an integrated health information system that spans the continuum of care for persons of all ages. In this and subsequent sections, interRAI data holdings are used to illustrate concepts discussed below. Table 2 provides a summary of the data sources used in these examples.
All interRAI instruments use common terminology to define the same items across all settings. Mood, psychosis, cognition, pain, function and physical health systems use the same definitions, inclusion/exclusion criteria, and phrasing wherever they appear irrespective of the type of setting. Items in the child/youth instruments only differ from the adult instruments when a developmental rationale requires the difference (e.g., performance of activities of daily living). Also, most instruments employ a standard 3-day look back period; except the ESP and BMHS use a 24-h look back due to the acute nature of the conditions addressed.
Common Conceptual Basis
All interRAI assessment instruments are designed first and foremost to support care planning, by embedding data elements where algorithms trigger the need for further intervention in a given area. The development process for instruments always begins with identifying the focal domains for care planning and then identifying the minimum item set to trigger the need for additional investigation. Secondary consideration is given to items that are not used for care planning, but could be used for outcome measurement, resource allocation, of quality management. Specific items to only track matters of research interest are not included in these systems.
Common Clinical Emphasis
The primary focus of interRAI assessments is on function and symptoms rather than diagnosis. These instruments track provisional and finalized psychiatric and medical diagnoses and they provide rich clinical information that is relevant to diagnosis, but they are not intended to be diagnostic systems. Rather they focus on what are the person's strengths and needs, how the person relates to others and engages in the community, what the person can do independently, and where support needs are required.
Common Data Collection Methods
All interRAI systems employ a common and consistent assessment methodology. For the clinician-administered instruments, the assessor is provided with specific item definitions, time frames, inclusion/exclusion criteria, lists of examples, coding guidelines, and instructions for the assessment approach. The assessments do not use a fixed narrative set of questions, and the order of completion can be adapted to the natural flow of the assessment process on a case-by-case basis. Assessors are aware of the information they need to acquire for a given item, but they are free to adapt approaches to each item to be culturally or gender-appropriate. Assessors use all sources of information available to them, including direct questions posed to the person, observation of the person's behavior and mannerisms, interviews with family or other members of the support network (where appropriate), information exchanges with other members of the circle of care, and medical or other records. When these information sources provide conflicting evidence, clinicians exercise their best judgement to record what response they believe to be the most correct answer for a given item. It is also possible to parcel out portions of the assessment to different staff members. For example, clerical staff could readily complete administrative and tracking elements or data on historical service utilization. Other mental health disciplines may complete some but not all portions of the assessment. For auditing practice, one individual must sign-off that the assessment is complete and as accurate as possible given available information.
This assessment approach is standardized for all interRAI systems and shared items are assessed in the same manner. What differs among instruments is the curated item sets and associated scales and algorithms. Thus, information between sectors can be compared directly, and staff with experience in interRAI systems in one sector can easily learn another interRAI instrument if they move to another sector. While items sets and clinical issues may be new, the approach to completing items remains consistent.
Common Core Elements
Mood, pain, cognition, and functional status are basic human characteristics that are relevant in any setting and any age group. In that light, there is a set of common items that are measured in almost all instruments. The main exceptions are screeners that are intended to use a reduced item set for limited targeting purposes, and the newborn to three year-old instrument that has items dealing with earliest stages of human development.
This consistency of measurement allows for the examination of some clinical issues at the population level. For example, Figure 1 shows the distribution of interRAI's Cognitive Performance Scale for about 1.7 million individuals across a variety of adult service settings in Canada (see Table 2 for data sources). Scores range from zero for cognitively intact to six for very severely cognitively impaired. There is a clear transition from less to greater cognitive impairment as one moves from populations whose usual residence is in the community to facility-based settings. The most severely cognitive impaired population is nursing home residents with a previous history of psychiatric hospital admissions. The third most severely impaired population is inpatient psychiatry patients who were admitted from nursing homes. Both of these trans-institutional populations have much more severe levels of cognitive impairment than the general inpatient psychiatry population.
Figure 1 Percentage distribution of cognitive performance scale scores across Canadian care settings.
Common Care Planning Protocols
interRAI's clinical assessment protocols are designed to inform the care planning process for issues identified in the assessment. These protocols are grouped according to clusters of care settings serving persons with common clinical issues who may nonetheless be at different points in the treatment trajectory. The MH, CMH, and ESP are adjacent sectors of the mental health system that use the same set of mental health protocols to address the needs of their shared populations. Similarly, the LTCF, HC and interRAI Community Health Assessment share common protocols for a population that is generally affected by geriatric issues or issues related to physical disability paired with medical complexity and/or cognitive impairment. The protocols for supporting adults with ID can be used in residential and community settings, and the ChYMH instruments have shared protocols for community and facility based settings for children/youth.
Psychometric Properties of interRAI Mental Health Instruments
One of the main value propositions for adopting research-based assessment systems over “home grown” bespoke forms is that decision-makers must have confidence in the veracity of the findings generated by the assessment. The data must be meaningful, accurate, and relevant to the issues affecting the person who has been assessed. Reliability and validity are two basic psychometric properties to be considered and the cornerstone for instrument development (240).
In any health care system it is essential that independent assessments by different trained clinicians yield consistent answers. To be cost-effective and to minimize assessment burden, it should be feasible to use assessment information completed by another colleague (including from a different care setting) to inform our own clinical decision-making process.
To appraise this, interRAI has a long history of engaging in inter-rater reliability testing of its assessment systems (142, 143, 191, 241–248). The approach used has been to conduct independent assessments of service recipients without contact between the assessors and with a limited time gap between assessments. This conservative, yet pragmatic, approach mimics what may occur in usual practice in health care settings. Although it is convenient to use vignettes in reliability research they are avoided because they do not capture the nuances and complexities of “real life”. Similarly, assessors do not work together in pairs because that approach violates assumptions of independence between observers (249), which could inflate reliability estimates and does not reflect usual practice patterns in health care.
Recognizing that interRAI assessments share a common core of data elements that are combined with specialized items unique to a given sector, it is feasible to use inter-rater reliability evidence from multiple sectors to evaluate instrument performance. In a 12-nation study of inter-rater reliability of five different interRAI instruments (MH, LTCF, HC, Palliative Care, and interRAI Post-Acute Care), the mean weighted kappa for the core items common to all instruments was 0.75 and the kappa of the specialized mental health items was 0.64 (145). Both results suggest “substantial” inter-rater reliability (250).
Challenges with inter-rater reliability studies are that they are expensive, intrusive to the person being assessed twice, and of no clinical benefit to that person. Therefore, it is useful to employ other statistical approaches to reliability evaluation that can be done continuously without additional assessment. When instruments use parallel form scales to measure underlying domains with multiple items, it is possible to compute measures of internal consistency using Cronbach's alpha (251). This approach is used to evaluate new scales developed for the interRAI suite, but it can also be deployed to monitor data quality continuously with large scale implementations as has been done with other interRAI assessments (252, 253).
Table 3 describes several clinical summary scales that are available in the interRAI mental health suite and Table 4 shows the internal consistency results for seven parallel form scales from four interRAI mental health instruments in three countries. The scales include symptoms typically of interest in mental health settings; however, the two measures of functional status (activities of daily living (ADL) and instrumental activities of daily living (IADL)) are relevant to understanding disability in the assessed populations. Virtually all scales in all settings examined in Table 4 meet or exceed the standard alpha value of 0.70 for good internal consistency and, in several instances, exceed 0.80 indicating excellent reliability. Good reliability is evident in both acute and longer-term service settings. In addition, the Positive Symptoms Scale, as assessed by police officers, has comparable reliability to that found when administered by mental health clinicians. The two exceptions in performance are the Aggressive Behavior Scale (ABS) in US community mental health settings and the Mania Scale in Canadian ESP settings. The US findings may be a function of an attenuated distribution of the ABS in the US pilot site (see Table 5). Site specific analyses of the Mania scale performance indicate that the problem was with one ESP pilot site, suggesting that training may have been a concern. Put differently, 39 of 41 evaluations of scale reliability in diverse mental health settings met or exceeded standard cut-points for good reliability.
Table 4 Internal consistency of clinical scales derived from interRAI Mental Health Instruments, by country.
Table 5 Univariate distributions of selected clinical scales derived from interRAI instruments by country.
Validity is a more complex psychometric issue than reliability, requiring evaluation through a number of methods. The key questions of interest include: does this item or scale measure what I think it measures? Does the assessment address the important characteristics affecting the person's well-being? How does the instrument compare with other widely used systems? Do risk indicators actually predict future outcomes of interest?
Face and Content Validity
Face and content validity are necessary, but not sufficient criteria to meet for developing an assessment system. The interRAI development process typically addresses these through extensive involvement of front-line practitioners and researchers combined with reviews of the current literature. For example, as part of the development effort for the care planning protocols, key informants from several nations were asked: a) How consistent is the MH-CAP with the Recovery Model as used by your organization? (83% rated the protocol reviewed to be mostly or completely consistent); b) How consistent is the MH CAP with guidelines/best practices used by your organization? (92% rated them as mostly or completely consistent); and c) How would you rate this CAP in terms of its use as a support for care planning in this area? (90% rated them as good or excellent). In addition, critical feedback from informants was used to inform final revisions to the penultimate versions of these care planning protocols.
The patterns of associations in a dataset can provide insights about whether the instrument measures what one thinks it measures. The initial version of the MH was successfully evaluated by examining the presence of expected associations with age of cognitive loss and functional decline, the relationship between depression measures in suicide related indicators, readmission rates and problems with medication management, and cognitive performance with behavior (248).
As an extension of this approach, one might examine the extent to which hallmark clinical symptoms of various psychiatric diagnoses are associated with the presence of those diagnoses in the assessment data. A clear positive association would provide reassurance that the instrument measures symptoms that have meaning to clinicians. Table 6 shows the relationship that could be expected between various mental status indicators and provisional diagnoses of neurocognitive disorders; substance related and addictive disorders; schizophrenia spectrum and other psychotic disorders; and depressive disorders. For the first three diagnoses in all three settings, the anticipated relationships with symptoms are strong, in the appropriate directions, and the c-statistics are generally at the 0.80 level or higher. For depression diagnoses, the odds ratios for the depressive severity index and social withdrawal index are in the appropriate directions, but the associations with depressive symptoms are stronger than with social withdrawal (which may relate to other diagnoses). The c-statistics for depression diagnoses are between 0.64 and 0.70.
Table 6 Odds ratios (95% CL) for provisional psychiatric diagnoses by associated symptoms and setting, Canada.
A common approach to validation is to compare a new instrument to another scale that is recognized as a “gold standard” measure. The challenge in mental health is that few measures are universally accepted as a gold standard, and biological markers are not particularly useful as they might be in somatic illness. Previous interRAI research has established the criterion validity of the following scales and items: pain scale vs Visual Analogue Scale (254); Cognitive Performance Scale versus Mini Mental State Examination and Montreal Cognitive Assessment (255, 256); Depression Rating Scale versus Hamilton and Cornell Scales and psychiatrists ratings (257); Aggressive Behavior Scale versus Cohen Mansfield Agitation Inventory (203); delirium algorithms versus the Confusion Assessment Method (258); recorded diagnoses versus acute hospital medical records (259); and mental health care planning triggers versus clinician judgement (260). In a developmental project to refine an earlier version of the interRAI MH, criterion validity studies with 876 patients in 11 psychiatric hospitals/units yielded the following correlations: Aggressive Behavior Scale with Neuropsychiatric Inventory Total Score r = 0.50 and with PANSS Aggression Risk Profile r = 0.58; Pain Scale with McGill Pain Questionnaire r = 0.64; Positive Symptoms Scale with PANSS Positive Symptoms r = 0.62; Negative Symptoms Scale with PANSS Negative Symptoms r = 0.49 (261).
Arguably the most important (and difficult) form of validity to establish for an assessment system is predictive validity. Presumably, the ultimate purpose of assessment is to guide interventions that will have an impact on a future clinical trajectory of change. This approach was used extensively to identify triggering rules for interRAI's mental health care planning protocols (see discussion below). Examples of publications reporting on this type of validity include studies of inpatient violence (262), re-hospitalization (82); and suicide behaviors (196, 197).
Applications of interRAI Instruments
Critically, all interRAI assessment instruments are designed to be used as part of normal clinical practice to serve multiple functions for multiple audiences (263), including: a) care planning; b) outcome measurement; c) resource allocation; and d) quality improvement. In addition, the aggregated data compiled as part of regular use can be employed for policy analysis, evaluation and research. Examples of peer-reviewed health services research based on interRAI MH data include: mental health needs of prisoners (264, 265); mental health service planning (266); access to psychiatrists by linguistic minorities (267) and nursing home residents (268); use of ECT by psychiatric inpatients (269); prolonged stays (270, 271); length of stay (272, 273); and restrictions in movement in forensic patients (172). Examples of clinically oriented research with these data include studies of: sexual dysfunction (274, 275); incontinence (276, 277); discharges against medical advice (229, 278); medication non-adherence and misuse (229, 279, 280); restraint and acute control medication use (281); and pharmacoepidemiological studies (282).
The Clinical Assessment Protocols (CAPs) associated with the suite of interRAI assessments are care planning guidelines designed with a common structure and clinical approach, but they are adapted to the populations and service settings for which they should be used (175, 192, 283–286). Each CAP contains five main components: a) a description of the importance of the clinical issue; b) goals of care underlying CAP design; c) triggering algorithms that use the assessment items to classify persons for different care planning approaches; d) guidelines to consider for both immediate actions to deal with safety concerns and longer-term strategies that may be used; and e) additional reference materials and resources that may be consulted.
The CAP development process involves three main approaches. First, interRAI's large international network is deployed to engage with clinicians, researchers, policy makers, and advocacy groups to discuss priorities and approaches to care recommended in different countries. Second, reviews of literature and best practice guidelines help identify practical, evidence-based interventions for specific clinical concerns. Third, interRAI's large longitudinal data holdings are used to develop and validate the predictive validity of CAP “triggering” algorithms. The results of these analyses are included in the mental health CAP descriptions to identify expected triggering rates and outcomes for inpatient, community and emergency settings. Finally, agreement with clinician ratings was also used to validate and refine the CAP triggering rules (287).
CAPs are designed using several key principles. The intervention guidelines must be rooted in empirical evidence from the peer reviewed literature in multiple continents so as not to reflect only one system of care. In addition, all mental health CAPs are framed on recovery principles (108, 111), including shared decision-making and support of the person's autonomy calibrated to his/her current level of functioning. The approach engages the person and, where appropriate, the informal support network in collaborative discussion to review the assessment findings about the person's strengths and needs and to identify the person's preferences for how CAP issues will be addressed, if at all. CAPs are not intended to be robotic care planning libraries. Rather, they provide qualitative “interview guides” that outline potential responses to the quantitatively defined triggering algorithms derived from the standardized assessment. Similarly, the CAPs are not a diagnostic system; they are designed to focus on intervention strategies at the person, family, and community levels that might enhance the person's quality of life in as many dimensions as possible.
The CAPs triggered by the interRAI mental health suite are listed in Table 7. The interRAI MH and CMH trigger all CAPs, but have somewhat different protocols for informal supports. The interRAI ESP triggers mainly the patient safety related CAPs as priority issues for crisis situations. Previous publications have highlighted triggering patterns and outcomes associated with the CAPs for traumatic life events (288), education and employment (289), and harm to others (262). Figure 2 shows the triggering rates for patient safety CAPs by homelessness in different service settings. The triggering rates for these CAPs in homeless persons in four settings (community dwelling recipients of Dutch homeless services, Canadian community mental health, emergency/mobile crisis, and inpatient settings) are lowest for the purposeful self-harm CAP and highest for the self-care CAP. With the transition from community to acute hospital-based services, the triggering rates for purposeful self-harm and harm to others are higher for all groups. Conversely, self-care triggers at the highest rate for homeless persons in community mental health settings. For non-homeless populations, the triggering rates for patient safety issues are generally lower than with homeless persons, but the rates and severity levels are also higher in emergency and hospital settings than in the community.
Table 7 List of clinical assessment protocols triggered by different interrai mental health instruments.
The CAPs can be used at the person-level to inform care planning, or aggregated at the organization or population levels to provide estimates for need analyses in various domains. The CAPs focus on current issues that increase the risk of adverse outcomes or those that might be addressed to support recovery. For example, the CAP for traumatic life events (288) targets two sub-groups for intervention: a) persons who face immediate threats to their safety due to assaults, abuse or criminal victimization that has occurred within the last 7 days; and b) persons who experienced potentially traumatic life events and who described those events as inducing a sense of intense fear or horror. The latter group were found to have elevated rates of PTSD related symptoms (288). However, from a population perspective it is also sometimes useful to examine patterns of lifetime exposure to adverse circumstances to inform policy development. Figure 3 shows the lifetime rates of criminal victimization (excluding assaults, abuse), and three types of assault or abuse by homelessness and gender. Whether considering the homeless population receiving services in the community or psychiatric inpatients admitted from homeless settings, the lifetime rates of criminal victimization, assault or abuse are higher than in the non-homeless population. Moreover, women (whether homeless or not) are systematically more likely to have experienced the three types of abuse in their lives.
Figure 3 Percentage distributions of lifetime history of victimization by homelessness and service setting, Canada and Netherlands.
interRAI instruments contain numerous embedded scales (see Tables 3 and 5) and individual items that summarize the presence/absence and severity of needs at a given point in time. These may be examined longitudinally with reassessments or when linked to previous interRAI assessments done in other care settings. At the person-level, these longitudinal changes provide insights about the person's changes in strengths and needs, response to treatment, and progression or recovery from illness.
Figure 4 shows the distribution of the Positive Symptoms Scale (see Table 3 for scale description) for homeless and non-homeless persons in five care settings. About 30% of persons in homeless services in the Netherlands, who receive only limited mental health supports, have signs of psychosis compared with almost half of the homeless persons in Canadian community mental health agencies. Severity of positive symptoms increases among those in contact with hospital or emergency mental health services, but it is most pronounced for homeless persons at the time of contact with police. In all settings, except police contacts, the severity of positive symptoms is greater for homeless than non-homeless persons.
Figure 4 Percentage distributions of positive symptoms scale score by homelessness and service setting, Canada and Netherlands.
Figure 5 shows the transitions in the Positive Symptoms Scores by homelessness from the time of admission to discharge for persons with stays of less than 90 days. Both populations improved substantially in symptom severity, but the scores were worse at admission and discharge for homeless persons.
Figure 5 Percentage distribution of positive symptoms scale scores at admission and discharge within 90 days, inpatient psychiatry, Canada.
interRAI has an extensive history of developing risk-adjusted outcome-based quality indicators. While scales and items tracked over time for individuals can be useful for person-level care planning, the longitudinal data may be aggregated at the population level to benchmark performance for internal quality improvement, accreditation and public reporting (290–297). Although interRAI's nursing home quality indicators are reported on-line nationally (298, 299), its mental health quality indicators (MHQIs) are currently provided to hospitals for internal use only. A detailed summary of these risk-adjusted quality indicators is provided elsewhere (300). The majority of these indicators are outcome-based, with the exception of a limited number of process indicators dealing with restraint and acute control medication use. There are two main outcomes: a) improvement in symptoms for persons who have non-zero values for the scale of interest at admission; b) failure to improve or worsening for persons who do not have maximum scores at admission. The outcomes include common psychiatric symptoms (e.g., depressive symptoms), functional indicators (e.g., medication management), and others that are less commonly tracked (e.g., pain).
Figure 6A shows the population-based risk-adjusted MHQI rates over time for three improvement indicators stratified by homelessness (homeless vs not homeless) in Canadian inpatient psychiatry. The rates are more volatile for the homeless indicators due to smaller sample sizes, but there are only small differences between the two subgroups for improvement in hallucinations and depressive symptoms. Both of these indicators show improvement rates to be above 0.70 as the risk-adjusted proportion improving from baseline to follow-up. On the other hand, for the homeless group, improvement rates are much lower for capacity to manage finances, despite this group having a somewhat better improvement rate than non-homeless persons.
Figure 6 Temporal changes in three risk adjusted mental health quality indicators by year, inpatient psychiatry, Canada. Figure 6A shows the risk adjusted rates of improvement in hallucinations, depressive symptoms, and capacity to manage finances in the last 90 days or at discharge (if discharged less than 90 days since baseline assessment). Figure 6B shows the risk adjusted rates of worsening of or failure to improve in hallucinations, depressive symptoms, and capacity to manage finances in the last 90 days or at discharge (if discharged less than 90 days since baseline assessment).
Figure 6B shows three risk-adjusted MHQIs for worsening or failure to improve in the same three domains. Rates of worsening or failure to improve increased (indicating poorer care) over time with notably higher rates for homeless persons. The indicator for depressive symptoms also showed poorer performance over time, but no substantive difference between subpopulations. On the other hand, rates of worsening or failure to improve in managing finances declined over time for both groups.
interRAI systems may also be used to inform decisions about the allocation of health care resources at the person and population levels. In non-mental health settings, interRAI systems have been developed to prioritize access to community and institutional services (301) and for eligibility determination in long-term care (302). Mental health counterparts to this work include a level of care framework based on the interRAI MH (303), analyses of service complexity for children with mental health needs and developmental disability based on the ChYMH-DD (304), and a decision-support algorithm describing resource intensity based on the interRAI ChYMH (305).
Case-mix systems have considerable value for informing a variety of health care decisions, including funding methodologies (306). These systems tend to have two main components: a) a classification system, based on clinical characteristics, to group service users into categories with similar levels of resource use; and b) a weighting system (case-mix indexes or CMIs) assigns ratio-level numeric values to these groups that can, among other functions, be applied to funding formulas in payment systems. interRAI-based case-mix systems are available for nursing homes (307–309), home care (310), intellectual disability services (311, 312), and intervener/interpreter services for dual sensory loss (313).
The Diagnosis Related Groups (314) system is widely used in acute general hospitals, but a consensus was reached over three decades ago that the system was inadequate for describing resource use in psychiatry (315). A number of studies pointed to the potential to use per-diem based case-mix systems that estimate costs of care per day of stay, rather than episode-based systems that attempt to predict length of stay (316–318). Most research of this type in mental health has been in hospital settings with only modest progress in community mental health services (319). In addition, although better than episode-based models, the ability to explain variance in resource use in psychiatry is lower than in more homogeneous care settings such as nursing homes (320).
The System for Classification of In-Patient Psychiatry (SCIPP) is a per-diem case-mix system for inpatient mental health services based on an earlier version of the interRAI MH (321–324). SCIPP was developed through a staff time measurement study of about 2,000 patients in 34 psychiatric hospitals in three Canadian provinces. The System for Classification of In-Patient Psychiatry (SCIPP) includes about 100 variables for a 47-group algorithm (see Figure 7) explaining about 26% of variance in per diem resource use among adult psychiatric patients. There is an 8.4 to 1 range in CMIs across the SCIPP groups. Careful attention was paid to avoiding the use of service variables, facility variables, gameable items and items that had poor psychometric properties. The SCIPP algorithm provides an important step forward in case-mix research for psychiatry. It achieves a higher explained variance than has been possible in episode-based systems, and does so without the use of independent variables that would be problematic to administer as part of a prospective payment system.
Figure 7 System for classification of inpatient psychiatry (SCIPP) schematic. Blue boxes represent decision points, gold ovals represent terminal SCIPP groups.
Figure 8 shows historical trends in the mean CMI scores in Canadian psychiatric hospitals/units using the MH as part of routine practice. Between 2006 and 2017, the mean admission CMIs rose from 1.567 to 1.657 equating to a 5.7% increase in resource intensity. On the other hand, the discharge CMIs were virtually unchanged over that time period at about 0.940. This indicates two main points: a) there is roughly a 55% drop in resource intensity from admission to discharge associated with the alleviation of symptoms related to mental health and co-morbid conditions; and b) hospitals admitted heavier patients over time without changes in their resource intensity when discharged. In addition, homeless persons were consistently more resource intensive at admission compared with the general populations with relative differences ranging between 5.6 and 4.2%.
Although much has been accomplished through interRAI's two decades of research on its mental health instruments, some limitations of the existing instruments must be addressed in order to continue to improve their utility.
There is a clear need for more cross-national research and implementation of the mental health suite. Low-middle income nations are of particular interest, but adaptations to lower resource environments are likely to be necessary. Many such countries are undertaking dramatic reforms of their mental health systems to place greater emphasis on decentralized networks and primary care (325–328) where interRAI's mental health systems have not yet been used. Another gap is the lack of a triage and screening tool for intake services and public “help” lines. These gaps may be addressed, in part, through the development of companion self-report instruments to reach populations not receiving mental health services. Such instruments might also be useful for settings with few available mental health professionals.
Another limitation is that the translation of interRAI instruments into normal clinical practice is heavily dependent on robust implementation strategies led by highly committed mental health professionals. The ability to realize the full potential of all clinical and management applications can be hindered by implementations that emphasize the data collection aspects of the system rather than its clinical use (329, 330). Moreover, even though interRAI instruments are designed to function as an integrated, cross-sectoral system, the sharing of assessment results between clinicians may not occur for reasons including ineffective communication and collaboration between those sectors (160). Further, relatively little work has been done to date on how best to present interRAI assessment results to patients as part of the shared decision-making process.
Also, while a great deal of validation work has been done, there is need for additional research on criterion validity related to mania, anxiety, and trauma. Predictive validity studies related to suicide and forensic risk are underway. It would be useful to complete inter-rater reliability studies between assessments completed by nurses or social workers and those done by clinical psychologists and psychiatrists. Of particular importance is the need for validation and refinement studies for the SCIPP system, particularly for specialized populations (e.g., forensics). Also, there is no case-mix system available at this stage for the community mental health instrument.
The availability of rich longitudinal datasets with over 1 million observations of over 400 clinical variables creates opportunities for applying new machine learning and artificial intelligence tools. When combined with real-time analytic capabilities in electronic medical records, the potential to create new personalized medicine applications that place the person's data in the context of population level data is considerable. In addition, linkage to other data sources—wearable devices, registry and administrative data, biomarker data (e.g., laboratory values, genetic data)—provides great opportunity for novel insights and innovative improvements to mental health services. Such discoveries have already been made by linking Icelandic genetic data and interRAI nursing home data to examine Alzheimer disease and cognitive decline (331, 332).
Finally, if the next 20 years of use of interRAI mental health systems sees the same degree of growth in its child/youth instruments, the future opportunities for understanding mental health from a life course perspective can be realized. The availability of scientifically sound, standardized, and fully compatible measures that follow persons living with mental illness from the earliest life stages throughout adulthood will be an unprecedented new opportunity to develop solutions to alleviate the impact of mental illness for persons of all ages.
The authors are all part of the interRAI network.
Ethics clearance for secondary analyses of interRAI data gathered by other organizations was obtained from the University of Waterloo (ORE#30173).
All authors contributed to the formulation of the ideas presented in the study and provided critical feedback to the manuscript. JPH drafted the first version of the manuscript, with major sections contributed by CE, MF, JH, AH, LM, CP, SS and CA. JPH, JF, MJ, BF, and LM made editorial changes to initial drafts and in response to reviewers' comments. JPH, BF, and CE conducted data analyses of Canadian, US and Dutch data, respectively.
Some aspects of this research were funded by the Health Transition Fund, Health Canada G03-05691. Research on homelessness in the Netherlands was commissioned and funded with support from various municipalities and human services organizations. Maastricht University and Radboud University, Nijmegen provide core financial support for the scientific use of data from these studies. Data from implementations in Canada, United States and Finland were made available based on data sharing agreements with interRAI.
Conflict of Interest
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be considered as a potential conflict of interest.
The authors would like to acknowledge the support of Jonathan Chen, Micaela Jantzi, and Britta Sohlman who provided analytic support toward portions of this manuscript. We thank the staff who conducted pilot implementations of various interRAI systems in their initial stages. We are particularly grateful to Dr. Paula Reaume-Zimmer, Integrated VP, Mental Health Services and the other staff of Bluewater Health and CMHA Lambton Kent who have played a major leadership role in the implementation and use of the full suite of interRAI Mental Health Instruments. Finally, the authors wish to acknowledge the contributions of Dr. Edgardo Perez as co-lead author of the original RAI-Mental Health assessment instrument.
The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fpsyt.2019.00926/full#supplementary-material
1. Kessler RC, Aguilar-Gaxiola S, Alonso J, Chatterji S, Lee S, Ormel J, et al. The global burden of mental disorders: an update from the WHO World Mental Health (WMH) surveys. Epidemiol Psichiatry Soc (2009) 18(1):23–3. doi: 10.1017/S1121189X00001421
3. World Health Organization. Regional Office for Europe. Mental health : facing the challenges, building solutions: report from the WHO European Ministerial Conference. Copenhagen, Denmark: World Health Organization, Regional Office for Europe (2005). xi, 181 pp.
4. Sheppard R, Deane FP, Ciarrochi J. Unmet need for professional mental health care among adolescents with high psychological distress. Aust New Z J Psychiatry (2018) 52(1):59–7. doi: 10.1177/0004867417707818
5. Murrell K. Lean Flow and the Impact of the Long-Stay Patient. In: Big Book of Emergency Department Psychiatry: A Guide to Patient Centered Operational Improvement, CRC Press: Taylor & Francis Group (2017). vol. 10. doi: 10.1201/b21955-20
8. Jonsson PV, Finne-Soveri H, Jensdottir AB, Ljunggren G, Bucht G, Grue EV, et al. Co-morbidity and functional limitation in older patients underreported in medical records in Nordic Acute Care Hospitals when compared with the MDS-AC instrument. Age Ageing (2006) 35(4):434–8. doi: 10.1093/ageing/afj060
9. Singer A, Kroeker AL, Yakubovich S, Duarte R, Dufault B, Katz A. Data quality in electronic medical records in Manitoba: do problem lists reflect chronic disease as defined by prescriptions? Can Fam Physician (2017) 63(5):382–89. doi: 10.1093/jamia/ocw013
10. Singer A, Yakubovich S, Kroeker AL, Dufault B, Duarte R, Katz A. Data quality of electronic medical records in Manitoba: do problem lists accurately reflect chronic disease billing diagnoses? J Am Med Inform Assoc (2016) 23(6):1107–12. doi: 10.1038/tp.2015.182
11. Rumshisky A, Ghassemi M, Naumann T, Szolovits P, Castro VM, McCoy TH, et al. Predicting early psychiatric readmission with natural language processing of narrative discharge summaries. Trans Psychiatry (2016) 6(10):e921. doi: 10.1136/bmj.h1885
12. Liao KP, Cai T, Savova GK, Murphy SN, Karlson EW, Ananthakrishnan AN, et al. Development of phenotype algorithms using electronic medical records and incorporating natural language processing. BMJ (2015) 350:h1885. doi: 10.1097/00001888-199908000-00012
14. Bjaastad JF, Jensen-Doss A, Moltu C, Jakobsen P, Hagenberg H, Joa I. Attitudes toward standardized assessment tools and their use among clinicians in a public mental health service. Nordic J Psychiatry (2019), 7: 1–10. doi: 10.1080/08039488.2019.1642383
15. Youngstrom EA, Choukas-Bradley S, Calhoun CD, Jensen-Doss A. Clinical guide to the evidence-based assessment approach to diagnosis and treatment. Cogn Behav Practice (2015) 22(1):20–5. doi: 10.1037/abn0000232
17. Schaefer JD, Caspi A, Belsky DW, Harrington H, Houts R, Horwood LJ, et al. Enduring mental health: prevalence and prediction. J Abnorm Psychol (2017) 126(2):212–24. doi: 10.1016/S0140-6736(13)62116-9
19. Patton GC, Coffey C, Romaniuk H, Mackinnon A, Carlin JB, Degenhardt L, et al. The prognosis of common mental disorders in adolescents: a 14-year prospective cohort study. Lancet (2014) 383(9926):1404–11. doi: 10.1080/09297049.2011.578572
20. Wittchen HU, Lieb R, Pfister H, Schuster P. The waxing and waning of mental disorders: evaluating the stability of syndromes of mental disorders in the population. Compr Psychiatry (2000) 41(2 Suppl 1):122–32. doi: 10.1016/j.appdev.2016.02.002
24. Studer J, Baggio S, Deline S, N'Goran AA, Henchoz Y, Mohler-Kuo M, et al. Peer pressure and alcohol use in young men: a mediation analysis of drinking motives. Int J Drug Policy (2014) 25(4):700–8. doi: 10.1080/09595230802392790
25. Lutz HR, McClure K, Armstrong S. Social problem solving and adolescent alcohol use within the context of well-established risk factors for adolescent alcohol use. J Child Adolesc Subst Abuse (2017) 26(3):229–41. doi: 10.1136/bmjopen-2017-016060
26. Montgomery C, Fisk JE, Craig L. The effects of perceived parenting style on the propensity for illicit drug use: the importance of parental warmth and control. Drug Alcohol Rev (2008) 27(6):640–49. doi: 10.1080/00981389.2017.1318800
27. Hodder RK, Freund M, Bowman J, Wolfenden L, Campbell E, Dray J, et al. Effectiveness of a pragmatic school-based universal resilience intervention in reducing tobacco, alcohol and illicit substance use in a population of adolescents: cluster-randomised controlled trial. BMJ Open (2017) 7(8):e016060. doi: 10.1186/s13195-016-0188-8
29. Prince M, Ali GC, Guerchet M, Prina AM, Albanese E, Wu YT. Recent global trends in the prevalence and incidence of dementia, and survival with dementia. Alzheimers Res Ther (2016) 8(1):23. doi: 10.1371/journal.pone.0019590
30. Prince M, Bryce R, Albanese E, Wimo A, Ribeiro W, Ferri CP. The global prevalence of dementia: a systematic review and metaanalysis. Alzheimers Dement (2013) 9(1):63–75 e2. doi: 10.1016/S2215-0366(17)30078-0
31. Chang CK, Hayes RD, Perera G, Broadbent MT, Fernandes AC, Lee WE, et al. Life expectancy at birth for people with serious mental illness and other major disorders from a secondary mental health care case register in London. PLoS One (2011) 6(5):e19590. doi: 10.1192/bjp.bp.111.092718
32. Hjorthoj C, Sturup AE, McGrath JJ, Nordentoft M. Years of potential life lost and life expectancy in schizophrenia: a systematic review and meta-analysis. Lancet Psychiatry (2017) 4(4):295–301. doi: 10.1111/acps.12842
38. Andrew MK, Rockwood K. Psychiatric illness in relation to frailty in community-dwelling elderly people without dementia: a report from the Canadian Study of Health and Aging. Can J Aging (2007) 26(1):33–8. doi: 10.3138/8774-758w-702q-2531
40. Chong SA, Abdin E, Nan L, Vaingankar JA, Subramaniam M. Prevalence and impact of mental and physical comorbidity in the adult Singapore population. Ann Acad Med Singapore (2012) 41(3):105–14. doi: 10.1016/S1470-2045(10)70085-1
41. Bjorkenstam E, Ljung R, Burstrom B, Mittendorfer-Rutz E, Hallqvist J, Weitoft GR. Quality of medical care and excess mortality in psychiatric patients–a nationwide register-based study in Sweden. BMJ Open (2012) 2:e000778. doi: 10.1001/archpsyc.58.6.565
42. Howard LM, Barley EA, Davies E, Rigg A, Lempp H, Rose D, et al. Cancer diagnosis in people with severe mental illness: practical and ethical issues. Lancet Oncol (2010) 11(8):797–804. doi: 10.1016/S1470-2045(10)70085-1
43. Druss BG, Bradford WD, Rosenheck RA, Radford MJ, Krumholz HM. Quality of medical care and excess mortality in older patients with mental disorders. Arch Gen Psychiatry (2001) 58(6):565–72. doi: 10.1177/1359786810382058
44. Naylor C, Das P, Ross S, Honeyman M, Thompson J, Gilburt H. The case for change: 10 areas where integration is needed most. In: . Bringing together physical and mental health - A new frontier for integrated care. The King's Fund: London, U.K (2016). p. 20–3. doi: 10.1186/s12913-014-0618-x
46. Kulkarni M, Huddlestone L, Taylor A, Sayal K, Ratschen E. A cross-sectional survey of mental health clinicians' knowledge, attitudes, and practice relating to tobacco dependence among young people with mental disorders. BMC Health Serv Res (2014) 14:618. doi: 10.1016/j.jad.2016.11.026
47. Guthrie DM, Declercq A, Finne-Soveri H, Fries BE, Hirdes JP. The health and well-being of older adults with Dual Sensory Impairment (DSI) in four countries. PLoS One (2016) 11(5):e0155073. doi: 10.1007/s40266-014-0233-x
49. Pompili M, Venturini P, Lamis DA, Giordano G, Serafini G, Belvederi Murri M, et al. Suicide in stroke survivors: epidemiology and prevention. Drugs Aging (2015) 32(1):21–9. doi: 10.1016/j.dhjo.2009.09.001
50. McConnell D, Hahn L, Savage A, Dube C, Park E. suicidal ideation among adults with disability in western canada: a brief report. Community Ment Health J (2016) 52(5):519–26. doi: 10.1016/j.copsyc.2017.07.018
51. Giannini MJ, Bergmark B, Kreshover S, Elias E, Plummer C, O'Keefe E. understanding suicide and disability through three major disabling conditions: intellectual disability, spinal cord injury, and multiple sclerosis. Disabil Health J (2010) 3(2):74–8. doi: 10.1007/s00127-016-1178-7
54. Matthews T, Danese A, Wertz J, Odgers CL, Ambler A, Moffitt TE, et al. Social isolation, loneliness and depression in young adulthood: a behavioural genetic analysis. Soc Psychiatry Psychiatr Epidemiol (2016) 51(3):339–48. doi: 10.1093/jurban/78.3.458
55. Holt-Lunstad J, Smith TB, Baker M, Harris T, Stephenson D. Loneliness and social isolation as risk factors for mortality: a meta-analytic review. Perspect Psychol Sci (2015) 10(2):227–37. doi: 10.1093/bjsw/bcr076
57. Tew J, Ramon S, Slade M, Bird V, Melton J, Le Boutillier C. Social factors and recovery from mental health difficulties: a review of the evidence. Br J Soc Work (2012) 42(3):443–60. doi: 10.1038/nrn4005
58. Santini ZI, Koyanagi A, Tyrovolas S, Mason C, Haro JM. The association between social relationships and depression: a systematic review. J Affect Disord (2015) 175:53–5. doi: 10.3928/00485713-20050501-06
67. Perese EF. Stigma, poverty, and victimization: Roadblocks to recovery for individuals with severe mental illness. J Am Psychiatr Nurses Assoc (2007) 13(5):285–95. doi: 10.1371/journal.pmed.0050225
69. Trani J-F, Bakhshi P, Kuhlberg J, Narayanan SS, Venkataraman H, Mishra NN, et al. Mental illness, poverty and stigma in India: a case–control study. BMJ Open (2015) 5(2):e006355. doi: 10.1016/S0140-6736(17)31869-X
70. Fazel S, Geddes JR, Kushel M. The health of homeless people in high-income countries: descriptive epidemiology, health consequences, and clinical and policy recommendations. Lancet (2014) 384(9953):1529–40. doi: 10.1177/1403494818759839
71. Fazel S, Khosla V, Doll H, Geddes J. The prevalence of mental disorders among the homeless in western countries: systematic review and meta-regression analysis. PLoS Med (2008) 5(12):e225. doi: 10.1177/0042098015587818
72. Nielsen SF, Hjorthøj CR, Erlangsen A, Nordentoft M. Psychiatric disorders and mortality among people in homeless shelters in Denmark: a nationwide register-based cohort study. Lancet (2011) 377(9784):2205–14. doi: 10.1016/S2214-109X(18)30539-4
73. Aldridge RW, Story A, Hwang SW, Nordentoft M, Luchenski SA, Hartwell G, et al. Morbidity and mortality in homeless individuals, prisoners, sex workers, and individuals with substance use disorders in high-income countries: a systematic review and meta-analysis. Lancet (2018) 391(10117):241–50. doi: 10.1080/02673037.2017.1344957
74. Benjaminsen L, Birkelund JF. Explaining excess morbidity amongst homeless shelter users: a multivariate analysis for the danish adult population. Scand. J Public Health (2018) 1403494818759839. doi: 10.1177/0042098014548138
76. Baranyi G, Scholl C, Fazel S, Patel V, Priebe S, Mundt AP. Severe mental illness and substance use disorders in prisoners in low-income and middle-income countries: a systematic review and meta-analysis of prevalence studies. Lancet Global Health (2019) 7(4):e461–e71. doi: 10.1111/dar.12448
78. Piat M, Polvere L, Kirst M, Voronka J, Zabkiewicz D, Plante M-C, et al. Pathways into homelessness: understanding how both individual and structural factors contribute to and sustain homelessness in Canada. Urban Stud (2015) 52(13):2366–82. doi: 10.4088/PCC.15m01784
80. Kingston REF, Marel C, Mills KL. A systematic review of the prevalence of comorbid mental health disorders in people presenting for substance use treatment in Australia. Drug Alcohol Rev (2017) 36(4):527–39. doi: 10.1016/B978-0-12-398336-7.00055-3
81. Prior K, Mills K, Ross J, Teesson M. Substance use disorders comorbid with mood and anxiety disorders in the Australian general population. Drug Alcohol Rev (2017) 36(3):317–24. doi: 10.1177/070674370404900405
82. Perlman CM, Hirdes JP, Vigod S. psychiatric rehospitalization: development of a person-level indicator for care planning and quality assurance. Prim Care Compan CNS Disord (2015) 17(4). doi: 10.1001/jama.2010.616
83. Drake RE, O'Neal EL, Wallach MA. A systematic review of psychosocial research on psychosocial interventions for people with co-occurring severe mental and substance use disorders. J Subst Abuse Treat (2008) 34(1):123–38. doi: 10.3109/09638237.2015.1036973
84. Deady M, Teesson M, Brady KT. Impact of substance use on the course of serious mental disorders. Principles Addiction: Compr Addictive Behav Disord (2013) 1:525–32. doi: 10.1080/00207411.1982.11448926
87. van Hoof F, Knispel A, Aagaard J, Schneider J, Beeley C, Keet R, et al. The role of national policies and mental health care systems in the development of community care and community support: an international analysis. J Ment Health (2015) 24(4):202–7. doi: 10.1080/00207411.2016.1167489
88. Richman A, Harris P. Mental hospital deinstitutionalization in Canada: A national perspective with some regional examples. Int J Ment Health (1982) 11(4):64–3. doi: 10.1002/j.2051-5545.2011.tb00060.x
94. Westman J, Gissler M, Wahlbeck K. Successful deinstitutionalization of mental health care: increased life expectancy among people with mental disorders in Finland. Eur J Public Health (2012) 22(4):604–6. doi: 10.5694/j.1326-5377.2006.tb00581.x
95. Taylor Salisbury T, Killaspy H, King M. The relationship between deinstitutionalization and quality of care in longer-term psychiatric and social care facilities in Europe: a cross-sectional study. Eur Psychiatry (2017) 42:95–102. doi: 10.1080/14789940802594445
96. Kim D-Y. Psychiatric deinstitutionalization and prison population growth: A critical literature review and its implications. Criminal Justice Policy Rev (2016) 27(1):3–21. doi: 10.1192/bjp.bp.114.161943
99. Sirdifield C, Gojkovic D, Brooker C, Ferriter M. A systematic review of research on the epidemiology of mental health disorders in prison populations: a summary of findings. J Forensic Psychiatry Psychol (2009) 20(S1):S78–S101. doi: 10.1080/10439463.2016.1219734
100. Winkler P, Barrett B, McCrone P, Csemy L, Janouskova M, Hoschl C. Deinstitutionalised patients, homelessness and imprisonment: systematic review. Br J Psychiatry (2016) 208(5):421–8. doi: 10.1016/j.ijlp.2016.02.013
104. Roy L, Crocker AG, Nicholls TL, Latimer E, Gozdzik A, O'Campo P, et al. Profiles of criminal justice system involvement of mentally ill homeless adults. Int J Law Psychiatry (2016) 45:75–8. doi: 10.1192/bjp.bp.110.083733
105. Penney SR, Seto MC, Crocker AG, Nicholls TL, Grimbos T, Darby PL, et al. Changing characteristics of forensic psychiatric patients in Ontario: a population-based study from 1987 to 2012. Soc Psychiatry Psychiatr Epidemiol (2019) 54(5):627–38. doi: 10.3109/09540261.2011.644847
108. Leamy M, Bird V, Le Boutillier C, Williams J, Slade M. Conceptual framework for personal recovery in mental health: systematic review and narrative synthesis. Br J Psychiatry (2011) 199(6):445–52. doi: 10.1002/wps.20084
110. Williams J, Leamy M, Bird V, Harding C, Larsen J, Le Boutillier C, et al. Measures of the recovery orientation of mental health services: systematic review. Soc Psychiatry Psychiatr Epidemiol (2012) 47(11):1827–35. doi: 10.1192/bjp.bp.106.025791
111. Slade M, Leamy M, Bacon F, Janosik M, Le Boutillier C, Williams J, et al. International differences in understanding recovery: systematic review. Epidemiol Psychiatr Sci (2012) 21(4):353–64. doi: 10.1002/j.2051-5545.2009.tb00218.x
112. Slade M, Amering M, Farkas M, Hamilton B, O'Hagan M, Panther G, et al. Uses and abuses of recovery: implementing recovery-oriented practices in mental health systems. World Psychiatry (2014) 13(1):12–0. doi: 10.1016/j.schres.2009.01.008
116. Farooq S, Large M, Nielssen O, Waheed W. The relationship between the duration of untreated psychosis and outcome in low-and-middle income countries: a systematic review and meta analysis. Schizophr Res (2009) 109(1–3):15–3. doi: 10.1016/j.jcgg.2013.09.001
117. Reynders A, Kerkhof AJ, Molenberghs G, Van Audenhove C. Help-seeking, stigma and attitudes of people with and without a suicidal past. A comparison between a low and a high suicide rate country. J Affect Disord (2015) 178:5–11. doi: 10.1093/gerona/53A.2.M92
121. Garrard J, Rolnick SJ, Nitz NM, Luepke L, Jackson J, Fischer LR, et al. Clinical detection of depression among community-based elderly people with self-reported symptoms of depression. J Gerontol Ser A: Biol Sci Med Sci (1998) 53(2):M92–M101. doi: 10.1037/t00742-000
122. Gentzler ER, Derry H, Ouyang DJ, Lief L, Berlin DA, Xu CJ, et al. Underdetection and undertreatment of dyspnea in critically ill patients. Am J Resp Crit Care Med (2019) 199(11):1377–84. doi: 10.1037/0022-006X.56.6.893
123. Folstein MF, Folstein SE, McHugh PR. “Mini-mental state”: a practical method for grading the cognitive state of patients for the clinician. J Psychiatr Res (1975) 12(3):189–98. doi: 10.1093/schbul/13.2.261
124. Inouye SK, van Dyck CH, Alessi CA, Balkin S, Siegal AP, Horwitz RI. Clarifying confusion: the confusion assessment method: a new method for detection of delirium. Ann Internal Med (1990) 113(12):941–48. doi: 10.1192/bjp.133.5.429
129. Falsetti SA, Resnick HS, Resick PA, Kilpatrick DG. The modified PTSD symptom scale: a brief self-report measure of posttraumatic stress disorder. Behav Ther (1993). 16: 161–162. doi: 10.1097/00005053-198001000-00006
131. Buysse DJ, Reynolds Iii CF, Monk TH, Berman SR, Kupfer DJ. The Pittsburgh Sleep Quality Index: a new instrument for psychiatric practice and research. Psychiatry Res (1989) 28(2):193–3. doi: 10.1007/BF00783424
132. Cohen-Mansfield J. Conceptualization of agitation: results based on the Cohen-Mansfield agitation inventory and the agitation behavior mapping instrument. Int Psychogeriat (1997) 8(S3):309–15. doi: 10.1176/appi.ajp.2011.10111704
133. McLellan AT, Luborsky L, Woody GE, O'Brien CP. An improved diagnostic evaluation instrument for substance abuse patients: the Addiction Severity Index. J Nervous Ment Disease (1980). 1: 26–33. doi: 10.1037/t04911-000
135. Janca A, Kastrup M, Katschnig H, López-Ibor JJ, Mezzich JE, Sartorius N. The World Health Organization Short Disability Assessment Schedule (WHO DAS-S): a tool for the assessment of difficulties in selected areas of functioning of patients with mental disorders. Soc Psychiatry Psychiatr Epidemiol (1996) 31(6):349–54.
136. Posner K, Brown GK, Stanley B, Brent DA, Yershova KV, Oquendo MA, et al. The Columbia–Suicide Severity Rating Scale: initial validity and internal consistency findings from three multisite studies with adolescents and adults. Am J Psychiatry (2011) 168(12):1266–77. doi: 10.1176/appi.ajp.2011.10111704
137. Yudofsky SC, Silver JM, Jackson W, Endicott J, Williams D. The Overt Aggression Scale for the objective rating of verbal and physical aggression. Am J Psychiatry (1986) 1: 35–39. doi: 10.1111/j.1532-5415.2007.01207.x
138. Steel K, Jonsson PV, Dupasquier JN, Gilgen R, Hirdes J, Schroll M, et al. Systems of care for frail older persons. InterRAI. Trans Am Clin Climatological Assoc (1999) 110:30. doi: 10.1093/geront/30.3.293
140. Carpenter GI, Hirdes JP. Using interRAI assessment systems to measure and maintain quality of long-term care. In: . A good life in old age? Monitoring and improving quality in long-term care. OECD Health Policy Studies. 28. OECD Publishing: Paris (2013). p. 2017. doi: 10.12927/hcq.2000.16756
141. Fries BE, Morris JN, Bernabei R, Finne-Soveri H, Hirdes J. Rethinking the resident assessment protocols. Journal of the American Geriatric Society (2007). 55(7): 113–114. doi: 10.1111/j.1532-5415.2007.01207.x
142. Morris JN, Hawes C, Fries BE, Phillips CD, Mor V, Katz S, et al. Designing the national resident assessment instrument for nursing homes. Gerontologist (1990) 30(3):293–7. doi: 10.1093/ageing/afl036
143. Morris JN, Fries BE, Steel K, Ikegami N, Bernabei R, Carpenter GI, et al. Comprehensive clinical assessment in community setting: applicability of the MDS-HC. J Am Geriat Soc (1997) 45(8):1017–24. doi: 10.1186/1472-6963-9-71
144. Hirdes JP, Marhaba M, Smith TF, Clyburn L, Mitchell L, Lemick RA, et al. Development of the Resident Assessment Instrument–Mental Health (RAI-MH). Hosp Q (2000) 4(2):44–1. doi: 10.1016/S0840-4704(10)60164-0
145. Hirdes JP, Ljunggren G, Morris JN, Frijters DH, Finne Soveri H, Gray L, et al. Reliability of the interRAI suite of assessment instruments: a 12-country study of an integrated health information system. BMC Health Serv Res (2008) 8:277. doi: 10.1007/s10578-017-0751-y
146. Hirdes JP. Addressing the health needs of frail elderly people: Ontario"s experience with an integrated health information system. Age and Ageing. Oxford University Press 35(4): 329–331. (2006). doi: 10.4137/HSI.S30775
147. Gray LC, Berg K, Fries BE, Henrard JC, Hirdes JP, Steel K, et al. Sharing clinical information across care settings: the birth of an integrated assessment system. BMC Health Serv Res (2009) 9:71. doi: 10.1016/j.ijlp.2016.02.031
148. Hirdes JP, Fries BE, Morris JN, Steel K, Mor V, Frijters D, et al. Integrated health information systems based on the RAI/MDS series of instruments. Healthcare Manage Forum (1999) 12(4):30–40. doi: 10.1016/j.jamda.2017.08.010
149. Lau C, Stewart SL, Saklofske DH, Tremblay PF, Hirdes J. Psychometric Evaluation of the interRAI Child and Youth Mental Health Disruptive/Aggression Behaviour Scale (DABS) and Hyperactive/Distraction Scale (HDS). Child Psychiatry Hum Dev (2018) 49(2):279–89. doi: 10.1017/S0714980814000579
150. Phillips CD, Hawes C. The interRAI Pediatric Home Care (PEDS HC) Assessment: evaluating the long-term community-based service and support needs of children facing special healthcare challenges. Health Serv Insights (2015) 8:17–24. doi: 10.1177/0898264315594138
151. Hoffman R, Hirdes J, Brown GP, Dubin JA, Barbaree H. The use of a brief mental health screener to enhance the ability of police officers to identify persons with serious mental disorders. Int J Law Psychiatry (2016) 47:28–35. doi: 10.1016/j.ijlp.2016.02.031
152. Morris JN, Declercq A, Hirdes JP, Finne-Soveri H, Fries BE, James ML, et al. Hearing the voice of the resident in long-term care facilities-an internationally based approach to assessing quality of life. J Am Med Directors Assoc (2018) 19(3):207–15. doi: 10.4236/health.2013.53A084
154. Kehyayan V, Hirdes JP, Tyas SL, Stolee P. Predictors of long-term care facility residents' self-reported quality of life with individual and facility characteristics in Canada. J Aging Health (2016) 28(3):503–29. doi: 10.12927/hcq.2000.16756
155. Morris J, Howard E, Geffen L, Hirdes J, Hogeveen S, Berg K, et al. interRAI Check-Up (CU) Assessment, Supplement, and Self-Reported Forms and User’s Manual. Version 10.1. Washington DC: interRAI (2018). doi: 10.1177/0840470415581240
157. Hirdes JP, interRAI (Organization). MH Coordinating Committee. interRAI mental health (MH) assessment form and user's manual for in-patient psychiatry. 9.1. ed. Ann Arbor, Mich.: interRAI; 2010. vii, 151 p. (2010). doi: 10.1186/1472-6963-14-457
158. Hirdes JP, Marhaba M, Smith TF, Clyburn L, Mitchell L, Lemick RA, et al. Development of the resident assessment instrument–mental health (RAI-MH). Hosp Q (2000) 4(2):44–51. doi: 10.12927/hcq.2000.16756
159. Stewart SL, Hirdes JP. Identifying mental health symptoms in children and youth in residential and in-patient care settings. Healthc Manage Forum (2015) 28(4):150–56. doi: 10.1177/0840470415581240
160. Hirdes JP, Morris JN, interRAI (Organization). CMH Coordinating Committee., interRAI (Organization). Instrument and Systems Development Committee. interRAI community mental health (CMH) assessment form and user's manual. 9.1, interRAI standard ed. Ann Arbor, Mich.: interRAI; 2010. vii, 151 p. (2010).
161. Martin L, Hirdes JP. Exploring the impact of common assessment instrumentation on communication and collaboration in inpatient and community-based mental health settings: a focus group study. BMC Health Serv Res (2014) 14:457. doi: 10.1186/1472-6963-14-457
164. Rabinowitz T, Morris JN, interRAI (Organization). ESP Coordinating Committee., interRAI (Organization). Instrument and Systems Development Committee. interRAI emergency screener for psychiatry (ESP) assessment form and user's manual. 9.1. ed. Ann Arbor, Mich.: interRAI; 2013. vii, 67 p. (2013).
165. Hirdes J, Hoffman R, Brown GP, Barbaree H, Curtin-Telegdi N, Morris J, et al. interRAI Brief Mental Health Screener (BMHS) Assessment Forms and User"s Manual: A Screening Level Assessment for Use by Police Officers and Other Front-line Service Providers (version 9.3). Washington DC: interRAI (2015).
167. Hoffman RE. Development of the interRAI brief mental health screener to enhance the ability of police officers to identify persons with serious mental disorder. (2013). doi: 10.1023/B:LAHU.0000046435.98590.55
168. Pearson Hirdes D. The Potential for Machine Learning in Mental Health Policing: Predicting Outcomes of Mental Health Related Calls for Service. McMaster University: Hamilton, ON, Canada (2019). doi: 10.1177/0840470415581232
170. Mathias K-L. Gender-based analysis of criminogenic risk and clinical need among Ontario forensic patients. University of Waterloo: Waterloo, ON, Canada (2015). doi: 10.1111/j.1741-1130.2006.00094.x
171. Seto MC, Harris GT, Rice ME. The criminogenic, clinical, and social problems of forensic and civil psychiatric patients. Law Hum Behav (2004) 28(5):577–86. doi: 10.1023/b:lahu.0000046435.98590.55
173. Hirdes J, Martin L, Curtin-Telegdi N, Fries B, James M, Rabinowitz T, et al. interRAI Intellectual Disability (ID) Assessment Form and User"s Manual. Version 9.2. interRAI. Washington DC: interRAI (2013). doi: 10.1080/19315860802029154
174. Martin L, Hirdes JP, Fries BE, Smith TF. Development and psychometric properties of an assessment for persons with intellectual disability—the interRAI ID. J Policy Pract Intellect Disabil (2007) 4(1):23–9. doi: 10.1111/j.1365-2788.2008.01041.x
175. Martin LA, Morris JN, interRAI (Organization). ID CAP Development Committee., interRAI (Organization). Instrument and Systems Development Committee. interRAI Intellectual Disability Collaborative Action Plans (CAPs) : for use with the intellectual disability assessment instrument. 9.2. ed. Ann Arbor, Michigan: interRAI; 2013. xi, 71. p. (2013).
176. Ashworth M, Hirdes JP, Martin L. The social and recreational characteristics of adults with intellectual disability and pica living in institutions. Res Dev Disabil (2009) 30(3):512–20. doi: 10.1515/ijdhd.2012.008
178. Langlois L, Martin L. Relationship between diagnostic criteria, depressive equivalents and diagnosis of depression among older adults with intellectual disability. J Intellect Disabil Res (2008) 52(11):896–904. doi: 10.1515/IJDHD.2010.033
179. Martin L, Hirdes J, ME P. Healthcare needs of community- and institution-based older adults with intellectual disabilities. J Policy Pract Intellect. Dev Disabil (2008). 5 (1): 28–29 doi: 10.1186/s12889-019-6912-0
180. Martin L, Ashworth M, Montague P. A longitudinal study of institutional downsizing and challenging behaviors among adults with intellectual disability. ijdhd (2012) 11(1):51. doi: 10.1111/j.1741-1130.2010.00261.x
181. Carfi A, Vetrano DL, Mascia D, Meloni E, Villani ER, Acampora N, et al. Adults with Down syndrome: a comprehensive approach to manage complexity. J Intellect Disabil Res (2019) 63(6):624–9. doi: 10.1179/2047387713Y.0000000021
182. Ashworth M, Martin L, Montague P. A longitudinal study of institutional downsizing and activity involvement among adults with intellectual disability. ijdhd (2010) 9(4):319. doi: 10.1515/IJDHD.2010.033
185. Martin L. Re-examining the relationship between intellectual disability and receipt of inpatient psychiatry and complex continuing care services in Ontario, Canada. Int J Dev Disabil (2014) 60(2):65–9. doi: 10.1179/2047387713Y.0000000021
186. Martin L, Hirdes JP, Fries BE. Examining the characteristics of persons with intellectual disability receiving hospital services, Part 2: Complex continuing care hospitals/units. J Dev Disabil (2007) 13(3):105–17.
187. Martin L, Ouellette-Kuntz H, McKenzie K. Care in the community: home care use among adults with intellectual and developmental disabilities over time. J Policy Pract Intel (2017) 14(3):251–4. doi: 10.1111/j.1532-5415.1997.tb02974.x
188. Martin L, Ouellette-Kuntz H, McKenzie K. Use of home care services among adults with intellectual and developmental disabilities: does where you live matter? Res Pract Intellect (2018) 5(2):192–201. doi: 10.1080/23297018.2018.1443023
189. Martin LR, Hirdes JP, Fries BE. Examining the characteristics of persons with intellectual disability receiving hospital services: Part 1 - Psychiatric hospitals/units. J Dev Disabil (2007) 13(3):89–103. doi: 10.1017/S0714980800013878
192. Morris JN, interRAI (Organization). Instrument and Systems Development Committee. InterRAI clinical assessment protocols (CAPs) for use with community and long-term care assessment instruments. 9.1. ed. Ann Arbor, MI: interRAI; 2010. xi, 179 p. (2010) doi: 10.1177/0840470415581233
193. Hirdes JP, Ikegami N, Jonsson PV, Topinkova E, Maxwell CJ, Yamauchi K. Cross-national comparisons of antidepressant use among institutionalized older persons based on the minimum data set (MDS). Can J Aging-Revue Can Du Vieillissement (2000) 19:18–27. doi: 10.1002/gps.4259
194. Schluter PJ, Lacey C, Porter RJ, Jamieson HA. An epidemiological profile of bipolar disorder among older adults with complex needs: A national cross-sectional study. Bipolar Disord (2017) 19(5):375–85. doi: 10.5770/cgj.20.228
195. Hirdes JP, Fries BE, Rabinowitz T, Morris JN. Comprehensive assessment of persons with bipolar disorder in long-term care settings: the potential of the interRAI family of instruments. In: Sajatovic M, Blow FC, editors. Bipolar disorders in late life. Johns Hopkins University Press: Baltimore, MD (2007). p. 27–9. doi: 10.1177/0898264317708072
197. Neufeld E, Hirdes JP, Perlman CM, Rabinowitz T. Risk and protective factors associated with intentional self-harm among older community-residing home care clients in Ontario, Canada. Int J Geriatr Psych (2015) 30(10):1032–40. doi: 10.1002/gps.2770
198. McGuire C, Kristman VL, Martin L, Bedard M. Characteristics and incidence of traumatic brain injury in older adults using home care in Ontario from 2003-2013. Can Geriatr J (2017) 20(1):2–9. doi: 10.1586/1473722.214.171.1243
199. McGuire C, Kristman VL, Martin L, Bedard M. The association between depression and traumatic brain injury in older adults: a nested matched case control study. J Aging Health (2018) 30(7):1156–68. doi: 10.1111/j.1532-5415.2008.02048.x
200. Colantonio A, Hsueh J, Petgrave J, Hirdes JP, Berg K. A profile of patients with traumatic brain injury within home care, long-term care, complex continuing care, and institutional mental health settings in a publicly insured population. J Head Trauma Rehab (2015) 30(6):E18–9. doi: 10.1016/j.jamda.2009.06.004
201. Boorsma M, Joling KJ, Frijters DH, Ribbe ME, Nijpels G, van Hout HP. The prevalence, incidence and risk factors for delirium in Dutch nursing homes and residential care homes. Int J Geriatr Psychiatry (2012) 27(7):709–15. doi: 10.1016/j.pain.2011.10.007
202. Rabinowitz T, Murphy KM, Nagle KJ, Bodor CI, Kennedy SM, Hirdes JP. Delirium: pathophysiology, recognition, prevention and treatment. Expert Rev Neurother (2003) 3(3):343–55. doi: 10.1046/j.1532-5415.2001.49035.x
204. Volicer L, Van der Steen JT, Frijters DH. Modifiable factors related to abusive behaviors in nursing home residents with dementia. J Am Med Directors Assoc (2009) 10(9):617–22. doi: 10.1080/13607863.2016.1247426
205. Tosato M, Lukas A, van der Roest HG, Danese P, Antocicco M, Finne-Soveri H, et al. Association of pain with behavioral and psychiatric symptoms among nursing home residents with cognitive impairment: Results from the SHELTER study. Pain (2012) 153(2):305–10. doi: 10.1093/geronj/49.4.M174
206. Ahronheim JC, Mulvihill M, Sieger C, Park P, Fries BE. State practice variations in the use of tube feeding for nursing home residents with severe cognitive impairment. J Am Geriatr Soc (2001) 49(2):148–52. doi: 10.1177/0891988715598231
207. Bansal S, Hirdes JP, Maxwell CJ, Papaioannou A, Giangregorio LM. Identifying fallers among home care clients with dementia and parkinson's disease. Can J Aging (2016) 35(3):319–31. doi: 10.1002/gps.2232
208. Garms-Homolova V, Notthoff N, Declercq A, van der Roest HG, Onder G, Jonsson P, et al. Social and functional health of home care clients with different levels of cognitive impairments. Aging Ment Health (2017) 21(1):18–3. doi: 10.1111/ggi.12707
211. Feng Z, Hirdes JP, Smith TF, Finne-Soveri H, Chi I, Du Pasquier JN, et al. Use of physical restraints and antipsychotic medications in nursing homes: a cross-national study. Int J Geriatr Psychiatry (2009) 24(10):1110–8. doi: 10.1002/gps.2232
212. Freeman S, Spirgiene L, Martin-Khan M, Hirdes JP. Relationship between restraint use, engagement in social activity, and decline in cognitive status among residents newly admitted to long-term care facilities. Geriatr Gerontol Int (2017) 17(2):246–55. doi: 10.2105/AJPH.83.3.342
213. Hawes C, Mor V, Phillips CD, Fries BE, Morris JN, Steele-Friedlob E, et al. The OBRA-87 nursing home regulations and implementation of the resident assessment instrument: effects on process quality. J Am Geriatr Soc (1997) 45(8):977–85. doi: 10.1097/00005650-199611000-00008
214. Heckman GA, Crizzle AM, Chen J, Pringsheim T, Jette N, Kergoat MJ, et al. Clinical complexity and use of antipsychotics and restraints in long-term care residents with parkinson's disease. J Parkinsons Dis (2017) 7(1):103–15. doi: 10.1093/ageing/26.suppl_2.43
217. Phillips CD, Hawes C, Mor V, Fries BE, Morris JN, Nennstiel ME. Facility and area variation affecting the use of physical restraints in nursing homes. Med Care (1996) 34(11):1149–62. doi: 10.1001/archinte.165.6.696
219. Foebel A, Ballokova A, Wellens NI, Fialova D, Milisen K, Liperoti R, et al. A retrospective, longitudinal study of factors associated with new antipsychotic medication use among recently admitted long-term care residents. BMC Geriatr (2015) 15:128. doi: 10.4088/JCP.15m10303
220. Foebel AD, Liperoti R, Onder G, Finne-Soveri H, Henrard JC, Lukas A, et al. Use of antipsychotic drugs among residents with dementia in European long-term care facilities: results from the shelter study. J Am Med Dir Assoc (2014) 15(12):911–7. doi: 10.1186/s12877-017-0636-8
221. Liperoti R, Gambassi G, Lapane KL, Chiang C, Pedone C, Mor V, et al. Conventional and atypical antipsychotics and the risk of hospitalization for ventricular arrhythmias or cardiac arrest. Arch Internal Med (2005) 165(6):696–71. doi: 10.1177/2158244019835942
223. Liperoti R, Sganga F, Landi F, Topinkova E, Denkinger MD, van der Roest HG, et al. Antipsychotic drug interactions and mortality among nursing home residents with cognitive impairment. J Clin Psychiatry (2017) 78(1):e76–2. doi: 10.4088/JCP.15m10303
224. Rios S, Perlman CM, Costa A, Heckman G, Hirdes JP, Mitchell L. Antipsychotics and dementia in Canada: a retrospective cross-sectional study of four health sectors. BMC Geriatr (2017) 17(1):244. doi: 10.4236/psych.2017.814157
225. Hirdes JP, Major J, Didic S, Quinn C, Sinclair C, Bucek J, et al. Study protocol and baseline comparisons for a Pan-Canadian initiative to reduce inappropriate use of antipsychotics in long-term care homes. SAGE Open (2019) 9(1):2158244019835942. doi: 10.1016/j.psychres.2013.09.024
226. interRAI (Organization). ChYMH Coordinating Committee, interRAI (Organization). Instrument and Systems Development Committee, interRAI (Organization). interRAI child and youth mental health (ChYMH) assessment form and user's manual: for use with in-patient and community-based assessments; includes: interRAI adolescent supplement (ChYMH-A). 9.3. ed. Ann Arbor, Michigan: interRAI; (2015). viii, 191 pages p.doi: 10.1016/j.psychres.2017.05.018
227. Stewart SL, Hirdes JP, McKnight M, interRAI (Organization). ChYMH Screener Coordinating Committee, interRAI (Organization). ChYMH Coordinating Committee, interRAI (Organization). Instrument and Systems Development Committee, et al. InterRAI child and youth mental health-screener (ChYMH-S): assessment form and user's manual. 9.3. ed. Washington, DC: interRAI; (2017). p. p.doi: 10.1016/j.chiabu.2017.04.011
228. Arbeau K, Theall L, Willoughby K, Berman JMJ, Stewart SL. What happened? Exploring the relation between traumatic stress and provisional mental health diagnoses for children and youth. Psychology (2017) 8(14):2485–95. doi: 10.1007/s10578-017-0751-y
229. Baiden P, den Dunnen W, Stewart SL. Discharge of adolescents with mental health problems against medical advice: Findings from adult mental health inpatient facilities across Ontario, Canada. Psychiatry Res (2013) 210(3):1161–67. doi: 10.1080/14999013.2015.1014528
230. Baiden P, Stewart SL, Fallon B. The mediating effect of depressive symptoms on the relationship between bullying victimization and non-suicidal self-injury among adolescents: Findings from community and inpatient mental health settings in Ontario, Canada. Psychiatry Res (2017) 255:238–47. doi: 10.1186/s12913-016-1970-9
231. Baiden P, Stewart SL, Fallon B. The role of adverse childhood experiences as determinants of non-suicidal self-injury among children and adolescents referred to community and inpatient mental health settings. Child Abuse Neglect (2017) 69:163–76. doi: 10.1016/j.chiabu.2017.04.011
232. Lau C, Stewart SL, Saklofske DH, Tremblay PF, Hirdes J. Psychometric evaluation of the interRAI child and youth mental health disruptive/aggression behaviour scale (DABS) and Hyperactive/Distraction Scale (HDS). Child Psychiat Hum D (2018) 49(2):279–89. doi: 10.1016/j.psychres.2019.06.013
233. Stewart SL, Baiden P, den Dunnen W, Hirdes JP, Perlman CM. Prevalence and correlates of criminal activity in adolescents treated in adult inpatient mental health beds in Ontario, Canada. Int J Forensic Ment (2015) 14(1):33–4. doi: 10.1177/1178632919827930
234. Stewart SL, Hamza CA. The Child and Youth Mental Health Assessment (ChYMH): an examination of the psychometric properties of an integrated assessment developed for clinically referred children and youth. BMC Health Serv Res (2017) 17: 82. doi: 10.1186/s12913-016-1970-9
236. Lau C, Stewart SL, Saklofske DH, Hirdes J. Scale development and psychometric properties of internalizing symptoms: The interRAI Child and Youth Mental Health Internalizing Subscale. Psychiat Res (2019). 278: 235–241. doi: 10.1080/13607863.2017.1408773
237. Laspina N, Stewart SL. Examining service complexity in children with intellectual disability and mental health problems who receive inpatient or outpatient services. J Intellec Dev Disabil (2019) 44(4): 464–473.
238. interRAI (Organization). ChYMH-DD Coordinating Committee, interRAI (Organization). Instrument and Systems Development Committee, interRAI (Organization). InterRAI child and youth mental health and developmental disability (ChYMH-DD) assessment form and user's manual : for use with in-patient and community-based assessments ; includes: interRAI adolescent supplement (ChYMH-A). 9.3. ed. Ann Arbor, Michigan: interRAI; (2015). viii, 201 pages p. doi: 10.1093/ageing/afl038
239. Stewart SL, Morris JN, Berg K, Björkgren M, Declercq A, Finne-Soveri H, et al. interRAI 0-3 Collaborative Action Plans (CAPs) for use with the interRAI 0-3 Assessment Instrument. Version 9.3, Standard Edition. interRAI: Washington DC (In press). doi: 10.1007/BF03353428
241. Carpenter GI. Accuracy, validity and reliability in assessment and in evaluation of services for older people: the role of the interRAI MDS assessment system. Age Ageing (2006) 35(4):327–29. doi: 10.1093/ageing/26.suppl_2.27
242. Carpenter GI, Teare GF, Steel K, Berg K, Murphy K, Bjornson J, et al. A new assessment for elders admitted to acute care: reliability of the MDS-AC. Aging (Milano) (2001) 13(4):316–30. doi: 10.1016/j.archger.2011.07.005
243. Hawes C, Morris JN, Phillips CD, Mor V, Fries BE, Nonemaker S. Reliability estimates for the Minimum Data Set for nursing home resident assessment and care screening (MDS). Gerontologist (1995) 35(2):172–78. doi: 10.1016/S0840-4704(10)60127-5
244. Kim H, Jung YI, Sung M, Lee JY, Yoon JY, Yoon JL. Reliability of the interRAI Long Term Care Facilities (LTCF) and interRAI Home Care (HC). Geriatr Gerontol Int (2015) 15(2):220–8. doi: 10.1007/BF02287348
245. Sgadari A, Morris JN, Fries BE, Ljunggren G, Jonsson PV, DuPaquier JN, et al. Efforts to establish the reliability of the Resident Assessment Instrument. Age Ageing (1997) 26Suppl 2:27–0. doi: 10.1093/ageing/26.suppl_2.27
246. Wellens NI, Van Lancker A, Flamaing J, Gray L, Moons P, Verbeke G, et al. Interrater reliability of the interRAI Acute Care (interRAI AC). Arch Gerontol Geriatr (2012) 55(1):165–72. doi: 10.2307/2529310
247. Poss JW, Jutan NM, Hirdes JP, Fries BE, Morris JN, Teare GF, et al. A review of evidence on the reliability and validity of Minimum Data Set data 2008. SAGE Publications Sage CA: Los Angeles, CA. (2008) doi: 10.5116/ijme.4dfb.8dfd
248. Hirdes JP, Smith TF, Rabinowitz T, Yamauchi K, Perez E, Telegdi NC, et al. The Resident Assessment Instrument-Mental Health (RAI-MH): inter-rater reliability and convergent validity. J Behav Health Serv Res (2002) 29(4):419–32. doi: 10.1186/s12911-017-0547-9
253. Hirdes JP, Poss JW, Caldarelli H, Fries BE, Morris JN, Teare GF, et al. An evaluation of data quality in Canada's Continuing Care Reporting System (CCRS): secondary analyses of Ontario data submitted between 1996 and 2011. BMC Med Inform Decis Mak (2013) 13:27. doi: 10.1093/ageing/29.2.165
255. Jones K, Perlman CM, Hirdes JP, Scott T. Screening cognitive performance with the resident assessment instrument for mental health cognitive performance scale. Can J Psychiat (2010) 55(11):736–40. doi: 10.1186/1472-6963-13-457
256. Travers C, Byrne GJ, Pachana NA, Klein K, Gray L. Validation of the interRAI cognitive performance scale against independent clinical diagnosis and the mini-mental state examination in older hospitalized patients. J Nutr Health Aging (2013) 17(5):435–9. doi: 10.1111/j.1365-2850.2009.01429.x
257. Burrows AB, Morris JN, Simon SE, Hirdes JP, Phillips C. Development of a minimum data set-based depression rating scale for use in nursing homes. Age Ageing (2000) 29(2):165–72. doi: 10.1093/ageing/29.2.165
259. Foebel AD, Hirdes JP, Heckman GA, Kergoat MJ, Patten S, Marrie RA. Diagnostic data for neurological conditions in interRAI assessments in home care, nursing home and mental health care settings: a validity study. BMC Health Serv Res (2013) 13:457. doi: 10.1177/0840470415581231
260. Martin L, Hirdes JP, Morris JN, Montague P, Rabinowitz T, Fries BE. Validating the Mental Health Assessment Protocols (MHAPs) in the Resident Assessment Instrument Mental Health (RAI-MH). J Psychiatr Ment. Hlt (2009) 16(7):646–53. doi: 10.1177/0306624X13507040
261. Hirdes JP, Smith T, Smith T, Arocha J, Perez E, Nadkarni S, et al. Technical Report to Primary Health Care Transition Fund, Health Canada: “Enhancing the Use of interRAI Instruments in Mental Health - Making an Integrated Mental Health Information System Happen”. University of Waterloo;: Waterloo, ON Canada (2007). Contract No.: Reference Number: G03-05691 x-ref G03-02877. doi: 10.1016/j.ijlp.2013.04.008
262. Neufeld E, Perlman CM, Hirdes JP. Predicting inpatient aggression using the InterRAI risk of harm to others clinical assessment protocol: a tool for risk assessment and care planning. J Behav Health Serv Res (2012) 39(4):472–80. doi: 10.1016/S0840-4704(10)60291-8
264. Brown GP, Hirdes JP, Fries BE. Measuring the prevalence of current, severe symptoms of mental health problems in a canadian correctional population: implications for delivery of mental health services for inmates. Int J Offender Ther Comp Criminol (2015) 59(1):27–0. doi: 10.1016/j.jamda.2019.01.121
265. Fries BE, Schmorrow A, Lang SW, Margolis PM, Heany J, Brown GP, et al. Symptoms and treatment of mental illness among prisoners: a study of Michigan state prisons. Int J Law Psychiatry (2013) 36(3–4):316–25. doi: 10.1097/YCT.0000000000000461
267. Tempier R, Bouattane el M, Hirdes JP. Access to psychiatrists by French-speaking patients in Ontario hospitals: 2005 to 2013. Healthc Manage Forum (2015) 28(4):167–71. doi: 10.1007/s10488-018-0898-2
268. Perlman C, Kirkham J, Velkers C, Leung RH, Whitehead M, Seitz D. Access to Psychiatrist Services for Older Adults in Long-Term Care: A Population-Based Study. J Am Med Directors Assoc (2019) 20(5):610–6. doi: 10.5430/jha.v4n2p15
270. Little J, Hirdes JP, Daniel I. ALC status in in-patient mental health settings: evidence based on the Ontario mental health reporting system. Healthc Manage Forum (2015) 28(4):146–9. doi: 10.1080/0092623X.2011.582437
271. Little J, Hirdes JP, Perlman CM, Meyer SB. Clinical predictors of delayed discharges in inpatient mental health settings across Ontario. Adm Policy Ment Health (2019) 46(1):105–14. doi: 10.1176/appi.psy.48.4.309
272. Ismail Z, Arenovich T, Grieve C, Willett P, Addington D, Rajji TK, et al. Predicting hospital length of stay for geriatric and adult patients with schizophrenia. J Hosp Adm (2015) 4:15–2. doi: 10.1176/ps.62.1.pss6201_0097
273. Stewart SL, Kam C, Baiden P. Predicting length of stay and readmission for psychiatric inpatient youth admitted to adult mental health beds in Ontario, Canada. Child Adolesc Ment Health (2014) 19(2):115–21. doi: 10.1111/j.1365-2702.2011.03862.x
275. Perlman CM, Martin L, Hirdes JP, Curtin-Telegdi N, Perez E, Rabinowitz T. Prevalence and predictors of sexual dysfunction in psychiatric inpatients. Psychosomatics (2007) 48(4):309–18. doi: 10.3109/10826084.2013.765482
276. Rabinowitz T, Martin L, Montague P, Hirdes JP. Prevalence and correlates of urinary incontinence in a large cohort of psychiatric inpatients. Psychiatr Serv (2011) 62(1):97–100. doi: 10.1016/j.psychres.2013.01.017
277. Martin L, Rabinowitz T, Montague P. Urinary incontinence in inpatient psychiatry: prevalence, change and relationship to length of stay. J Clin Nursing (2012) 21(15–16):2219–27. doi: 10.1177/0840470415581230
278. Masson PC, Perlman CM, Ross SA, Gates AL. Premature termination of treatment in an inpatient eating disorder programme. Eur Eating Disord Review: Prof J Eating Disord Association (2007) 15(4):275–82. doi: 10.1002/erv.762
280. Stewart SL, Baiden P. An exploratory study of the factors associated with medication nonadherence among youth in adult mental health facilities in Ontario, Canada. Psychiatry Res (2013) 207(3):212–17.
282. Zuo S, Fries BE, Szafara K, Regal R. Valproic acid as a potentiator of metabolic syndrome in institutionalized residents on concomitant antipsychotics: fat chance, or slim to none? P T (2015) 40(2):126–32.
283. Hirdes JP, Morris JN, interRAI (Organization). MH CAP Coordinating Committee., interRAI (Organization). Instrument and Systems Development Committee. interRAI Mental Health Clinical Assessment Protocols (CAPs): for use with community and hospital-based mental health assessment instruments. 9.1. ed. Ann Arbor, Mich.: interRAI; (2011). xv, 197 p. p. doi: 10.1111/j.1365-2850.2009.01429.x
284. interRAI (Organization). ChYMH CAP Development Committee, interRAI (Organization). Instrument and Systems Development Committee, interRAI (Organization). InterRAI child and youth mental health collaborative action plans (CAPs): for use with the child and youth mental health assessment instrument. 9.3. ed. Washington, DC: interRAI; (2015). xvi, 239 pages p. doi: 10.1007/s10597-010-9308-2
285. interRAI (Organization). ChYMH-DD CAP Development Committee, interRAI (Organization). Instrument and Systems Development Committee, interRAI (Organization). InterRAI child and youth mental health and developmental disability collaborative action plans (CAPS): for use with the ChYMH-DD assessment instrument : version 9.3. Washington, DC: interRAI; (2016). xvi, 188 pages p. doi: 10.1007/s10597-014-9694-y
286. Steel RK, Morris JN, interRAI (Organization), interRAI (Organization). Palliative Care CAP Coordinating Committee., interRAI (Organization). Instrument and Systems Development Committee. interRAI palliative care clinical assessment protocols (PC CAPs): for use with the palliative care assessment instrument. 9.1. ed. Ann Arbor, Mich.: interRAI; 2013. xi, 57 p. p. (2013) doi: 10.1097/00006199-200603001-00012
287. Martin L, Hirdes JP, Morris JN, Montague P, Rabinowitz T, Fries BE. Validating the Mental Health Assessment Protocols (MHAPs) in the Resident Assessment Instrument Mental Health (RAI-MH). J Psychiatr Ment Health Nurs (2009) 16(7):646–53. doi: 10.1186/s12877-015-0146-5
288. Mathias K, Hirdes JP, Pittman D. A care planning strategy for traumatic life events in community mental health and inpatient psychiatry based on the InterRAI assessment instruments. Community Ment Health J (2010) 46(6):621–7. doi: 10.1007/s10597-010-9308-2
289. Doran DM, Harrison MB, Laschinger HS, Hirdes JP, Rukholm E, Sidani S, et al. Nursing-sensitive outcomes data collection in acute care and long-term-care settings. Nurs Res (2006) 55(2 Suppl):S75–81. doi: 10.1186/1472-6963-10-96
290. Foebel AD, van Hout HP, van der Roest HG, Topinkova E, Garms-Homolova V, Frijters D, et al. Quality of care in European home care programs using the second generation interRAI Home Care Quality Indicators (HCQIs). BMC Geriatr (2015) 15:148. doi: 10.1186/s12877-018-0842-z
291. Gray L, Morris JN, interRAI (Organization). Instrument and Systems Development Committee., interRAI (Organization). interRAI clinical and management applications manual for use with the interRAI acute care assessment instrument: scales, screeners, problems, clinical action points, and quality indicators. 9.1. ed. Ann Arbor, Mich.: interRAI; (2013). 75 p. doi: 10.1186/1471-2318-13-127
292. Guthrie DM, Harman LE, Barbera L, Burge F, Lawson B, McGrail K, et al. Quality indicator rates for seriously Ill home care clients: analysis of resident assessment instrument for home care data in six canadian provinces. J Palliat Med (2019). 11: 1346–1356. doi: 10.1093/geront/44.5.665
297. White N, Poss J. An embarrassment of data: how e-assessments are supporting front line clinical decisions and quality management across Canada and around the world. Stud Health Technol Inform (2009) 143:155–60. doi: 10.1093/geront/42.4.462
298. Information CIfH. Your Health System - Antipsychotic Use Ottawa: CIHI; 2019 [Available from: https://yourhealthsystem.cihi.ca/hsp/indepth;jsessionid=78ktV1ghtjJhqDtSoo0tjIKQ.yhs?lang=en#/indicator/008/2/C5001/ . (2019) doi: 10.1007/s11414-007-9088-9
299. Perlman CM, Hirdes JP, Barbaree H, Fries BE, McKillop I, Morris JN, et al. Development of mental health quality indicators (MHQIs) for inpatient psychiatry based on the interRAI mental health assessment. BMC Health Serv Res (2013) 13:15. doi: 10.1111/jir.12423
300. Hirdes JP, Poss JW, Curtin-Telegdi N. The Method for Assigning Priority Levels (MAPLe): a new decision-support system for allocating home care resources. BMC Med (2008) 6(1):9. doi: 10.1177/1178632919827930
301. Fries BE, Shugarman LR, Morris JN, Simon SE, James M. A screening system for Michigan's home-and community-based long-term care programs. Gerontologist (2002) 42(4):462–74. doi: 10.1177/0840470414551866
302. Gibbons C, Dubois S, Ross S, Parker B, Morris K, Lim T, et al. Using the resident assessment instrument-mental health (RAI-MH) to determine levels of care for individuals with serious mental illness. J Behav Health Serv Res (2008) 35(1):60–0. doi: 10.1111/j.1532-5415.1991.tb05899.x
303. Stewart SL, Falah Hassani K, Poss J, Hirdes J. The determinants of service complexity in children with intellectual disabilities. J Intellect Disability Res (2017) 61(11):1055–68. doi: 10.1097/00005650-199407000-00002
304. Stewart SL, Poss JW, Thornley E, Hirdes JP. Resource Intensity for Children and Youth: The Development of an Algorithm to Identify High Service Users in Children's Mental Health. Health Serv Insights (2019) 12: 1–11. doi: 10.1177/1178632919827926
307. Fries BE, Schneider DP, Foley WJ, Gavazzi M, Burke R, Cornelius E. Refining a case-mix measure for nursing homes: Resource Utilization Groups (RUG-III). Med Care (1994) 32(7):668–85. doi: 10.1177/1744629511413506
308. Turcotte LA, Poss J, Fries B, Hirdes JP. An Overview of International Staff Time Measurement Validation Studies of the RUG-III Case-mix System. Health Serv Insights (2019) 12:1–11. doi: 10.1186/1472-6963-13-137
310. Fries BE, James ML, Martin L, Head MJ, Park PS. A case-mix system for adults with developmental disabilities. Health Serv Insights (2019) 12: 1–8. doi: 10.1097/00005650-198407000-00002 1178632919856011
311. Martin L, Fries BE, Hirdes JP, James M. Using the RUG-III classification system for understanding the resource intensity of persons with intellectual disability residing in nursing homes. J Intellect Disabil (2011) 15(2):131–41. doi: 10.1097/00005650-198905000-00009
315. Ashcraft ML, Fries BE, Nerenz DR, Falcon SP, Srivastava SV, Lee CZ, et al. A psychiatric patient classification system. An alternative to diagnosis-related groups. Med Care (1989) 27(5):543–57. doi: 10.1017/S204579601300067X
320. Minoletti A, Alegría M, Barrioneuvo H, Grana D, Hirdes JP, Perez E, et al. Translating psychiatric diagnosis and classificiation into public health usage. Public Health Aspects Diagnosis Class. Ment Behav Disord (2012), 201–41. doi: 10.1590/1982-02752017000400006
321. Hirdes JP, Fries BE, Botz C, Ensley C, Marhaba M, Perez E. The System for Classification of In-Patient Psychiatry (SCIPP): a new case-mix methodology based on the RAI–Mental Health (RAI-MH). Toronto ON, Canada: interRAI (2003). doi: 10.1590/1413-81232014201.21152013
322. Ontario Joint Policy and Planning Committee MHTWG. Grouper and weighting methodology for adult inpatient mental health care in Ontario a technical report. Toronto, ON Canada: JPPC; (2008). doi: 10.26633/RPSP.2018.136
326. Minoletti A, Soto-Brandt G, Sepúlveda R, Toro O, Irarrázaval M. Capacidad de respuesta de la atención primaria en salud mental en Chile: una contribución a Alma-Ata. Rev Panamericana Salud Pública (2018) 42:e136. doi: 10.1038/nature11283
328. Urbanoski KA, Mulsant BH, Willett P, Ehtesham S, Rush B. Real-world evaluation of the Resident Assessment Instrument-Mental Health assessment system. Can J Psychiatry (2012) 57(11):687–95. doi: 10.1177/070674371205701106
329. Perlman C, Martin L, Hirdes J. Using routinely collected clinical assessments in mental health services: the resident assessment instrument-mental health. Can J Psychiatry-Revue Can Psychiatrie (2014) 59(7):399.
331. Jonsson T, Stefansson H, Steinberg S, Jonsdottir I, Jonsson PV, Snaedal J, et al. Variant of TREM2 associated with the risk of Alzheimer's disease. New Engl J Med (2013) 368(2):107–16. doi: 10.1056/NEJMoa1211103
333. van der Wolf E, van Hooren SAH, Waterink W, Lechner L. Well-being in elderly long-term care residents with chronic mental disorder: a systematic review. Aging Ment Health (2019) 23(3):287–96. doi: 10.1093/geront/35.2.172
Keywords: care planning, outcomes, quality, case-mix, psychometric properties, homelessness, integration
Citation: Hirdes JP, van Everdingen C, Ferris J, Franco-Martin M, Fries BE, Heikkilä J, Hirdes A, Hoffman R, James ML, Martin L, Perlman CM, Rabinowitz T, Stewart SL and Van Audenhove C (2020) The interRAI Suite of Mental Health Assessment Instruments: An Integrated System for the Continuum of Care. Front. Psychiatry 10:926. doi: 10.3389/fpsyt.2019.00926
Received: 14 August 2019; Accepted: 21 November 2019;
Published: 17 January 2020.
Edited by:Barbara D'Avanzo, Mario Negri Pharmacological Research Institute, Italy
Reviewed by:Giacomo Deste, Civil Hospital of Brescia, Italy
Frances Louise Dark, Metro South Addiction and Mental Health Services, Australia
Copyright © 2020 Hirdes, van Everdingen, Ferris, Franco-Martin, Fries, Heikkilä, Hirdes, Hoffman, James, Martin, Perlman, Rabinowitz, Stewart and Van Audenhove. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: John P. Hirdes, firstname.lastname@example.org