Criteria for inclusion were (a) observational measures of momentary well-being (b) assessed by independent observers (c) during direct observation or video-recordings, containing (d) observable operationalizations of well-being such as positive and negative emotions/affect, or behavioral displays of satisfaction or engagement. Instruments should assess well-being (e) before, during and/or after psychosocial interventions over (f) short time intervals (minutes or hours). At least one psychometric property should be reported, and g) instruments not exclusively assessing well-being could be included, but only the well-being domain would be assessed. Instruments developed for the general population could be included if they also were specifically tested in people with dementia. Only English peer-reviewed journal articles were included.

Exclusion criteria were observational instruments (a) focusing merely on ill-being, such as negative emotions, anxiety, depression or neuropsychiatric symptoms, and instruments measuring (b) observable physiological indicators of well-being only (such as biomarkers or startle reflex).

Searches were performed on April 21st, 2020, and repeated on April 06th, 2021, in the databases MEDLINE, EMBASE, PsycINFO (all via OVD), Web of Science^TM, CINAHL (via EBSCOhost) and ProQuest^® (Psychology and Nursing and Allied Health). A combination of the words “well-being,” “dementia,” “observation,” “measurement,” and “psychometric properties” were searched for, using both Boolean operators and truncations. We utilized the published search filter with words describing measurement properties of outcome instruments from Terwee et al. (2009). The full search strategy corresponding to the databases is available in Supplementary Appendix A.

The search was limited to peer-reviewed journal articles, searching in title, abstract and subject headings. In addition, we hand-searched reference lists of relevant reviews, investigated reference lists and forward chained citations of the included publications. Authors of relevant articles were contacted when the publication did not provide the full observation tool. Other publication types, such as conference proceedings, editorials and books were excluded (Prinsen et al., 2018), as were articles where the instrument was not accessible and lacked a full description of the operationalizations of the items in the publication.

The first author (KM) carried out the searches in the databases, imported the results to Endnote^® (RRID:SCR_014001) where the results were checked, and duplicates removed. Next, KGM screened the titles and imported the records eligible for screening of abstract to Rayyan QCRI^® (RRID:SCR_017584). KM also conducted hand searches of relevant records and imported these to Rayyan. The first (KM) and last (IN) author independently screened the records in Rayyan based on the eligibility criteria. Next, the results from the independent screening were compared, and all conflicts and their solutions of abstract screening were logged to ensure transparency. The next step was to evaluate the included publication based on full text. KM and IN read the full text independently and evaluated the publications against eligibility criteria in team meetings.

Extraction was conducted by the first author (KM) and reviewed by a team including three of the authors (KM, EF-G, and IN). 20% of the data was extracted twice by the first author (KM) to ensue correct extraction. The extraction procedure was predefined and based on the COSMIN extraction tables (Prinsen et al., 2018). The first category addressed conceptualization (overarching conceptualization of well-being, population the instrument was developed in, and well-being domains assessed). The second category addressed central study characteristics (population, setting, methods, and results) for publications reporting on any of the measurement properties “content validity,” “structural validity,” “internal consistency,” “cross-cultural validity/measurement invariance,” “reliability,” “measurement error,” “construct validity” through hypothesis testing, and “responsiveness” (Mokkink et al., 2017; Prinsen et al., 2018; Terwee et al., 2018). The third category addressed feasibility (procedure, granularity, concreteness, training, requirements) and interpretability (measurement level and scoring, primary recording units, distribution, and sensitivity; Bakeman and Quera, 2012; Mokkink et al., 2017; Prinsen et al., 2018; Terwee et al., 2018).

Granularity refers to how fine grained and detailed the instrument is. Concreteness refers to how physically based the items are, where high concreteness involves bodily movement and low concreteness allows for interpretation of inner states. Measurement level defines which research questions may be asked, from nominal and ordinal to continuous output. Lastly, the primary recording unit defines how you sample the observations, from counting specific events in continuous or pre-specified intervals, to continuous recordings of duration (Bakeman and Quera, 2012; Chorney et al., 2015).

An overview of the COSMIN-definitions of central measurement properties of health-related instruments are provided in Table 1.

Study specific RoB-ratings from multiple sources per instrument were ranked with the categories “very good,” “adequate,” “doubtful,” “inadequate,” and “not applicable.” RoB-ratings were conducted by KM and IN in collaboration. Conflicting ratings were discussed with EF-G or NP. Rating criteria were based on the COSMIN RoB Checklist (Mokkink et al., 2017; Prinsen et al., 2018). The COSMIN-framework is created for patient-reported measurement instruments. To fit the COSMIN evaluations to the specific requirements for observational measures, some adaptations to the COSMIN-criteria were necessary. These mainly regarded the evaluation of content validity of the instruments. Our adaptations were based on recommendations from Bakeman and Quera (2011) and Bakeman and Quera (2012), and can be found in the Supplementary Material (Supplementary Table 1).

Consensus-based Standards for selection of health Measurement Instruments (COSMIN)-criteria for the content validity of self-reported measures are strongly based on feedback from the target group to assess relevance, comprehensiveness, and comprehensibility of the content of an instrument. Criteria for “relevance” requires items to be relevant for the construct of interest, the target population, and the context of use. To be “comprehensive,” the items need to cover all key aspects of the construct (Terwee et al., 2018). We adapted the evaluations of content validity to observational measures based on Bakeman and Quera (2012); Chorney et al. (2015); and Perugia et al. (2018b). To get an “adequate” or “good” rating of content validity, our team decided at least two of the following approaches were required: theoretical approaches with literature reviews, qualitative field work and development of coding scheme or ethogram, and quantitative survey or qualitative interviews including the target group (people with dementia or their close care givers and/or experts from all relevant disciplines). In addition, lack of pilot field testing followed by evaluation and revision of the “comprehensibility” of the instrument lead to a rating of “inadequate.”

Content validity is context- and population specific, implying that in this review the instruments’ content validity is evaluated for the specific construct (well-being) in the specific context of evaluating psychosocial interventions for persons living with dementia (Terwee et al., 2018). Thus, evidence of content validity in other populations or contexts may not be generalizable and are not included.

As lack of a priori hypotheses is a common bias in health-related measurement development, we used a recommended generic hypothesis from COSMIN for evaluating construct validity and responsiveness (Prinsen et al., 2018, Table 4, p. 1154). COSMIN recommends similar constructs to be evaluated against a threshold of ± ≥ 0.5, and related but dissimilar constructs to be evaluated against a threshold of ± ≥ 0.3. Defining constructs as similar or only related a priori is a complex task. Relevant sources of measurement error identified in previous reviews are: (1) comparisons between state or trait dimensions (Curyto et al., 2008); (2) comparing self-, proxy- and observer-rated measures (Ferring and Boll, 2010); and (3) comparing instruments with different timeframes (Shiffman et al., 2008). Thus, we chose to use the recommended threshold of ± ≥ 0.3 as our threshold of comparison.

In addition, we did not expect decreasing well-being-scores to correlate with increasing dementia severity or cognitive impairment, as these constructs are found to be independent in several reviews (e.g., Missotten et al., 2008; Martyr et al., 2018).

Inter-rater reliability and agreement are particularly important properties of observational measures, and the new COSMIN-consensus regarding ratings of reliability and measurement error for clinician rated instruments was incorporated (Mokkink et al., 2020). The principle for overall quality scorings is ‘the worst score counts’, and one uses the lowest rating of the measurement property to indicate RoB (Mokkink et al., 2017; Prinsen et al., 2018; Terwee et al., 2018). COSMIN guidelines are available at www.cosmin.nl.

After the initial study specific evaluation, the total evidence provided for each instrument was rated against adapted COSMIN quality criteria using the ratings “good” (+), “unclear” (?), “inadequate” (-), = “conflicting” (±), “not evaluated” (NE), and “not applicable” (NA). Table 2 provides an overview of the quality criteria. As most instruments were investigated in one publication only, no quantitative data synthesis was obtainable except for construct validity. For construct validity, the summarized number of hypotheses supporting the construct was divided by the sum of hypotheses (Prinsen et al., 2018).

The trustworthiness of the summarized quality criteria rating was ranked with Grading of Recommendations Assessment, Development and Evaluation (GRADE) principles (GRADE Handbook, 2013), modified in the COSMIN approach for the context of health-related outcome measures (Prinsen et al., 2018). Four factors are assessed on instrument level: “risk of bias,” “inconsistency,” “imprecision,” and “indirectness” of the evidence, graded as “high,” “moderate,” “low,” or “very low”. Ratings were conducted in team meetings with KM and IN, including EF-G if consensus was not met.

Search results and reasons for exclusion is presented in Figure 2. After removing duplicates, KM screened 4309 records by title. Then, the 255 publications eligible for evaluation of abstracts was blind screened for inclusion by KM and IN (82% agreement). Additionally, 25 publications were added through hand search of relevant records. After full-text review of 87 records by KM and IN, 36 articles describing a total of 22 instruments were included, of which three originated from the hand-search.

Key characteristics of the instrument, target population and domains are presented in Table 3. The included instruments are sorted in the three (not mutually exclusive) categories (a) observations of emotions, (b) observations of positive behavioral expressions, and (c) observations of engagement. Instruments are presented in chronological order within each category.

Extracted data on measurement properties and study characteristics are reported in Supplementary Table 2 together with the study specific RoB-ratings. As most publications use data from repeated observations of the same subjects, both number of participants and number of observations are reported when available. Measurement properties are presented under three headings: (a) content validity, (b) construct validity, including structural validity, measurement invariance and hypothesis testing (for construct validity), and (c) reliability, including internal consistency, inter-rater, intra-rater or test–retest reliability, and measurement error. None of the included publications reported cross-cultural validity and responsiveness, using the methodological definition and criteria of COSMIN (see Tables 1, 2).

The ratings against quality criteria for the available evidence of the measurement properties on instrument level are presented in Table 4. Ten of 22 instruments had only one publication describing the development and measurement properties. More than half of the instruments were developed or tested in small samples [11 of 36 studies have n < 20, mean n = 89.4 (SD = 102)]. The trustworthiness of the summarized result per property evaluated by the GRADE approach (GRADE Handbook, 2013; Prinsen et al., 2018) are presented in Table 4.

Extracted data regarding feasibility and interpretability are reported in Supplementary Table 3. Additional publications from the search process describing use of the instrument in clinical settings or research were extracted here.

Problems with skewed distributions or low frequencies of negative emotions, behaviors or expressions are reported for the majority of the instruments (Cfr. Supplementary Table 3). This complicates parametric approaches assuming a normal distribution of items. We suggest that assessing psychosocial interventions for people living with dementia should mainly focus on increases in well-being. Negative symptoms in dementia have a diversity of causes, some of which will necessarily be less modifiable by psychosocial interventions (Kales et al., 2015; Kolanowski et al., 2017; Livingston et al., 2017). However, momentary well-being is particularly achievable through modifying environmental factors (Lawton, 1994; Kolanowski et al., 2020). Moving the focus from ill-being (such as agitation or apathy) to well-being, has three advantages. First, it will decrease the labor intensiveness of the observational assessment because less items are assessed. Second, it will bring about data better fitted for statistical approaches because the distribution of ill-being items in the clinical studies using these instruments often were skewed and not normally distributed (see Asplund et al., 1995; Magai et al., 1996, 2002; Perrin, 1997; Lawton et al., 1999; Judge et al., 2000; Beck et al., 2002; Kinney and Rentz, 2005; Wood, 2005; Phillips et al., 2010; Casey et al., 2014; Beerens et al., 2016; de Boer et al., 2016; Lokon et al., 2019). Lastly, it will increase the likelihood of correct conclusions about the positive effects of the psychosocial interventions because this is operationally defined as an increase in positive expressions and not as a decrease in negative expressions. Ill-being should still be monitored during psychosocial interventions, but the absence of ill-being is not synonymous with well-being (Martyr et al., 2018).

While 15 of 22 instruments could detect statistically significant changes, definitions to guide interpretation of these change-scores were not provided. An option for future studies is to calculate MIC and the SDC or LoA (de Vet et al., 2011) for continuous level instruments. MIC is important because it is defined as the smallest clinical meaningful change as evaluated by patients or clinicians (de Vet and Terwee, 2010). SDC indicates whether change scores are reflecting a “true” change in the construct, as opposed to expected random error or natural fluctuation. Test–retest values may be used to calculate SDC for continuous scores (Prinsen et al., 2018; Mokkink et al., 2020). Several instruments were operationalized at a nominal or ordinal level, while using total score as continuous in statistical analyses. However, using the total score implies that the score reflects, predicts, or describes well-being validly. Although several instruments claim the total score to reflect level of well-being or engagement, adequate evidence of this relationship is rarely provided. Specifically, the formative instruments are hampered by unclear clinical interpretation.

Test–retest reliability reflects the instrument’s measurement error in repeated measurement of stable constructs (de Vet et al., 2011). This required “stability” may be unattainable for fluctuating phenomena such as pain. In this review, several instruments provide evidence suggesting momentary well-being in dementia is a fluctuating phenomenon (Clare et al., 2012; McDermott et al., 2014; Junge et al., 2020). Fluctuations in the construct of interest between measurements creates an ambiguous reliability estimate (Jensen, 2003) and discarding instruments with a cut-off score < 0.70 (Prinsen et al., 2018) is not necessarily useful in this context. It is reasonable to assume test–retest scores reflect a natural fluctuation or variability in well-being in people with dementia, as the presence of neuropsychiatric symptoms such as apathy are episodic and fluctuating as well (Kales et al., 2015). Examining the natural variation of the construct by investigating test–retest reliability is nevertheless important, as the range of variation in fluctuating constructs influence the accuracy when interpreting scores of an instrument. Thus, a clinically significant score needs to be larger than the measurement error inflicted by this natural variation (de Vet et al., 2011). If test–retest reliability is not investigated, we cannot know if the measure can detect change in the observed persons beyond measurement error (Mokkink et al., 2020). This is a significant problem, that may lead to erroneous conclusions in both research and care. In addition, adjusting the interval of the repeated measurements to increase the likelihood of stability is essential, as longer time intervals may reflect the degenerative path of dementia and not instrument reliability.

Developing fine grained instruments used for ecological momentary sampling requires repeated assessment of the same subjects (Shiffman et al., 2008). Investigating behavior as it unfolds over time is labor intensive, and naturally includes smaller samples, often with numerous repeated observations. Standard approaches to develop self-rated instruments require large samples to investigate structural validity with factor analysis (N > 100), or scalability through for example Mokken analysis (N > 2000; Prinsen et al., 2018). Investigating large samples in labor intensive instruments is in many cases unrealistic. Additionally, using serially dependent repeated observations in the same subjects to increase the sample size violates basic assumptions required for these methods (Manolov and Moeyaert, 2017).

Most instruments in this review require further investigations of construct validity to ensure that the output is consistent with the underlying theoretical constructs. Comparisons with global rating scales are recommended when investigating the construct-validity of new instruments (de Vet et al., 2011). While developing COMMUNI-CARE, a validated clinician-rated global scale was used for this purpose (Lopez et al., 2016), but the same non-blinded investigator was rating both scales, contributing to a considerable risk of bias. In OME (Cohen-Mansfield et al., 2009), a similar approach is used, only with blinded ratings of a non-validated global engagement-scale. Thus, investigating construct validity through correlations with similar instruments is a challenge in the face of a lack of a “gold standard measure,” as one must rely on existing instruments with their respective limitations (de Vet et al., 2011). Sometimes the hypothesized correlations included comparisons of well-being levels from long-term versus momentary instruments (Clare et al., 2012). Well-being states and traits do not necessary correlate (Curyto et al., 2008; Cohen-Mansfield, 2011). Therefore, investigating correlations with other momentary assessment approaches is recommended.

When assessing momentary well-being in dementia, two domains seem important to control for to interpret changes in well-being scores more accurately. Several of the instruments included in this review have a well-being score that is associated with (1) dementia severity or (2) level of function. However, research suggests that these constructs are not expected to be systematically related (Missotten et al., 2008; Barca et al., 2011; Cohen-Mansfield, 2011; Martyr et al., 2018). This has implications for how we interpret changes in well-being scores over time. If well-being scores of a particular instrument are lowered as a consequence of the dementia progressing, is this reflecting lack of treatment effect, poorer dementia care, or neurodegenerative development? Future studies assessing the measurement properties of these instruments should assess if a relationship between well-being and dementia severity or level of function is present. Such covariance may indicate that the instrument is tapping both cognitive functioning as well as well-being (for example if the score is relying on verbal expression). Understanding these relationships is required to accurately interpret changes in well-being scores during psychosocial interventions.

Personal well-being refers to a subjective evaluation, and observational measures use behavioral expressions to infer about an inner state. Hence, the most crucial property of a measurement instrument is content validity. Content validity will vary with the context, population, and construct to be measured, and affects all other psychometric properties of an instrument (Terwee et al., 2018). Together with agreement between observers, these two aspects are considered the most important for observational instruments (Bakeman and Quera, 2012; Chorney et al., 2015). Moreover, evidence of structural validity or construct validity, ensuring that an increase in the score reflects an increase in the construct, is important when making inferences about inner states. Cross-validating scores with other instruments, particularly self-report instruments, will strengthen this.

As no evidence of cross-cultural validity or responsiveness was detected, special attention to investigating this knowledge-gap and establishing these properties are important in future studies using any of the instruments in this review. In relation to cross-cultural validity, we make the following recommendation: Behavioral expressions of momentary well-being are likely to differ across cultures (Lim, 2016). Thus, securing cross cultural validity by establishing content validity in new cultural contexts is in our evaluation an alternative to statistical evaluation of cross-cultural validity for observational measures. This can be achieved through the recommended qualitative approaches involving clinical expertise from people with dementia, family- and professional caregivers, as well as clinical experts and field testing (Terwee et al., 2018).

In relation to the lack of responsiveness, we make the following recommendation: Several instruments have provided evidence of their capacity to statistically detect changes in intervention studies (Conf. Supplementary Table 3). However, this is not adequate evidence of responsiveness, as we do not know if the lack of detecting change is due to lack of responsiveness or lack of intervention effect. Responsiveness of these instruments needs to be investigated through correlations with change-scores in similar instruments (de Vet et al., 2011).

The clinical utility of an instrument is specific to the context and aims of the user, and is influenced by its feasibility, interpretability, benefits, and shortcomings (Smart, 2006; Terwee et al., 2018). To recommend a specific instrument to assess observed well-being is not our intention. However, we generally recommend identifying instruments with proper conceptualizations, which are feasible for the specific purpose, context, and target population (Terwee et al., 2018). Choosing instruments with acceptable content validity should be followed by investigation or adaptation to solve the additional instrument-specific issues addressed in this review. An overview of the issues of each instrument is provided in Table 4, Supplementary Tables 2, 3. Establishing or evaluating if the instrument has good content validity in the applied context is vital, especially in securing relevance and comprehensiveness (Chorney et al., 2015).

A final note worth commenting regards the large number of instruments identified in the hand search, of which three were included in this review. This suggests that researchers may not be choosing appropriate keywords when publishing articles relating to observational measures for people living with dementia.

Of the instruments measuring emotions with acceptable content validity (OERS and MAX), OERS is the most frequently used (Lee et al., 2019) instrument with the most extensively documented psychometric properties (Lawton et al., 1996, 1999). MAX (and FACS) requiring a close view of the face; problems with interpreting facial movement in persons wearing glasses, having facial hair, or facing more than 45 degrees away from the camera (Cohn et al., 2007) reduces the clinical utility of these instruments in people living with dementia. Thus, the feasibility of instruments relying on facial expressions and excluding bodily expressions may decrease the instruments’ sensitivity to detect expressions of well-being in the dementia population (Seidl et al., 2012). However, as negative emotions are infrequent, the feasibility of the full OERS scale in research and clinical setting is limited (Algar et al., 2016). Thus, for investigating well-being in people with dementia, the positive emotions in OERS may be best suited. However, from these findings, emotions in people with mild to moderate dementia seem to be best measured through self-report (instruments are reviewed in Ferring and Boll, 2010; Stoner et al., 2019; and Clarke et al., 2020).

Users looking for instruments investigating positive expressions are recommended to consider any instruments with acceptable content validity (DCM 8, PRS, ACT, GCWBT, SM-GCWBT, AwareCare, BEAM, MEDLO-tool and QUALIDEM ILA). PRS and MEDLO-tool are instruments with high granularity, detecting changes on micro-levels that offer interval-sampling from 30 s to 2 min. While DCM, ACT, GCWBT, SM-GCWBT offers somewhat fine-grained observations (5-10 min), AwareCare offers fine-grained observations as they unfold over time, and BEAM consists of both fine-grained and aggregated scores. QUALIDEM-ILA is best suited for total evaluations of interventions (30-45 min). Users looking for behavioral or movement-anchored operationalizations of positive expressions with high levels of concreteness may look at PRS, ACT and AwareCare. DCM, GCWBT, SM-GCWBT, BEAM, MEDLO-tool and QUALIDEM-ILA offer more contextual cues and social interpretations.

AwareCare appears clinically useful for people with very severe dementia, and BEAM is feasible for moderate dementia. AwareCare detected signs of awareness in all participants and suggests a clinically useful index for interpretation as well (Clare et al., 2012). PRS needs to be investigated in a larger sample but is a promising tool in very severe dementia (Perrin, 1997). BEAM covers behavior, engagement, and affect, through direct observation in various settings without being very labor intensive and while avoiding observer’s fatigue (Casey et al., 2014). Further investigation of its construct validity may, however, be required, in addition to an improved evaluation of inter-rater reliability. The clinical sensitivity of DCM has been questioned (Cooke and Chaudhury, 2013), and the well-being (ME-score) of DCM 8 is probably not sensitive enough to detect clinical change reliably in intervention studies on a group level. DCM 8 seems better suited for clinical practice (Villar et al., 2015) on an individual level (Brooker and Surr, 2006). MEDLO-tool’s mood score is based on DCM as well, and shows the same problems (Beerens et al., 2016; de Boer et al., 2016), lowering the utility of this instrument for assessing well-being. ACT is based on a thorough development (Wood, 2005), and seems like a feasible and clinically useful instrument, but needs further investigation of construct validity. GCWBT should be omitted due to evidence of low structural validity (Gross et al., 2015), but the revised SM-GCWBT needs further modification and investigation of a proposed two-factor structure, as well as exclusion of some unrelated items (Lokon et al., 2019). Further investigation of QUALIDEM-ILA, in terms of both inter-rater reliability and use in clinical/research contexts are required (Junge et al., 2020). Still, QUALIDEM-ILA is one of the most recent instruments included in this review, and further publications are expected.

Of the instruments assessing engagement with acceptable content validity (OME, VC-IOE, EPWDS, ELICSE, MiDAS, and MTED), users searching for instruments with high granularity may look at VC-IOE or ELICSE (both continuous sampling), EPWDS or MiDAS (5-min intervals), or OME (15 min including both duration-based and aggregated scores). MTED provides an aggregated score based on the intervention-session. ELICSE and VC-IOE offers the highest level of concreteness, and EPWDS, OME, MiDAS, and MTED is less concrete and more interpretative. However, higher levels of concreteness will often increase labor intensiveness (Bakeman and Quera, 2011) and offer broader generalizability, at the cost of lower sensitivity to individual variations. In clinical contexts, allowing for interpreting idiographic expressions of well-being may sometimes be an advantage.

Ethographic and Laban Inspired Coding System of Engagement (ELICSE) is based on an exemplary solid development-phase with subsequent theoretical and conceptual development (Perugia et al., 2018a,b, 2020). Nevertheless, the system is highly context specific to the manipulation of objects when sitting down and may not be as easily adaptable to other activities or clinical contexts. Developers of OME describe a need for further work on increasing the clinical utility of the scale (Cohen-Mansfield et al., 2011, 2012), and it is critiqued for lack of interpretability (Jones et al., 2015, 2018; Perugia et al., 2018b). VC-IOE needs further evaluation of reliability and construct validity (Jones et al., 2015). MiDAS strength is the inclusion of the target group in the development (McDermott et al., 2015), but needs further investigation of psychometric properties and is hampered by low intra-rater reliability (McDermott et al., 2014). MTED appear to be a good option when evaluating engagement in clinical music therapy processes, but the scale is not intended for evaluating intervention effect (Tan et al., 2019).

Engagement of a Person with Dementia Scale (EPWDS) stands out as a feasible, easily administered scale that may allow for assessing engagement in contexts other than robot-based play (Jones et al., 2018). Formal evaluation of its structural validity is required, but indications of test–retest reliability are promising given the common problems of low stability between assessments in this population.

The first strength of this review is that the protocol was pre-registered in PROSPERO. The second strength is that we used the most relevant systematic approach, the COSMIN-guidelines (Prinsen et al., 2018; Terwee et al., 2018; Mokkink et al., 2020). The third strength is that when required, these guidelines were adapted for evaluating observational instruments based on relevant literature (Bakeman and Quera, 2011, 2012; Chorney et al., 2015; Perugia et al., 2018b). The fourth strength is the extensive review of study-specific and instrument-specific evaluation and overarching methodological issues that provides relevant knowledge to both researchers and practitioners.

A first limitation of this review is that by including instruments reporting at least one psychometric property, instruments describing promising content validity only were not evaluated (such as Morse and Chatterjee, 2018). A second limitation is that the COSMIN-criteria of construct validity requires at least 75% of hypotheses to be supported. This may lead to somewhat unbalanced ratings, as publications reporting only one or two supportive correlations may be given a more positive rating than studies examining multiple correlations. However, testing several hypotheses provides more detailed knowledge about construct validity. A third limitation is the use of correlations of > 0.3 as the cutoff for supporting construct validity. This cutoff may seem low, and less conservative than the original suggestion of correlations ≥ 0.5 with instruments measuring similar constructs (Prinsen et al., 2018). However, the majority of the correlated instruments were assessing related and not similar constructs, indicating that correlations > 0.3 are an adequate expectation. Finally, the blinding procedure within our team of raters could have been more extensive, as completely blinded ratings are considered the gold standard (Mokkink et al., 2017).

Several instruments may validly assess well-being through observation in people with dementia. Evaluating their context specific clinical utility and content validity are more important than choosing the instrument with the best ratings or psychometric properties. However, piloting the instruments, investigating the effects of cultural context and study-specific inter-rater agreement and measurement error is advised. Moreover, utilizing an instrument in a clinical study provides the opportunity to investigate hypotheses that may further inform the construct validity. All measurement approaches come with some strengths and some weaknesses, and observational measures are vulnerable to misinterpretation when they are used to infer about inner states. Nevertheless, observations offer unique opportunities to investigate associations between external stimuli and well-being that can provide important knowledge of the usefulness of various interventions for people living with dementia.