Brain M-App’s Structure and Usability: A New Application for Cognitive Rehabilitation at Home

Introduction

The population of people aged 65 and up is expected to reach 2 billion by 2050 (Kinsella and Phillips, 2005), with major implications for health and social care planning and delivery. For instance, aging is linked to structural and physiological changes, as well as a lifelong accumulation of molecular and cellular damages caused by a complex network of maintenance and repair processes (Kirkwood, 2005). Frailty is a preclinical condition of increased vulnerability following minor stressor events, which is related with a reduction in reserve and function across a variety of physiological processes, including the ability to cope with daily or acute stresses (Fried et al., 2001; Walston et al., 2006; Eeles et al., 2012). It results from aging-related deterioration, which raises the risk of adverse effects, including falls, disability, delirium, and death (Clegg et al., 2013). Fried and colleagues (Fried et al., 2001) have operationalized this condition as the coexistence of three out of five phenotypic criteria that indicate compromised energy: low grip strength, low energy, slowed walking speed, low physical activity, and/or unintentional weight loss. However, these frailty impairments are accelerated, as are homoeostatic processes, which begin to fail at a faster rate (Ferrucci et al., 2002; Hahr, 2019).

Cognitive impairment may be linked with frailty, as Brigola et al. (2015) highlighted in their review. In fact, some recent research has begun to include cognition in the definition of frailty (Delrieu et al., 2016). Frail subjects, in particular, have deficiencies in executive, attention, free recall, and delayed free recall (Delrieu et al., 2016). Moreover, physical weakness is associated with the worst cognitive abilities (Delrieu et al., 2016). Based on these premises, “cognitive frailty” is defined as a clinical condition characterized by both physical frailty and cognitive impairment, without reaching the criteria for dementia.

Nevertheless, the state of frailty may be reversed (Gill et al., 2006; Xue, 2011) considering that many organ systems have physiological reserves needed to compensate for age-related and disease-related changes (Lipsitz, 2002).

In this regard, rehabilitation interventions attempt to restore a person’s functional capacity and avoid further deficits. The major goal is to assist patients in performing ordinary personal duties without the assistance of another person, or at the very least to remove the need for additional support, using adaptive approaches and facilities (Cameron and Kurrle, 2015). As a result, detecting frailty early and easily is critical to delaying its progression and preventing disability in the elderly. With the timely implementation of strategic rehabilitation measures, this procedure may be possible (Ma and Chan, 2020).

Depending on patient preferences and program availability, recovery services can be delivered in a wide variety of settings, including conventional hospital rehabilitation wards, outpatient facilities, acute hospital wards, and high-dependency units (Cameron and Kurrle, 2015). One example of an interdisciplinary rehabilitation program is the “Frailty Intervention Trial Program” (Fairhall et al., 2015) which includes geriatrics, physiotherapy, nursing, psychology, and health economics professionals.

Several personal health care and aging technologies have recently been proposed to help prevent, mitigate, or alleviate some of the most common illnesses that plague the aged.

New technology can be used in practically any situation where weak patients require assistance (Mugueta-Aguinaga and Garcia-Zapirain, 2017; Gallucci et al., 2021) for both motor and cognitive purposes (Pedroli et al., 2016, 2019b; Serino et al., 2017b). Among all technologies, Virtual Reality (VR) is one of the most successful examples of applications for cognitive rehabilitation (Riva et al., 2020). In this section, we will focus on its use for cognitive rehabilitation.

VR is a computer-generated technology that simulates life-like environments artificially, providing a “convincing illusion and a sensation of being inside an artificial world that exists only in the computer” (Tieri et al., 2018; Riva et al., 2019). Advances in virtual technologies have created platforms for displaying 3D objects in a dynamic, consistent, and accurate manner, allowing for the presentation of complex stimuli in a way that allows for both careful monitoring of laboratory measures and naturalistic observation of real-world situations (Matheis et al., 2007; Parsons, 2015).

Virtual-based interventions provide users a sense of presence or the sensation of being there in the virtual world. This sensation might be classified as a neuropsychological phenomenon resulting from our biological heritage and our experiences as active agents (Riva, 2008). The potential of virtual reality is based on the display and management of complex perceptual inputs that may be utilized to test neurocognitive and affective perception while participants are immersed in realistic simulations.

According to Riva and colleagues (Riva et al., 2006), individuals can benefit from the VR experience and boost their engagement to produce better results, by developing challenging and interactive tasks, resulting in promising cognitive and physical rehabilitation outcomes (Robinson et al., 2015; Mancuso et al., 2020; Riva et al., 2020).

This technology has been used in several therapeutic procedures. Among them, Pedroli et al. (2019a) proposed a VR-based motor protocol for frailty rehabilitation, implying a dual-task intervention (cognitive and motor).

In this regard, 360° videos are a newer technology that preserves many of the benefits of VR at a cheaper cost in terms of time and resources. They’re spherical recordings captured by sophisticated cameras with omnidirectional lenses that can collect images from all over the scene. Users can glance about the area as they would in a real-life scenario. Users can control the viewing direction by moving their head: looking up, they can see the sky/roof; looking down, they can see the floor/ground; and moving their head, they can see what is going on all around them. They’ve been used in a number of contests because of their adaptability, affordability, low prices, and ease of usage. Furthermore, 360° movies can be seen from a variety of perspectives: for example, if the camera is mounted on the user’s head while the video is being recorded, the user can experience the world from a first-person perspective. Otherwise, the user can enjoy the scene as an outside spectator by placing the camera in any section of it (third-person perspective). Thus, 360° videos can give immersive experiences by instilling a sense of presence, allowing to concentrate on the virtual experience by making users feel physically present in the scene. Several studies have shown that 360° videos are similar to real-world environments in terms of psychological, physiological (Higuera-Trujillo et al., 2017), and cortical responses (Robertson et al., 2016), despite the fact that they have not been properly regarded as realistic VR technology, providing a sub-optimal experience.

Additionally, these technologies are especially well-suited to home-based rehabilitation. Home-based rehabilitation activities can be an effective means of reducing frailty and decreasing cognitive decline. These initiatives are one of the most recent responses to the requirements of the aging and frail population, fostering independence, ensuring independent home-living, and resulting in positive quality-of-life outcomes (Stones and Gullifer, 2016). Moreover, aging in place is undoubtedly preferred by older people because it gives them control and influence on their lives, helping them preserve their identity and wellbeing (Cutchin, 2004). Residential solutions, on the other hand, might cause emotional stress, despair, loneliness, adaptation challenges, functional degradation, and diminished wellbeing (Chapin and Dobbs-Kepper, 2001).

Even for frailty rehabilitation, the promise of technology-based remote treatments in boosting elder independence has been well recognized (Kim et al., 2017; Serino et al., 2017a). For example, a study used 360° technologies to create a home-based motor rehabilitation program that can be accessed by watching movies on an iPad while doing cycling activities (Pedroli et al., 2020).

Overall, home-based rehabilitation allows patients to continue an intervention begun in a hospital setting while also ensuring support and assistance when access to health-care systems is limited, such as during the present pandemic situation.

Based on these premises, in the following section, we present Brain m-App, a tablet-based application designed for home-based cognitive rehabilitation of frail subjects, addressing spatial memory, attention, and executive functions. The app was created using the authors’ immersive High-end VR cognitive protocol described in a previous study (Pedroli et al., 2019a). The goal of this home-based cognitive app is to ensure that the rehabilitation process runs smoothly between the hospital and the patient’s home. Although using a Head-Mounted Display is advised for improved immersion and a sense of presence, it is not recommended for usage on its own without assistance for safety reasons. As a result, we created this tablet application to overcome this limitation, allowing 360° videos to be explored without the use of an immersive gear. The results of the first usability study are presented; the usability methodology is based on a previous paper (Pedroli et al., 2020), in which the authors designed and evaluated another home-based motor rehabilitation application. A pilot investigation of clinical use is discussed in the final section.

Brain m-App

Brain m-App was born within a project aimed at continuing the high-end rehabilitation started in the hospital at home. The theoretical foundation of Brain m-App is described in a previous paper (Pedroli et al., 2019a). The design process was based on the User-Centered Design (UCD). In particular we simplified the structure of the tasks, we made the things visible, we planned for possible errors and generally we made it easy to determine what actions are possible at any moment. These principles allow us to consider the global experience of the subjects during the interaction with the application (Gaggioli et al., 2017).

Description

The app starts with a screen that displays 10 buttons, each representing one of the training days. To make things easier, the app allows users to choose only the appropriate training day. Furthermore, the user’s options inside each screen are limited to reduce the risk of incorrect adherence to the procedure during the performance. Once the user begins the daily training session, he or she is only allowed to click on the buttons to advance to the next screen, pause or resume a video, and answer questions, with no ability to go backward. Every daily training session starts with a welcome screen that provides information about the session (number of the day) and reminds to perform each activity according to the instructions. When the user selects the button at the bottom of the welcome screen, the training begins. Depending on the training day, the user is presented with a variety of exercises ranging from 7 to 9, all of which follow the same flow (see Figure 1). The training day is complete when all of the exercises have been completed, and the application prevents it from being repeated.

FIGURE 1

Figure 1. Exercise’s flow: (A) select the day, (B) read the instructions, (C) watch a video, answer the question, and have feedback.

Application Architecture

The application has been developed using Unity, a cross-platform game engine developed by Unity Technologies. The application consists of a set of ad hoc scripts, C# code executed during the “play” state, managing the game’s behavior during its different tasks. Figure 2 shows the architecture of the application and the interaction between the scripts. The GameManager script is the core of the application. It is responsible for providing the graphic contents, texts, images, videos, audio, and buttons required by each step of the training session. It is in charge of managing the user’s interactions with the contents generated by giving positive or negative acoustic feedback depending on the user’s answer and showing texts describing the user’s choice results. Moreover, it manages the flow of the training session steps and, if the daily training session has been completed, it saves the information on the device, preventing the user from repeating the session. The (GM) script invokes the GameDayManager (GDM) script at each application launch. GDM is responsible for establishing if the current date is different from the one saved in memory. No date saved in the memory means that the application has never been launched before by the current user, and so the application will activate only the day 1 button. If a date is saved in memory and it is the same as the current date means that the user has already performed the training session: if the daily session has been completed, the buttons to choose the training day are all disabled, otherwise, the session is interrupted, and the user can repeat it from the beginning by clicking on the active button (current day) in the Day choice screen.

FIGURE 2

Figure 2. Architecture of the application.

Lastly, the GM script invokes the SwitchToCardboard script whenever the video is shown as a 360° video, allowing the user to view it in multiple directions from a fixed central point using the device’s gyroscope sensor.

Tasks

The Brain m-app is made up of 10 days of activities that include a variety of exercises. The activities were chosen based on those that patients used to do during their clinical practice in the hospital. They want to improve their independence and autonomy in daily tasks by training spatial memory, executive functions, and attention skills. Patients were asked to participate in two activities (days) every week for a total of 5 weeks, based on a previous pilot study which reported significant results after 10 sessions (Serino et al., 2017a). Every day, roughly 30 min are required. All of the tasks require patients to watch a 360° video while facing or holding the iPad. There are 22 distinct sorts of exercises, ranging in difficulty from easy to challenging, spread out across the 10 days. A group of exercises consists of watching the video and identifying, saying it aloud, the aim of the action, e.g., tidying the room or saying all the actions performed or listing the actions seen or recognizing unnecessary actions. All of these activities targeted attention and executive functions (planification, working memory, and inhibition of unnecessary information). Another group of exercises required to recall whether some objects were to their right or left, or if they were to the right or left of another object, after watching a video orto recall whether a certain sound (such as an alarm clock) came from their right or left. These exercises required patients to encode and then recall some information from both an egocentric and allocentric perspective (spatial memory). A tougher assignment requires patients to recognize the map of a home after exploring it by clicking on a sign located on each door as they move through the rooms. Another is to get to one room (for example, the bedroom) by starting from another (for example, the kitchen) and without passing through another (e.g., bathroom). Another activity requires patients to walk through the house and count how many target objects (such as tennis balls) they come across. All these exercises aimed to train both working memory and spatial memory. Patients were encouraged to appropriately set up the room before beginning a new session (e.g., establish a place without carpets, domestic animals, or canes) and to wear appropriate, comfortable shoes to protect their safety.

Usability Test

When new tools and technology are developed, usability should be analyzed, as one of the most important aspects of human-technology interaction. Usability can be defined as the degree to which a subject can use a given system to achieve specific goals effectively (the possibility for the users to achieve goals), efficiently (the effort made by the users to reach the goal), and satisfactorily (what users think about the interaction with the system) within a well-defined context of use (Argent et al., 2018). A recent systematic review by Tuena and colleagues (Tuena et al., 2020) outlined that to evaluate the usability of an application for an elderly population it is necessary to:

(1) identify obstacles and facilitators

(2) develop appropriate tasks for the sample

(3) define the usability criteria

(4) test its clinical use.

In the present study, usability has been assessed using The System Usability Scale (SUS) (Brooke, 1996), the Senior technology Acceptance Model (STAM) (Chen and Chan, 2014), the Thinking Aloud protocol (TAP) (Lewis, 1982) and a part of the Independent Television Commission Sense of Presence Inventory (ITC-SOPI) about the cybersickness (Lessiter et al., 2001). The evaluations aimed at collecting information about the usability and final users’ interaction with the technology.

SUS is a “quick and easy to use” questionnaire composed of ten items (Brooke, 1996). Subjects need to express the degree of the agreement on a Likert scale which can range from 0 to 4 for each statement. The final score can range from 0, lack of usability, to 100, optimal usability. The qualitative interpretation of the score was developed by Bangor and colleagues (Bangor et al., 2009). The SUS has some advantages: it is adaptable to evaluate a wide range of technologies; it is fast and easy to use by both users and researchers; it provides scores easy to understand; it has a fairly low cost of administration (Bangor et al., 2009).

The STAM is a 13-item scale that investigates 4 components of the Senior Technologies Acceptance Model: beliefs, perception of control, anxiety related to technologies, and general health conditions (Chen and Chan, 2014).

The TAP (Lewis, 1982) is a technique that is generally administered to assess usability when a new technology is developed. Subjects are required to express their opinion regarding the technology employment and criticism while performing the task. The observer (e.g., the therapist in our study) is asked to take notes of participants’ observations and concerns without attempting to interpret their actions and words. All the verbalizations are transcribed and analyzed to develop the formal usability report.

The Independent Television Commission - Sense of Presence Inventory (ITC-SOPI) (Lessiter et al., 2001) is a 42-items self-report questionnaire that investigates several aspects of the VR experience. Participants are required to rate their degree of agreement-disagreement with a 5-point Likert scale. Specifically, it measures the Sense of Physical Space (SOPS), Engagement, Ecological Validity, and the Negative Effects of the VR experience. We administered only the 6 items that form the Negative Effects sub-scale, e.g., if they are tired or experience dizziness and cybersickness.

Finally, we developed a semi-structured interview to collect information about the use for the tablet (Have you ever used a tablet/iPad or smartphone/iPhone? Yes No; If so, how often do you use it in a week? rarely/sometimes/often) and, eventually, to deepen important aspects that emerged from the TAP.

Participants

Five frail patients were recruited for the usability study at the I.R.C.C.S. Istituto Auxologico Italiano. Exclusion criteria were: (i) visual and auditory impairments that could affect the app’s usability; (ii) upper limb motor impairments that could affect the app’s usability; and (iii) a score lower of 24 on the Mini-Mental State Examination (MMSE) (Spinnler and Tognoni, 1987). All participants signed the informant consent form before the usability session. The study received ethical approval from the Ethical Committee of the Istituto Auxologico Italiano.

All the subjects’ demographic data and MMSE scores are reported in Table 1.

TABLE 1

Table 1. Demographic data.

Protocol

The usability test lasted 40 min and included 5 phases:

1. Preliminary exchange of information with the patient regarding the aim of the study and the possible negative effects of 360° videos.

2. Informative consensus sign

3. Collection of subjects’ data (age, sex, education, MMSE) and their confidence with technology and electronic devices

4. Brain m-App demo session (Day 5)

5. Administration of usability questionnaires and semi-structured interviews.

Before starting the activity, the experimenter trained patients on how to turn on the iPad, interact with the icons and with the touch screen. Then, once ensured they have understood basic iPad operations, the device is turned off.

The experimenter then gave these directions. “Open the iPad app store and search for “Auxologico 360.” Choose “Day 5” and follow the on-screen directions. As required by the TAP, the patient was asked to comment aloud on the activities while performing tasks.

To test usability, we used Day 5 since it includes a variety of exercises that patients could encounter all over the days. The activity starts with: “Welcome to the fifth session of the training path! Follow the instructions to complete the exercises. Click the symbol below when you are ready to start.”

In Exercise 1, patients have to watch a video while paying close attention to the actions taken. Following that, it is necessary to explain aloud, listing any steps that may have preceded the one just demonstrated.

Exercise 2 demands to watch the same video a second time, this time paying close attention to the setting and the objects. The user is now asked to recall whether certain objects (such as a pasta pan) were to the right or left of others while they were in a specific position (e.g., in front of the stove). In exercise 3, the patients are given another video and asked to concentrate on the objects. Then they are asked to recall the relative positions of various items.

In exercises 4 and 6, they must watch another video while paying close attention to the actions taken. “You saw yourself acting to PREPARE THE SUITCASE in the video. While doing so, some unnecessary actions were carried out that had nothing to do with the suitcase’s preparation. Make a list of all the things you’ve done that aren’t necessary.”

Exercises 5 and 7 require moving the iPad to actively explore the house that will be shown, moving between the rooms. To change rooms, they must click on the sign above each door. The exploration of the house will come to an end when you select the “EXIT” icon on the front door. The goal is to choose the map that best describes the house they’ve just explored from those displayed on the screen.

Finally, patients were instructed to choose the Day 6 symbol and then turn off the iPad. The list of tasks that were analyzed during the TAP is presented in the first column of Table 2.

TABLE 2

Table 2. Results of the thinking aloud protocol.

Results

Starting from quantitative data, the mean score of the SUS is 61.0 (SD = 22.0). According to Bangor and colleagues (Bangor et al., 2009) this score places our app in a marginal zone between High and Low acceptability and level of usability that can be defined as “ok,” as shown in Figure 3.

FIGURE 3

Figure 3. Results of the SUS scale.

Analyzing the information about the use of the tablet, we notice that 2 out of 5 subjects have never used this technology and, among those who have used it, one reported using it often while the other rarely. Table 3 shows the results of the questionnaires used.

TABLE 3

Table 3. Results of usability questionnaires.

The results of the Stam Scale results reveal that our sample has a positive attitude toward technology (M = 7.4/10; SD = 2.01), has good control/access to technological devices (M = 7.7/10; SD = 1.1), has a low level of technology-related anxiety (M = 3/10; SD = 1.46), and considers themselves in good health conditions (M = 8.4/10; SD = 0.63).

As shown by the ITC-SOPI sub-scale investigating negative effects, all subjects reported minimal side effects (M = 1.1/5; SD = 0.14).

Qualitative results of the thinking aloud protocol are shown in Table 2 is structured as follows:

1 column: description of the task

2 column: problem encountered by patients

3 column: number of patients that encountered that problem

4 column: some possible solution for that problem

Overall, patients reported more difficulties in the 5th exercise. They had to actively explore the house by moving the iPad around and between rooms until they saw the Exit button. They completed the experiment by choosing the map that best described the house they had just explored from the options displayed on the screen. Two patients either could not understand or forgot the instructions during the activity. Two patients used the “Exit” button instead of exploring the house; three persons had trouble interacting with the exercise progress buttons that allowed them to explore the house. Two participants followed the instructions from the prior activities, while other two had trouble clicking the “Exit” button. A viable answer to all of the aforementioned issues could be training in which patients move the iPad around to explore 360° settings. The exercise will be easier to understand as a result of this instruction, and the progression buttons will be more accessible. The house maps were not realistic, according to one patient, who suggested simplifying the activity by reducing the house’s size and number of rooms. Another patient had difficulty with the activity as well. To address these issues, we will allow patients to pick between two house maps rather than four. Overall, the patients had some issues interacting with the Brain M-app: they were unable to interrupt the exercises and return to the prior activities or instructions. To address this issue, we’ll include a pause button that will allow you to read the instructions again before beginning the exercise.

Clinical Pilot

Sample

We involve nine patients with Mild Neurocognitive Disorder (World Health Organization, 2019) for the clinical pilot study, divided into two groups. Four patients are in the experimental group (EG) (Brain M-App condition) and five in the control group (CG) (Treatment-As-Usual, TAU). The demographic data (age, years of education, and MMSE) are in Table 4.

TABLE 4

Table 4. Demographic data.

The first one (EG) uses Brain M-App at home for 10 sessions three times a week, while the CG performed 10 sessions of classical paper and pencil exercises at home as well. The exercises of the CG are like those of Brain M-app and are also proposed with increasing difficulty. For example, the patient will have to memorize the matrices or images and then recall them, reorder actions or schedule workdays, observe a picture and then remember the position of some objects, describe actions aimed at reaching a specific goal. All these exercises aimed at training executive functions, attention and spatial memory.

Methods

We used two measures of clinical change to assess the effectiveness of the VR-based (Brain M-app) and TAU intervention, namely the reliable change index (RCI) and the clinically significant change (CSC) (Evans et al., 1998). The former measure if change in the test score is due to measuring unreliability, the latter measure effective clinical change in the examined test in the patient. Normative data of the MMSE (Magni et al., 1996), Frontal Assessment Battery (FAB) (Appollonio et al., 2005), Corsi block-tapping test (Spinnler and Tognoni, 1987), Forward Digit Span (FDS; Spinnler and Tognoni, 1998), Phonemic Verbal Fluency (PVF) (Novelli et al., 1986), Semantic Verbal Fluency (SVF) (Novelli et al., 1986), story recall (Spinnler and Tognoni, 1987), and Corsi supra-span (Spinnler and Tognoni, 1987) was used. For the computation of RCI and CSC of the story recall, Corsi block-tapping test and supra-span, and FDS a measure of test-retest was missing in the normative data, and we used a hypothetical Cronbach’s alpha of 0.90. Jacobson-Truax plot was used to depict the RCI-CSC findings (Evans et al., 1998). The plot shows the RCI 95%CI and the CSC cut-off. RCI is divided into unchanged, RCI improvement, and RCI deterioration between pre-test and post-test evaluations. Results are reported in percentage, expressing the percentage of patients improved, deteriorated, and unchanged on the total, and for the CSC the percentage of patients with clinical change on the total of RCI improved.

Results

Table 5 shows the findings regarding the clinical change in the patients for each neuropsychological test. The VR training improved especially (i.e., patients with CSC) the FAB, the story recall, and the Corsi block-tapping test, whereas the TAU improved the Corsi supra-span, the Corsi block-tapping test, and the MMSE. A reliable change without CSC was found for the SVF in the VR group and the FAB for the TAU. Figure 4 shows the FAB results of the VR training compared to the TAU. The pictures highlight that two out of four patients of the experimental group show a clinical improvement beyond the measurement error (the dots in the upper left dial). On the other hand, none of the subjects in the CG showed clinical improvement; the performance of two of them can be classified as “reliable deterioration.”

TABLE 5

Table 5. Results of VR and TAU training.

FIGURE 4

Figure 4. Jacobson-Truax plot shows the RCI 95%CI (blue lines) and the CSC cut-off (green lines). The upper-left rectangle delimited by the green CSC lines shows the area of significant clinical change. The upper-right and lower-left rectangles delimited by the green csc lines show they are of RCI without a CSC.

Discussion

Aging is a process characterized by a loss of physical, sensory, and mental abilities, as well as increased morbidity and multimorbidity, which can lead to disability. Thus, the main goal of successful aging management is to keep older people healthy, active, and independent for as long as possible, addressing functional decline. Using assistive health technology (AHT; i.e., technologies devoted to maintaining or improving functionality, autonomy, and wellbeing) or medical devices (MD; i.e., technologies used for prevention, diagnosis, and treatment) may also benefit older people (Garçon et al., 2016). However, ensuring accessibility and use of these technologies among the elderly is crucial (World Health Organization, 2015; Beard et al., 2016).

In this research, an innovative tablet device for cognitive home-rehabilitation for frail people is presented. The preliminary usability test, which was conducted on five older people, revealed a sufficient level of usability, although the sample size was modest. This approach is one promising option for continuing cognitive rehabilitation that began in the hospital with the therapist and continues at home. The therapist may normally provide cognitive exercises and activities at the end of the inpatient treatment to guarantee that the rehabilitation can continue at home. Unfortunately, without a therapist or help, patients might have difficulty completing the exercises on their own. Our system is intended to serve as a guide for people undergoing cognitive-home rehabilitation. In addition, our app exploits 360° video technology which results in a higher ecological validity, a higher engagement in a simple way. For instance, it is critical to develop an easy-to-use system with a strong theoretical foundation. Although the small number of individuals participating in this study, the homogeneity of the data obtained from the usability interviews allows us to hypothesize that once the difficulties that emerged through usability study are resolved, Brain m-App might achieve a higher level of usability. In addition, other studies have conducted usability tests with small groups of participants (between 4 and 7) (Yip and Man, 2009; Kiselev et al., 2015; Pedroli et al., 2018, 2020; Riaz et al., 2021; Webber et al., 2021). Moreover, according to Virzi (1992), a sample of four or five subjects could detect 80% of the usability issues; more subjects might reveal less and less new information; and the most severe usability issues are likely to have been detected in the first few subjects.

The issues that emerged from the usability study are not related to the application’s structure and the updated version may require some small interventions. For this purpose, a new usability test will be performed after the upgrade of the application with a larger sample.

Regarding the pilot clinical trial, our sample has some critical issues like the limited size and the diversity in MMSE scores. Despite these limitations, it seems that our software may improve target clinical domains such as verbal-auditory memory and executive functioning. In fact, for the FAB and Story Recall, the benefit is greater than traditional training, and for the Corsi block-tapping test, it is comparable. From the clinical point of view, further research is needed to get more conclusive results, but the ones presented here are promising. So, the next steps will regard new clinical testing with a larger group of both frailty and Mild Neurocognitive Disorder patients. We’ll see how effective our software is comparable to traditional paper and pencil tasks. Moreover, we will develop new tasks targeting various cognitive domains like attention, verbal and semantic memory, language and visuo-spatial abilities.

Data Availability Statement

The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.

Ethics Statement

The studies involving human participants were reviewed and approved by the Ethical Committee of Istituto Auxologico Italiano. The patients/participants provided their written informed consent to participate in this study.

Author Contributions

EP, VM, and PC: conceptualization. CT, VM, and PC: methodology. EP, VM, and CS-B: writing—original draft. CT, PC, VM, and EP: data curation and analyses. All authors have read and agreed to the published version of the manuscript.

Funding

Research funded by the Italian Ministry of Health (PE-2013-02355948) and partially supported by the “Ricerca Corrente” (POSTECH 39C801_2018).

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher’s Note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

References

Appollonio, I., Leone, M., Isella, V., Piamarta, F., Consoli, T., Villa, M. L., et al. (2005). The Frontal Assessment Battery (FAB): normative values in an Italian population sample. Neurol. Sci. 26, 108–116. doi: 10.1007/s10072-005-0443-4

PubMed Abstract | CrossRef Full Text | Google Scholar

Argent, R., Daly, A., and Caulfield, B. (2018). Patient involvement with home-based exercise programs: can connected health interventions influence adherence? JMIR MHealth UHealth 6:47. doi: 10.2196/mhealth.8518

PubMed Abstract | CrossRef Full Text | Google Scholar

Bangor, A., Kortum, P., and Miller, J. (2009). Determining what individual SUS scores mean. J. Usability Stud. 4, 114–123.

Google Scholar

Beard, J. R., Officer, A., De Carvalho, I. A., Sadana, R., Pot, A. M., Michel, J. P., et al. (2016). The World report on ageing and health: a policy framework for healthy ageing. Lancet 387, 2145–2154.

Google Scholar

Brigola, A. G., Rossetti, E. S., Dos Santos, B. R., Neri, A. L., Zazzetta, M. S., Inouye, K., et al. (2015). Relationship between cognition and frailty in elderly: a systematic review. Dement. Neuropsychol. 9, 110–119. doi: 10.1590/1980-57642015dn92000005

PubMed Abstract | CrossRef Full Text | Google Scholar

Brooke, J. (1996). SUS -A quick and dirty usability scale Usability and context. Usability Evaluation in Industry 189, 4–7.

Google Scholar

Cameron, I. D., and Kurrle, S. E. (2015). Frailty and Rehabilitation. Interdiscip. Top. Gerontol. Geriatr. 41, 137–150. doi: 10.1159/000381229

PubMed Abstract | CrossRef Full Text | Google Scholar

Chapin, R., and Dobbs-Kepper, D. (2001). Aging in place in assisted living: philosophy versus policy. Gerontologist 41, 43–50. doi: 10.1093/geront/41.1.43

PubMed Abstract | CrossRef Full Text | Google Scholar

Chen, K., and Chan, A. H. S. (2014). Gerontechnology acceptance by elderly Hong Kong Chinese: a senior technology acceptance model (STAM). Ergonomics 57, 635–652. doi: 10.1080/00140139.2014.895855

PubMed Abstract | CrossRef Full Text | Google Scholar

Clegg, A., Young, J., Iliffe, S., Rikkert, M. O., and Rockwood, K. (2013). Frailty in elderly people. Lancet 381, 752–762. doi: 10.1016/S0140-6736(12)62167-9

CrossRef Full Text | Google Scholar

Cutchin, M. P. (2004). Using Deweyan philosophy to rename and reframe adaptation-to-environment. Am. J. Occup. Ther. 58, 303–312. doi: 10.5014/ajot.58.3.303

PubMed Abstract | CrossRef Full Text | Google Scholar

Delrieu, J., Andrieu, S., Pahor, M., Cantet, C., Cesari, M., Ousset, P. J., et al. (2016). Neuropsychological Profile of “Cognitive Frailty” Subjects in MAPT Study. J. Prev. Alzheimer’s Dis. 3, 151–159. doi: 10.14283/jpad.2016.94

PubMed Abstract | CrossRef Full Text | Google Scholar

Eeles, E. M. P., White, S. V., O’mahony, S. M., Bayer, A. J., and Hubbard, R. E. (2012). The impact of frailty and delirium on mortality in older inpatients. Age Ageing 41, 412–416. doi: 10.1093/ageing/afs021

PubMed Abstract | CrossRef Full Text | Google Scholar

EVANS, C., Margison, F., and Barkham, M. (1998). The contribution of reliable and clinically significant change methods to evidence-based mental health. Evid. Based Ment. Health 1, 70–72. doi: 10.1136/ebmh.1.3.70

CrossRef Full Text | Google Scholar

Fairhall, N., Kurrle, S. E., Sherrington, C., Lord, S. R., Lockwood, K., John, B., et al. (2015). Effectiveness of a multifactorial intervention on preventing development of frailty in pre-frail older people: study protocol for a randomised controlled trial. BMJ Open 5:e007091. doi: 10.1136/bmjopen-2014-007091

PubMed Abstract | CrossRef Full Text | Google Scholar

Ferrucci, L., Cavazzini, C., Corsi, A., Bartali, B., Russo, C. R., Lauretani, F., et al. (2002). Biomarkers of frailty in older persons. J. Endocrinol. Invest. 25, 10–15.

PubMed Abstract | Google Scholar

Fried, L. P., Tangen, C. M., Walston, J., Newman, A. B., Hirsch, C., Gottdiener, J., et al. (2001). Frailty in older adults: evidence for a phenotype. J. Gerontol. Ser. A Biol. Sci. Med. Sci. 56, M146–M156. doi: 10.1093/gerona/56.3.m146

PubMed Abstract | CrossRef Full Text | Google Scholar

Garçon, L., Khasnabis, C., Walker, L., Nakatani, Y., Lapitan, J., Borg, J., et al. (2016). Medical and assistive health technology: meeting the needs of aging populations. Gerontologist 56(Suppl. 2) S293–S302.

Google Scholar

Gaggioli, A., Riva, G., Peters, D., and Calvo, R. A. (2017). “Positive Technology, Computing, and Design: Shaping a Future in Which Technology Promotes Psychological Well-Being,” in Emotions and Affect in Human Factors and Human-Computer Interaction, ed. M. Jeon (Cambridge: Academic Press), 477–502.

Google Scholar

Gallucci, A., Trimarchi, P. D., Abbate, C., Tuena, C., Pedroli, E., Lattanzio, F., et al. (2021). ICT technologies as new promising tools for the managing of frailty: a systematic review. Aging Clin. Exp. Res. 33, 1453–1464. doi: 10.1007/s40520-020-01626-9

PubMed Abstract | CrossRef Full Text | Google Scholar

Gill, T. M., Gahbauer, E. A., Allore, H. G., and Han, L. (2006). Transitions between frailty states among community-living older persons. Arch. Intern. Med. 166, 418–423. doi: 10.1001/archinte.166.4.418

PubMed Abstract | CrossRef Full Text | Google Scholar

Hahr, J. Y. (2019). Physiology of aging. Med. Hypotheses. 123, 83–85. doi: 10.1016/j.mehy.2018.12.016

PubMed Abstract | CrossRef Full Text | Google Scholar

Higuera-Trujillo, J. L., López-Tarruella Maldonado, J., and Llinares Millán, C. (2017). Psychological and physiological human responses to simulated and real environments: a comparison between Photographs, 360° Panoramas, and Virtual Reality. Appl. Ergon. 65, 398–409. doi: 10.1016/j.apergo.2017.05.006

PubMed Abstract | CrossRef Full Text | Google Scholar

Kim, K. I., Gollamudi, S. S., and Steinhubl, S. (2017). Digital technology to enable aging in place. Exp. Gerontol. 88, 25–31. doi: 10.1016/j.exger.2016.11.013

PubMed Abstract | CrossRef Full Text | Google Scholar

Kinsella, K., and Phillips, D. R. (2005). Global Aging: the challenge of success. Popul. Bull. 60, 3–40.

Google Scholar

Kirkwood, T. B. L. (2005). Understanding the odd science of aging. Cell 120, 437–447. doi: 10.1016/j.cell.2005.01.027

PubMed Abstract | CrossRef Full Text | Google Scholar

Kiselev, J., Haesner, M., Gövercin, M., and Steinhagen-Thiessen, E. (2015). Implementation of a home-based interactive training system for fall prevention: requirements and challenges. J. Gerontol. Nurs. 41, 14–19. doi: 10.3928/00989134-20141201-01

PubMed Abstract | CrossRef Full Text | Google Scholar

Lessiter, J., Freeman, J., Keogh, E., and Davidoff, J. (2001). A cross-media presence questionnaire: the ITC-sense of presence inventory. Presence 10, 282–297. doi: 10.1162/105474601300343612

CrossRef Full Text | Google Scholar

Lewis, C. (1982). Using the “thinking Aloud” Method in Cognitive Interface Design. In IBM Research Report, RC-9265. Cambridge: IBM T.J. Watson Research Center.

Google Scholar

Lipsitz, L. A. (2002). Dynamics of stability: the physiologic basis of functional health and frailty. J. Gerontol. Ser. A Biol. Sci. Med. Sci. 57, 115–125. doi: 10.1093/gerona/57.3.B115

PubMed Abstract | CrossRef Full Text | Google Scholar

Ma, L., and Chan, P. (2020). Understanding the physiological links between physical frailty and cognitive decline. Aging Dis. 11, 405–418. doi: 10.14336/AD.2019.0521

PubMed Abstract | CrossRef Full Text | Google Scholar

Magni, E., Binetti, G., Bianchetti, A., Rozzini, R., and Trabucchi, M. (1996). Mini-Mental State Examination: a normative study in Italian elderly population. Eur. J. Neurol. 3, 198–202. doi: 10.1111/j.1468-1331.1996.tb00423.x

PubMed Abstract | CrossRef Full Text | Google Scholar

Mancuso, V., Stramba-Badiale, C., Cavedoni, S., Pedroli, E., Cipresso, P., and Riva, G. (2020). Virtual reality meets non-invasive brain stimulation: integrating two methods for cognitive rehabilitation of mild cognitive impairment. Front. Neurol. 11:1117. doi: 10.3389/fneur.2020.566731

PubMed Abstract | CrossRef Full Text | Google Scholar

Matheis, R. J., Schultheis, M. T., Tiersky, L. A., DeLuca, J., Millis, S. R., and Rizzo, A. (2007). Is learning and memory different in a virtual environment? Clin. Neuropsychol. 21, 146–161. doi: 10.1080/13854040601100668

PubMed Abstract | CrossRef Full Text | Google Scholar

Mugueta-Aguinaga, I., and Garcia-Zapirain, B. (2017). Is technology present in frailty? Technology a back-up tool for dealing with frailty in the elderly: a systematic review. Aging Dis. 8:176. doi: 10.14336/AD.2016.0901

PubMed Abstract | CrossRef Full Text | Google Scholar

Novelli, G., Papagno, C., Capitani, E., and Laiacona, M. (1986). Tre test clinici di ricerca e produzione lessicale. Archivio Di Psicologia, Neurologia e Psichiatria 47, 477–506.

Google Scholar

Parsons, T. D. (2015). Virtual reality for enhanced ecological validity and experimental control in the clinical, affective and social neurosciences. Front. Hum. Neurosci. 9:660. doi: 10.3389/fnhum.2015.00660

PubMed Abstract | CrossRef Full Text | Google Scholar

Pedroli, E., Cipresso, P., Serino, S., Toti, M., Goulen, K., Grigioni, M., et al. (2019b). “Beyond cognitive rehabilitation: immersive but noninvasive treatment for elderly,” in Lecture Notes of the Institute for Computer Sciences, Social-Informatics and Telecommunications Engineering, LNICST, eds P. Cipresso, S. Serino, and D. Villani (Berlin: Springer), 263–273.

Google Scholar

Pedroli, E., Cipresso, P., Greci, L., Arlati, S., Boilini, L., Stefanelli, L., et al. (2019a). An immersive motor protocol for frailty rehabilitation. Front. Neurol. 10:1078. doi: 10.3389/fneur.2019.01078

PubMed Abstract | CrossRef Full Text | Google Scholar

Pedroli, E., Cipresso, P., Greci, L., Arlati, S., Mahroo, A., Mancuso, V., et al. (2020). A new application for the motor rehabilitation at home: structure and usability of Bal-App. IEEE Trans. Emerg. Topics Comput. 9, 1–12. doi: 10.1109/TETC.2020.3037962

CrossRef Full Text | Google Scholar

Pedroli, E., Greci, L., Colombo, D., Serino, S., Cipresso, P., Arlati, S., et al. (2018). Characteristics, usability, and users experience of a system combining cognitive and physical therapy in a virtual environment: positive bike. Sensors 18:2343. doi: 10.3390/s18072343

PubMed Abstract | CrossRef Full Text | Google Scholar

Pedroli, E., Serino, S., Stramba-Badiale, M., and Riva, G. (2016). “An innovative virtual reality-based training program for the rehabilitation of cognitive frail patients,” in Pervasive Computing Paradigms for Mental Health, eds N. Oliver, S. Serino, A. Matic, P. Cipresso, N. Filipovic, and L. Gavrilovska (Berlin: Springer), 62–66.

Google Scholar

Riaz, W., Khan, Z. Y., Jawaid, A., and Shahid, S. (2021). Virtual Reality (VR)-Based Environmental Enrichment in Older Adults with Mild Cognitive Impairment (MCI) and Mild Dementia. Brain Sci. 11:1103. doi: 10.3390/brainsci11081103

PubMed Abstract | CrossRef Full Text | Google Scholar

Riva, G. (2008). From virtual to real body: virtual reality as embodied technology. J. Cyber Ther. Rehabilit. 1, 7–22.

Google Scholar

Riva, G., Castelnuovo, G., and Mantovani, F. (2006). Transformation of flow in rehabilitation: the role of advanced communication technologies. Behav. Res. Method. 38, 237–244. doi: 10.3758/BF03192775

PubMed Abstract | CrossRef Full Text | Google Scholar

Riva, G., Mancuso, V., Cavedoni, S., and Stramba-Badiale, C. (2020). Virtual reality in neurorehabilitation: a review of its effects on multiple cognitive domains. Expert Rev. Med. Devices 17, 1035–1061. doi: 10.1080/17434440.2020.1825939

PubMed Abstract | CrossRef Full Text | Google Scholar

Riva, G., Wiederhold, B. K., and Mantovani, F. (2019). Neuroscience of Virtual Reality: from Virtual Exposure to Embodied Medicine. Cyberpsychol. Behav. Soc. Netw. 22, 82–96. doi: 10.1089/cyber.2017.29099.gri

PubMed Abstract | CrossRef Full Text | Google Scholar

Robertson, C. E., Hermann, K. L., Mynick, A., Kravitz, D. J., and Kanwisher, N. (2016). Neural Representations Integrate the Current Field of View with the Remembered 360° Panorama in Scene-Selective Cortex. Curr. Biol. 26, 2463–2468. doi: 10.1016/j.cub.2016.07.002

PubMed Abstract | CrossRef Full Text | Google Scholar

Robinson, J., Dixon, J., Macsween, A., van Schaik, P., and Martin, D. (2015). The effects of exergaming on balance, gait, technology acceptance and flow experience in people with multiple sclerosis: a randomized controlled trial. BMC Sports Sci. Med. Rehabilit. 7:8. doi: 10.1186/s13102-015-0001-1

PubMed Abstract | CrossRef Full Text | Google Scholar

Serino, S., Pedroli, E., Tuena, C., De Leo, G., Stramba-Badiale, M., Goulene, K., et al. (2017b). A Novel Virtual Reality-Based Training Protocol for the Enhancement of the “Mental Frame Syncing” in Individuals with Alzheimer’s Disease: a Development-of-Concept Trial. Front. Aging Neurosci. 9:240. doi: 10.3389/fnagi.2017.00240

PubMed Abstract | CrossRef Full Text | Google Scholar

Serino, S., Barello, S., Miraglia, F., Triberti, S., and Repetto, C. (2017a). Virtual reality as a potential tool to face frailty challenges. Front. Psychol. 8:1541. doi: 10.3389/fpsyg.2017.01541

PubMed Abstract | CrossRef Full Text | Google Scholar

Spinnler, H., and Tognoni, G. (1987). Italian Group on the Neuropsychological Study of Ageing: italian standardization and classification of neuropsychological tests. Ital. J. Neurol. Sci. 6, 1–120. doi: 10.2147/NDT.S73179

PubMed Abstract | CrossRef Full Text | Google Scholar

Stones, D., and Gullifer, J. (2016). “At home it’s just so much easier to be yourself”: older adults’ perceptions of ageing in place. Ageing Soc. 36, 449–481. doi: 10.1017/S0144686X14001214

CrossRef Full Text | Google Scholar

Tieri, G., Morone, G., Paolucci, S., and Iosa, M. (2018). Virtual reality in cognitive and motor rehabilitation: facts, fiction, and fallacies. Expert Rev. Med. Devices 15, 107–117. doi: 10.1080/17434440.2018.1425613

PubMed Abstract | CrossRef Full Text | Google Scholar

Tuena, C., Pedroli, E., Trimarchi, P. D., Gallucci, A., Chiappini, M., and Goulene, K. (2020). Usability Issues of Clinical and Research Applications of Virtual Reality in Older People: a Systematic Review. Front. Hum. Neurosci. 14:93. doi: 10.3389/fnhum.2020.00093

PubMed Abstract | CrossRef Full Text | Google Scholar

Virzi, R. A. (1992). Refining the test phase of usability evaluation: How many subjects is enough? Hum. Fact. 34, 457–468.

Google Scholar

Walston, J., Hadley, E. C., Ferrucci, L., Guralnik, J. M., Newman, A. B., and Studenski, S. A. (2006). Research agenda for frailty in older adults: toward a better understanding of physiology and etiology: summary from the American Geriatrics Society/National Institute on Aging research conference on frailty in older adults. J. Am. Geriat. Soc. 54, 991–1001. doi: 10.1111/j.1532-5415.2006.00745.x

PubMed Abstract | CrossRef Full Text | Google Scholar

Webber, S., Baker, S., and Waycott, J. (2021). Virtual visits: reminiscence in residential aged care with digital mapping technologies. Australas. Ageing 40, 293–300. doi: 10.1111/ajag.12902

PubMed Abstract | CrossRef Full Text | Google Scholar

World Health Organization (2015). World Health Statistics 2015. Geneva: World Health Organization.

Google Scholar

World Health Organization (2019). Icd-11: International Classification Of Diseases (11th Revision). Geneva: World Health Organization.

Google Scholar

Xue, Q. L. (2011). The Frailty Syndrome: definition and Natural History. Clin. Geriatr. Med. 27, 1–15. doi: 10.1016/j.cger.2010.08.009

PubMed Abstract | CrossRef Full Text | Google Scholar

Yip, B. C., and Man, D. W. (2009). Virtual reality (VR)-based community living skills training for people with acquired brain injury: a pilot study. Brain Injury 23, 1017–1026. doi: 10.3109/02699050903379412

PubMed Abstract | CrossRef Full Text | Google Scholar