Effects of Combined Physical and Cognitive Virtual Reality-Based Training on Cognitive Impairment and Oxidative Stress in MCI Patients: A Pilot Study

Introduction

In recent years, a dramatic increase of the incidence of age-related disorders due to the rise in average life-span has been recorded. One of the most common age-related pathologies is Mild Cognitive Impairment (MCI), which often represents a transition from healthy aging to Alzheimer’s disease (AD), a devastating neurodegenerative disorder so far without a cure.

MCI patients report a cognitive decline characterized by symptoms such as impaired memory, attention, orientation and executive functions, which however do not interfere remarkably with daily life activities.

Recently developed brain plasticity theories and findings about the cellular synapses reconstruction ability of nervous system due to interaction with enriched environments have spurred new researches. As a consequence, cognitive rehabilitation interventions adopting non-invasive non-pharmacological treatments have gained growing attention (Ferrucci et al., 2008; Cotelli et al., 2012; Kim et al., 2016; Marceglia et al., 2016; Chiu et al., 2017). The therapeutic goal of cognitive training is oriented to stimulate, restore or re-train the cognitive processes implied in the initiation, planning and execution of activities of daily living (ADLs), in order to reduce the side effects, maximize the patient’s ecological autonomy and quality of life. Cognitive training in MCI may stimulate pre-existing neural circuits or prompt the recruitment of alternate neural pathways prompting neuroplastic re-organization. According to a recent meta-analysis (Sherman et al., 2017) cognitive interventions adopting memory-based strategies, targeting multiple cognitive domains or using a multi-componential approach (i.e., targeting lifestyles) proved effective to improve performances on cognitive outcomes for people with MCI.

Besides, Alzheimer’s Association, based on a perspective population-based study, has established that one of the strategies to reduce cognitive decline risk and dementia development is regular physical exercise (Baumgart et al., 2015). Improvements in tests of executive function (Symbol Digit Modalities, Verbal Fluency and Stroop) were shown in a large randomized trial on aerobic exercise in MCI patients (Baker et al., 2010).

Furthermore, according to a recent systematic review (Cammisuli et al., 2017) there are evidences that aerobic exercise in MCI patients improves global cognition, logical memory, inhibitory control and divided attention. Decrease in oxidative stress (OxS) and increase in antioxidant and anti-inflammatory capacity are some of the molecular mechanisms underlining the beneficial effect of regular physical activity. Indeed, reactive oxygen species (ROS) play a dual role by means of detrimental effects linked to increased uncontrolled production, evident in the development of neurodegenerative diseases and particularly in AD (Tönnies and Trushina, 2017) vs. essential neuroprotective cellular mechanisms based on the concept of mitohormesis. The latter suggests that an exposure to intermittent sub-lethal doses of ROS generated by exercise could lead to a mitochondrial adaptation inducing mitochondrial biogenesis and an antioxidant response. The development of strategies to mitigate OxS in neurodegenerative diseases should carefully consider these aspects.

One major issue regarding usual neuro-rehabilitation programs is represented by the transferability in real life of the benefits obtained in the conventional therapeutic setting. In recent years, ICT technologies, such as virtual reality (VR), has become an increasingly used approach in neuroscience to provide an ecologically valid treatment (Coyle et al., 2015). The major features of VR are aimed at providing easy and intuitive interaction and multidimensional sensory feedback, offering the patient an opportunity to practice activities of daily living that cannot occur in conventional rehabilitation programs, through motivational activities facilitating treatment compliance and customizable on the patient’s characteristics (Rizzo and Kim, 2005; García-Betances et al., 2015b). Therefore, research on VR applied to neurorehabilitation has recently flourished, with increasing evidence of efficacy (Rizzo and Kim, 2005; Serino et al., 2017; Anderson-Hanley et al., 2018; Tieri et al., 2018).

Currently, there is limited but promising evidence on the application of VR technologies in subjects with MCI or mild dementia (Coyle et al., 2015).

In neuroscience, the computerized approach provided by VR is used in order to allow interventions in a controlled environment, able to monitor movement, cognitive and other variables (García-Betances et al., 2015a). Recently, the attention of clinicians and researchers in the field of MCI has been attracted by motor and cognitive training based on VR, which has emerged as an encouraging tool in many therapeutic and rehabilitative domains (Rizzo and Kim, 2005; García-Betances et al., 2015b). In a randomized controlled trial, Talassi et al. (2007) compared the effectiveness of a computerized training specifically developed to address different cognitive domains, with respect to physical activity in a sample of patients with MCI and mild dementia. Authors reported that significant improvements of the patients’ cognitive and affective status were obtained only in the group undertaking the computerized program, indicating that non-punctual stimulation is not able to elicit any effect.

This pilot study aims to evaluate the impact of an innovative combined physical activity and cognitive training based on VR, in MCI patients. The two main fields of investigation of this multi-domain assessment are: (1) the effects of this intervention on cognitive impairment and on OxS, assessed as indicator of the disease progression and (2) the evaluation of the user acceptability, with the final goal of verifying the potentialities of this kind of approach in counteracting MCI evolution.

Materials and Methods

Experimental Session

A controlled pilot study was carried out in 10 patients—4 males (M) and 6 females (F)—aged ≥65 years, with: (a) one or more test scores indicating a compromise of visuospatial abilities, (b) one or more test scores indicating mild-moderate cognitive impairment according to Mini Mental State Examination (MMSE) criteria (Folstein et al., 1975; Magni et al., 1996a,b). Exclusion criteria adopted were: cognitive and/or functional impairment affecting the ability to participate in the study; cardiovascular pathologies advising the undergoing a minimal physical training; acute pain of lower back or extremities; peripheral neuropathy; rheumatic and orthopedic diseases; inability to provide informed consent. Moreover, each subject who could potentially be enrolled had first to undergo an assessment of risk factors (cardiovascular pathologies and other co-morbidities) and an effort electrocardiogram (ECG) to exclude pathologies that may cause harm to the patients during the physical exercise. All decisions about eligibility were made before the block-randomization, and performed on the basis of cognitive decline and assessed through psychometric tests. Participants were randomized to an intervention (EXP) group (n = 5, 2M/3F) which underwent the intervention or to a control (CTR) group (n = 5, 2M/3F), who received no treatment. Three MCI patients and two patients with mild dementia (i.e., an impairment of cognition that significantly affected instrumental skills of daily life) were included in each experimental group. Individuals participating in the study provided informed written consent. This study was carried out in accordance with GCP recommendations of the Ethical Committee of IRCCS Santa Lucia Foundation, which approved the protocol (Protocol Number: CE/PROG.524, 28/09/2015). All subjects gave written informed consent in accordance with the Declaration of Helsinki.

In order to verify the effects of the training, evaluations were performed before (PRE) and at the end (POST) of the experimental 6 weeks (see Figure 1). At these time points, anthropometric measures, cognitive profile and collection of blood and urine samples were carried out in both the experimental and the control group. Moreover, at the same time points, questionnaires to evaluate the treatment acceptability were administered to the subjects in the intervention group.

FIGURE 1

Figure 1. Experimental protocol of the virtual reality (VR)-based training program. The training consisted of 18 sessions for six consecutive weeks, with three sessions per week. All the assessments were carried out at baseline (PRE) and after (POST) 6 weeks for both groups (EXP—see upper part of the figure and CTR—lower part of the figure).

Equipment

The implemented system was designed to allow MCI patients to both perform physical exercise and train their cognitive abilities while being supported by VR means. The three virtual environments (VEs) developed enable to emulate the performance of three activities of daily living (see Figure 2). This choice has the aim of facilitating the comprehension of the tasks for patients and of enhancing the transfer of the acquired capabilities into real life. The three VEs thus represented the following scenarios: (1) riding a bike in a park, (2) crossing roads—avoiding cars and (3) grocery shopping in a supermarket. The first scenario is dedicated to the accomplishment of the physical exercise, whereas scenarios (2) and (3) are designed to provide the cognitive training. They all were developed using Unity3D. The hardware devices composing the training system are a cycle-ergometer (Cosmed EuroBike 320), a smart garment (Wearable Wellness System, Smartex), for a real time measure of heart rate (HR), a finger touch projector (EB-1430WI, Epson) and a PlayStation controller anchored on the cycle-ergometer handlebars (Sony).

FIGURE 2

Figure 2. Sketch of the Physical and Cognitive Training protocol. (A) The cycle-ergometer. (B) The back view of the subject while conducting the test in virtual environments (VEs, 1,3). (C,D) VEs (1): “riding a bike in a park”. (E,F) VEs (2): “crossing roads—avoiding cars”. (G,H) VEs (3): “making the grocery shopping in a supermarket”.

VE developed in the first scenario (1) represents a trail in a park in which the user can navigate by pedalling on the cycle-ergometer. The path to follow is predefined to avoid the user from getting lost or reaching the edges of the designed scene; bends and slopes are smooth and have a large curvature to reduce the occurrence of sickness as much as possible. To increase the user’s presence in the VE, the realism of the scene requires that the visual flow is synchronized with the cycling velocity in real-time, when the wind is blowing the tree leaves and grass move, wild animals appear on the trail sides or in the sky. Realistic 3D sounds reproducing birds’ chirp and the wind have been are added to the scene too. Indicators showing the bike velocity and revolutions-per-minute, the user’s HR (measured thanks to the smart garment) and the elapsed time are shown as on-screen text to provide the patients with general information about the exercise and to help them to maintain the right cycling speed (see Training Protocol).

The second virtual environment (2) represents an urban scenario that the patient is supposed to reach after the physical activity performed in the park. In this scenario, the user—on the cycle-ergometer—has to cross several traffic-congested crosses by stopping on the edge of the sidewalk and checking if the route is safe from moving cars. Cars are generated randomly and move at different velocities. 3D sounds simulating an accident are reproduced if a car hits the user while he/she is crossing. After the accident, the user is brought in a safe-position and can restart to pedal after a few seconds, when the cars disappear from the cross. To brake and to turn the point of view, the patient has to use the PlayStation controller: respectively, the X button and the right joystick; to move toward the other side, he/she has to restart cycling.

The third virtual environment (3) represents a supermarket. Before entering, the user is shown a list of five grocery items that he/she has to buy by touching the projected screen to make his/her selection. The purchase of each item is performed by completing two tasks: (3.1) selecting the right aisle and then (3.2) picking the right product from the shelves. Five different levels of difficulty were implemented for both tasks. The increase in difficulty is obtained mainly by the increase of distractors and of their similarity and by their position on the shelf. For the aisle task, a further stimulus to promote language and attentional training is obtained by the introduction of a word that is orthographically or semantically similar to the target object name (from Level #3). For the shelf task, higher levels are characterized by the presence of different formats (Level #4) and discounted versions of the same product (Level #5), so that the attentional and visuospatial demands are increased.

The system intervenes if the user commits an error or does not interact with the VE for more than 20 s (latency) and gives him/her a hint to help in proceeding to the next task.

Training Protocol

The intervention group performed the physical and cognitive training for 6 weeks, 3 sessions/week (18 sessions total). The structure of VR-based training session, carried out in a quiet room, was the same for each participant and lasted approximately 40–45 min long: 15 (for the first 3 weeks) or 20 min of cycling in the virtual park, about 5 min to perform five crossroads passing and 20 min of shopping at the supermarket. Before the beginning of the training program, the participants were assisted in a preliminary session so that they could familiarize with VR technologies and learn how to interact within the three virtual environments.

Cycling exercise intensity was set as to correspond to about 65%–70% of individual maximal HR, calculated on the basis of the age-predicted value. During the training session, the patients were instructed to keep the cycling velocity between 60 and 70 revolutions per minute; the cycle-ergometer workload was continuously adjusted (thanks to a digital controller based on a proportional-integral feedback mechanism) with the aim of maintaining the target HR at a constant frequency.

Dealing with the supermarket scenario (3), after having completed three subsequent shopping sessions making one error at the most (or waiting once for more than 20 s), the subject could move to the next level of difficulty. When patients completed Level #5 for three times without committing errors or latencies, the training was restarted from the first level in which until he/she failed for the first time with the list displayed on the screen. During this second cycle, the shopping list was kept hidden with the aim of training patients’memory too.

Assessment of Cognitive Response

An in-depth cognitive assessment was performed by a trained neuropsychologist at PRE and POST intervention. Each subject was assessed by the same evaluator at PRE and POST, in order to avoid inter-rater variability effects.

The entire neuropsychological battery was composed by an initial screening test for cognitive abilities (MMSE, Magni et al., 1996a,b) and a subsequent extensive battery of specific scales for each cognitive domain.

Episodic verbal memory was evaluated by Immediate Recall and Delayed of Rey Auditory (RAVLT_I and RAVLT_D, Carlesimo et al., 1996): in the RAVLT_I, a list of 15 words is presented to the subject, which he/she has to memorize and repeat. The task is repeated five times and the number of words correctly recalled is recorded. The RAVLT_D requires the recall, after 15 min, of the same words.

Visuo-spatial functions were assessed with the Rey-Osterrieth Complex Figure Test (ROCFT; Caffarra et al., 2002), the Attentional Matrices Test (AM; Spinnler and Tognoni, 1987), and the Trail Making Test A (TMT-A, Giovagnoli et al., 1996). The ROCFT is commonly used for the evaluation of constructive apraxia, and requires to copy a geometric drawing composed of 18 sub-elements: the correct execution and collocation of the sub-elements are both criteria for scoring. AM inquires selective visuo-spatial attention and requires to delete, as quickly as possible, all the numbers identical to the targets among distractors. In the TMT-A, the subject must perform a visuo-spatial search to connect in ascending order, with a stroke of pen, a series of 25 numbers presented in a random distribution on a sheet.

The executive functions were investigated with the Frontal Assessment Battery (FAB, Appollonio et al., 2005) which evaluates categorization, planning, inhibition, sensitivity to interference and environmental autonomy and with the Trail Making Test B (TMT-B, Giovagnoli et al., 1996) which requires to combine alternatively numerical and alphabetical stimuli, in order to explore visual-motor coordination, set-shifting and mental flexibility.

Lexical finding skills were assessed with the Verbal Fluency test (VF, Novelli et al., 1986), in which the subject must name the greatest possible number of words belonging to a given semantic category in a minute.

Finally, the subject’s independency in daily life activities was inquired with the caregiver, using the Functional Activity Questionnaire (FAQ).

Assessment of Oxidative Stress

ROS Determination

ROS production rate was determined adopting a recently developed Electron Paramagnetic Resonance (EPR) mini-invasive method (Mrakic-Sposta et al., 2012, 2014, 2015, 2017), using 50 μL of capillary blood taken from the fingertip and immediately treated with CMH (1-hydroxy-3-methoxycarbonyl-2,2,5,5-tetramethylpyrrolidine) probe solution (1:1). For data acquisition 50 μL of the obtained solution was put in a glass EPR capillary tube (Noxygen Science Transfer & Diagnostics, Germany) placed inside the cavity of a X-band EPR instrument (E-Scan-Bruker BioSpin, Billerica, MA, USA). Acquisition parameters were determined previously (Mrakic-Sposta et al., 2012). A Temperature and Gas Controller “Bio III” unit, interfaced to the spectrometer, was used in order to stabilize sample temperature at 37°C. Absolute ROS production rate (μmol.min⁻¹) was obtained converting relative quantitative determination, allowed by EPR measurements, by adopting the stable radical CP• (3-Carboxy2,2,5,5-tetramethyl-1-pyrrolidinyloxy) as external reference. The software supplied by Bruker (Win EPR System, V. 2.11) was adopted for acquisition parameters and spectra handling.

Antioxidant Capacity

A capillary blood sample (10 μL) was used in order to assess Blood Reducing Capacity by mean of a commercial EDEL potentiostat electrochemical analyzer (Edel Therapeutics, Switzerland) equipped with a redox sensor in a three-electrode arrangement able to respond to all water-soluble compounds in biological fluids, which can be oxidized within a defined potential range (Liu et al., 2005, 2006). The blood sample was loaded onto a chip and the result was then pseudo-titrated to account for the most biologically relevant antioxidants (Tacchini et al., 2013). Data were expressed in nW.

Enzymatic Assays

Urine samples were collected in a sterile container and stored in multiple aliquots at −80° C until assessment performed within 2 weeks from collection.

8-hydroxy-2-deoxy Guanosine (8-OH-dG), as a marker of oxidative DNA damage, and 8-isoprostane (8-iso), an established marker of lipid peroxidation, were assessed by commercially enzyme immunoassay kit (Cayman Chemical, Ann Arbor, MI, USA). The procedures described in detail by the manufacturer were followed.

As the collection of the 24 h urine was not possible, urinary parameters were standardized based on the amount of the excreted creatinine, in order to avoid the well-known considerable changes occurring over time. Indeed, the excretion rate of creatinine keeps relatively constant in the absence of renal disease. Creatinine concentration was assessed by commercial enzymatic assay kit (Cayman Chemical, Ann Arbor, MI, USA). The procedures described in detail by the manufacturer were performed.

Assessment of the Treatment Acceptability

Acceptability is defined as the degree to which non-professional stakeholders found an intervention to be fair, reasonable, intrusive and consistent with treatment expectations (Wolf, 1978; Kazdin, 1980). It refers to the overall evaluation of the applied procedures and constitutes an important dimension of the treatment evaluation, besides treatment efficacy and effectiveness (Kazdin, 1980). In this study, the assessment of the patients’ opinions toward the intervention was made through a semi-structured interview administered at the beginning of the training program and at the end of week 6 by a psychologist and a biomedical engineer. In particular, closed questions were used to enable the collection of precise information about a specific topic (i.e., “was the smart garment comfortable?”), whereas the open ones had the goal of better exploring patients’ subjective perceptions and feelings. Indeed, open questions constituted as a “topic guide” to ensure the completeness and comparison between different responses, but a high level of flexibility was kept to indulge each participant’s attitude. Before starting the questionnaire administration, the purpose of the interview and the assurance that all opinions were valuable and confidential were disclosed to the patients to encourage honest feedback.

Answers were transcribed by one of the administrators during the interview. Descriptive statistics (median and interquartile ranges) were used to provide a summary of the data gathered through closed questions. Thematic analysis was subsequently performed to further investigate patients’ subjective perceptions, gathered through open questions and free comments.

Statistical Analysis

For the purpose of comparability between different neuropsychological tools, the scores of all the administered tests were standardized according to their normative mean scores and standard deviations (Novelli et al., 1986; Spinnler and Tognoni, 1987; Carlesimo et al., 1995; Giovagnoli et al., 1996; Magni et al., 1996a,b; Caffarra et al., 2002; Appollonio et al., 2005). Only three participants, in PRE, were able to complete the TMT-B, so, this scale was excluded from subsequent comparisons.

GraphPad Prism package (GraphPad Prism 7, GraphPad Software Inc., San Diego, CA, USA) was used for statistical analysis. All statistical analyses were performed using non-parametric tests, due to the small sample size. The effect of the combined physical and cognitive VR-based training on the neuropsychological assessments and the OxS levels between the CTR and the EXP groups were compared using Mann Whitney U-test for independent samples. In detail, such test was used to compare the scores at PRE and POST conditions for OxS, and the change in the scale scores from PRE to POST conditions for psychometric tests, between EXP and CTR groups.

Non-parametric Wilcoxon matched pair signed rank test was adopted to compare the change in the scale scores from PRE to POST conditions within the EXP and CTR groups. For cognitive and functional scores ITT-LOCF analyses are presented. Data are presented as mean ± SD and a p < 0.05 was considered statistically significant.

Results

The socio-demographic and clinical characteristics of the sample are substantially comparable to those used to assess the effectiveness of interventions aimed at slowing down cognitive decline (mean age = 73.3 ± 5.6 years, mean schooling = 7.6 ± 4.4 years, mean MMSE = 23.0 ± 3.4).

A drop out of one patient both in EXP and CTR group was recorded for reasons independent from the intervention. Adherence, for the remaining patients in the EXP group, was judged good (more than 75%, reaching 89% in two cases) for three of them, and partial (more than 50% and less than 75%) for one patient (67%).

The age, the anthropometric characteristics and the physiological parameters (i.e., peripheral oxygen saturation (SpO₂), HR) of the subjects, separated in the examined groups, are reported in Table 1. No significant differences between the two groups examined at baseline were found.

TABLE 1

Table 1. Baseline characteristics.

Assessment of Cognitive Response

The sample was composed of subjects with a mild deterioration of global cognition that is classifiable as mild (mean MMSE score = 23.0 ± 3.4).

The subjects globally presented a slight impairment in all the investigated cognitive domains (Table 2) and a moderate impairment in the visuo-constructive and executive functions (ROCFT standardized scores were −2.15 ± 2.85 in EXP and −2.71 ± 2.45 in CTR; FAB standardized scores were −1.66 ± 1.49 in EXP and −2.29 ± 2.59 in CTR). A mild functional impairment was observed in both groups (FAQ scores 8.80 ± 7.19 for EXP and 10.60 ± 8.08 for CTR). No statistically significant differences were observed between the scores obtained by the EXP and CTR group in any of the scales administered at PRE.

TABLE 2

Table 2. Pre-post assessment.

At POST, EXP showed a slight improvement in MMSE, ROCFT, FAB and AM, while CTR worsened. Both groups showed improvement over time in memory tests and word finding for EXP the improvement was higher, but not statistically different from that of CTR. In the TMT-A, EXP and CTR resulted substantially stable in time (Table 2).

Both groups showed a deterioration of ADLs between PRE and POST. For the EXP this deterioration was more pronounced (1.80 ± 4.60 at the FAQ) than that of the CTR (1.00 ± 1.73). However, this variation has been heavily influenced by the performance of a single subject belonging to the EXP, who lost 10 FAQ-points due to a marked deterioration of global health. The same subject was re-evaluated within the Cognitive Disorders Center 3 months after POST and showed an improvement in the FAQ scores of 6 points, compared to the PRE. Excluding that subject as an outlier, EXP obtained a slight improvement of 0.25 ± 0.5 in FAQ scores.

None of the comparisons within and between groups reached statistical significance at the Wilcoxon Matched Pair Test and at the Mann-Whitney U-Test, reasonably due to small sample size, which amplifies the effect of the slight heterogeneity in scores between subjects, even if a weak effect of the treatment on global cognition, on visuo-constructive abilities, on visuo-spatial attention and, in particular, on executive functions was observed.

Assessment of Oxidative Stress

Comparing every baseline value of the examined biological parameters, no significant difference between EXP and CTR group was recorded.

As previously reported (Marzorati et al., 2016), an increase (+8%) in ROS production was recorded at POST in the CTR group; on the contrary in the EXP group a decrease (−4%) was observed. Comparing the two experimental groups at POST, ROS production rate resulted statistically (p < 0.05) lower in EXP vs. CTR (2.03 ± 0.23 vs. 2.58 ± 0.33 μmol · min⁻¹, respectively Figure 3A).

A similar trend was observed in 8-iso concentration: compared to CTR, in the EXP group after training intervention 8-iso resulted statistically (p < 0.05) lower (495.20 ± 44.56 vs. 379.30 ± 54.82 pg·mg⁻¹ creatinine, respectively; Figure 3C).

FIGURE 3

Figure 3. The influence of physical and cognitive training on oxidative stress (OxS) is shown by the histogram plots of: (A) reactive oxygen species (ROS) production rate (μmol · min⁻¹), (B) total antioxidant capacity (TAC; nW), (C) lipids peroxidation (8-iso; pg.mg⁻¹ creatinine) and (D) DNA damage (8-OH-dG; ng.mg⁻¹ creatinine) obtained from capillary blood and urine samples in the control (CTR) and experimental (EXP) groups at PRE and POST (after 6 weeks). Results are expressed as mean ± SD. Statistically significant differences symbols: *p < 0.05, ^#p < 0.01, and ^§p < 0.001.

8-OH-dG concentration significantly increased (p < 0.01) respect to basal value in the CTR group (5.80 ± 1.40 vs. 10.32 ± 1.28 ng·mg⁻¹ creatinine) and was significantly (p < 0.001) lower in the EXP group with respect to the CTR after training intervention (5.14 ± 1.05 vs. 10.32 ± 1.28 ng·mg⁻¹ creatinine, respectively; Figure 3D).

On the contrary after 6 weeks of training, no differences were found comparing the relative post values in the CTR and EXP groups in the TAC level (CTR: 149.75 ± 30.44 vs. post: 147.25 ± 22.20 nW; EXP: 152.25 ± 25.55 vs. post: 163.50 ± 42.35 nW; Figure 3B).

Assessment of Treatment Acceptability

Results of the closed questions are presented in Table 3 according to the Likert scale proposed, with range from 0 (not at all) to 4 (very much).

TABLE 3

Table 3. Results (median and iqr) of the questionnaire administered after the sixth week of training.

As highlighted in the table, patients expressed in general a neutral position towards the training and a positive confirmation of their expectations consisting, according to open questions, in an improvement or, at least, to maintain their current status (2,5). These data showed that in general, the intervention was judged acceptable and enjoyable by all the subjects. The interaction with the technology was easy and intuitive for all of them and only few side effects were reported. The setup was defined comfortable and changing clothes to wear the smart garment was not an issue. In particular, considering the scenarios supporting the physical training (park and road-crossing), participants scored positively with the joystick use and did not complain about the difficulty of the physical exercise, also in terms of maintaining the speed required (0). Also in the supermarket scenario for the cognitive training, the different visual and auditory supports (shopping list, guiding voice and hints) were judged clear and useful (3).

Difficulties related to the cognitive tasks were mainly related to graphical issues or products’ design, which did not match the packages the participants used to buy. Moreover, answers from participants reported that the different scenarios did not elicit negative feelings as anxiety and boredom (0) except for a slight sense of sickness in the park and road crossing scenarios (0.5). Some interesting considerations on acceptability arose also from the open questions analysis, confirming and deepening specific aspects highlighted in the Table 3. First, all participants stated that they would continue with the training, if possible, and most of them also at their own home with a proper setup, thus confirming a positive acceptance level. Concerning expectations, all of them reported to have received benefits from the intervention, both in terms of motivation and perceived improvements. It is important to underline that, according to the self-perceived improvements, subjects of EXP group reported better performance in real life regardless of real improvements from results of cognitive tests, probably due to a related reduced level of anxiety.

Discussion

Dementia pathogenesis is a multifactorial disorder where mechanisms involved are not fully understood, but an important role appears to be played by the interaction of genetic and environmental factors.

The disease prevention appears to be possible, as many factors involved in the onset and progression of dementia are modifiable or susceptible to management. Therefore, encouraging the implementation of preventive measures throughout life may be most a effective strategy. Indeed, in Europe, three major multi-center trials for the prevention of cognitive decline and dementia have been launched, and they all foresee multicomponent interventions (Vellas et al., 2014; Ngandu et al., 2015; Moll van Charante et al., 2016).

The evidence suggests that a multi-dimensional approach aimed at increasing physical activity levels and stimulating cognitive functions may have an effect on slowing cognitive decline in a population of elderly people at risk of dementia.

The novelty of this VR-based system supporting physical and cognitive training enabled to conduct a preliminary evaluation of the efficacy and of the acceptability of an innovative non-pharmacological system able to mitigate some of the dementia effects, by means of exploiting VR technologies for the training of both functions in elderly with MCI.

The combination of training and VR technologies, adopted in the presented protocol, was revealed able to achieve the expected goals, promoting a tendency towards improvement in people affected by cognitive impairments. The application allowed to train important cognitive functions such as executive function, navigation, planning and memory and support procedures for reducing behavioral and psychological symptoms of MCI and early-stage AD patients. Obtained results are in agreement with recent reviews (Coyle et al., 2015; Hill et al., 2016) on computerized and VR-based training of cognitive abilities in MCI population, which showed that attention, executive functions and memory are the domains that can be better addressed with this type of training.

Even if this study has average sample and time of intervention smaller than that of others (Etnier et al., 2006) evaluating the effects of physical exercise on cognition, the presented results suggest that the adopted training protocol was actually able to affect MMSE tasks and to increase the global cognition levels of MCI. Anyways, according to the above-mentioned review, a longer intervention would be more suitable to improve cognition in older subjects incrementing physical activity level.

Numerous evidences support the role of OxS in the development of AD. Indeed, lipid and DNA damage caused by OxS is closely associated with the development of AD (Nunomura et al., 2006) and very strong correlations among lipid peroxidation, antioxidant enzymes and senile plaques in AD brains are reported (Feng and Wang, 2012).

OxS is characterized by an imbalance between ROS production and antioxidant defense system which are responsible for the removal of ROS (Harman, 1981), both systems are considered to have relevant roles in neurodegeneration and cognitive decline, processes age-related. A decrease in plasma antioxidant defense mechanism was reported that it was associated with memory impairments related to aging (Haider et al., 2014). Thus, the development of strategies to mitigate OxS in neurodegenerative diseases is relevant. The reduction of an uncontrolled ROS production mitigating the deleterious effects should be positively considered.

Although not conclusive, the recorded effects in the present study (i.e., reduction of ROS production and oxidative damage of lipids and DNA), support the evidence that the adoption of physical and cognitive training decreases OxS levels, and speculatively, this would be associated with the slight improvement in some cognitive functions.

Concerning user acceptability, qualitative data retrieved from the questionnaires and the subjects’ comments revealed high levels of engagement and motivation, enabled mainly by the use of the VR technologies that they defined as a new way to approach the cognitive training. Feedbacks on the VEs design revealed appreciation, leading to the conclusion that the training was well accepted by all the patients, who—in the majority of the cases—would continue with the program also at home. Having reached high levels of acceptability of the proposed technology-based treatment is indeed a promising result. In fact, acceptability represents a key issue for promoting the successful employment of innovative technologies in therapy, especially when dealing with older population (Dillon and Morris, 1996).

Finally, it is important to highlight that the subjects enrolled in the experimental group reported to “feel better” and to have reduced their levels of anxiety during the accomplishment of common ADLs. This feeling, though self-perceived and not supported by psychometric results, probably due to the small sample, enabled a real improvement of their quality of life (QoL). Subjective perceptions have been recognized as one of the outcomes of the rehabilitation path, and several studies have shown how this aspect can contribute to increase patients’ satisfaction (Cicerone et al., 2000; Dohnke et al., 2005; Lexell et al., 2014) and self-efficacy, leading indeed towards more autonomy and more effectiveness in the performance of several behaviors (Grembowski et al., 1993).

Limitations

Limitations of this clinical trial are linked to its pilot study characteristics (i.e., small sample and a short period of treatment). Moreover, the multimodal approach adopted, including treatment with multiple compounds with diverse properties that could improve several mechanisms and functions, was not compared with another active intervention (e.g., a matched cycle-ergometer training program) but without the VR component, therefore we cannot estimate neither the impact of every single component nor the additive effects. Long-term follow-ups are indispensable to evaluate the rehabilitation efficacy too. In spite of the small number of patients the strength of this study includes the supervised exercise program, a high rate of adherence to the intervention and the use of validated outcome measures. Lack of monitoring of the physical activity outside the experimental sessions and the lack of a social activity program in the control group weaken the study too.

Conclusion

The potentialities of the proposed innovative approach have been demonstrated, even if this study should be considered as a proof-of-concept that requires further developments.

Results obtained concerning both the efficacy and the acceptability of the novel VR-based combined training in MCI populations are encouraging, but a greater number of participants are required to support the preliminary data with statistical significance. Other future challenges foresee the inclusion of a follow-up phase to provide more consistent results and highlight the long-term effectiveness of the proposed combined program, which could represent a solution to the development of a future home-based training service.

Author Contributions

SM-S planned and conducted the experiment, analyzed data and drafted the manuscript and figures. SD and FF administered the cognitive tests, conducted the experiment, analyzed data and reviewed the manuscript. SA, AZ, LG and SM designed and developed the virtual environments and reviewed the manuscript. SA and AZ assessed the treatment acceptability. SM and NJ conducted the experiment and reviewed the manuscript. MM designed training protocol, reviewed the manuscript. GR reviewed the manuscript. AV coordinated research protocol, conducted the experiment and finalized the manuscript.

Funding

This work has been partly funded by MIUR under the "Progetto di interesse Invecchiamento CNR: Goji”.

Conflict of Interest Statement

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Acknowledgments

The authors wish to acknowledge their gratitude and appreciation to the project partners for their contribution during the development of this research. The authors also thank EDEL Therapeutics SA, PSE-B/EPFL, 1015 Lausanne, Switzerland, for technical support and supply of material for antioxidant capacity determination.

References

Anderson-Hanley, C., Barcelos, N. M., Zimmerman, E. A., Gillen, R. W., Dunnam, M., Cohen, B. D., et al. (2018). The aerobic and cognitive exercise study (ACES) for community-dwelling older adults with or at-risk for mild cognitive impairment (MCI): neuropsychological, neurobiological and neuroimaging outcomes of a randomized clinical trial. Front. Aging Neurosci. 10:76. doi: 10.3389/fnagi.2018.00076

PubMed Abstract | CrossRef Full Text | Google Scholar

Appollonio, I., Leone, M., Isella, V., Piamarta, F., Consoli, T., Villa, M. L., et al. (2005). The frontal assessment battery (FAB): normative values in an Italian population sample. Neurol. Sci. 26, 108–116. doi: 10.1007/s10072-005-0443-4

PubMed Abstract | CrossRef Full Text | Google Scholar

Arlati, S., Zangiacomi, A., Greci, L., Di Santo, S. G., Franchini, F., and Sacco, M. (2017). “Virtual environments for cognitive and physical training in elderly with mild cognitive impairment: a pilot study,” in International Conference on Augmented Reality, Virtual Reality and Computer Graphics (Cham: Springer), 86–106.

Google Scholar

Baker, L. D., Frank, L. L., Foster-Schubert, K., Green, P. S., Wilkinson, C. W., McTiernan, A., et al. (2010). Effects of aerobic exercise on mild cognitive impairment: a controlled trial. Arch. Neurol. 67, 71–79. doi: 10.1001/archneurol.2009.307

PubMed Abstract | CrossRef Full Text | Google Scholar

Baumgart, M., Snyder, H. M., Carrillo, M. C., Fazio, S., Kim, H., and Johns, H. (2015). Summary of the evidence on modifiable risk factors for cognitive decline and dementia: a population-based perspective. Alzheimers Dement. 11, 718–726. doi: 10.1016/j.jalz.2015.05.016

PubMed Abstract | CrossRef Full Text | Google Scholar

Caffarra, P., Vezzadini, G., Dieci, F., Zonato, F., and Venneri, A. (2002). Rey-Osterrieth complex figure: normative values in an Italian population sample. Neurol. Sci. 22, 443–447. doi: 10.1007/s100720200003

PubMed Abstract | CrossRef Full Text | Google Scholar

Cammisuli, D. M., Innocenti, A., Franzoni, F., and Pruneti, C. (2017). Aerobic exercise effect upon cognition in mild cognitive impairment: a systematic review of randomized controlled trials. Arch. Ital. Biol. 155, 54–62. doi: 10.12871/000398292017126

CrossRef Full Text | Google Scholar

Carlesimo, G. A., Caltagirone, C., Gainotti, G., Nocentini, U., Fadda, L., Gallassi, R., et al. (1995). Batteria per la valutazione del deterioramneto mentale (parte II): standardizzazione e affidabilità diagnostic nell’identificazione di pazienti affetti da sindrome demenziale. Arch. Psicol. Neurol. Psichiatr. 56, 471–488.

Google Scholar

Carlesimo, G. A., Marfia, G. A., Loasses, A., and Caltagirone, C. (1996). Perceptual and conceptual components in implicit and explicit stem completion. Neuropsychologia 34, 785–792. doi: 10.1016/0028-3932(95)00162-x

PubMed Abstract | CrossRef Full Text | Google Scholar

Chiu, H. L., Chu, H., Tsai, J. C., Liu, D., Chen, Y. R., Yang, H. L., et al. (2017). The effect of cognitive-based training for the healthy older people: a meta-analysis of randomized controlled trials. PLoS One 12:e0176742. doi: 10.1371/journal.pone.0176742

PubMed Abstract | CrossRef Full Text | Google Scholar

Cicerone, K. D., Dahlberg, C., Kalmar, K., Langenbahn, D. M., Malec, J. F., Bergquist, T. F., et al. (2000). Evidence-based cognitive rehabilitation: recommendations for clinical practice. Arch. Phys. Med. Rehabil. 81, 1596–1615. doi: 10.1053/apmr.2000.19240

PubMed Abstract | CrossRef Full Text | Google Scholar

Cotelli, M., Manenti, R., Zanetti, O., and Miniussi, C. (2012). Non-pharmacological intervention for memory decline. Front. Hum. Neurosci. 6:46. doi: 10.3389/fnhum.2012.00046

PubMed Abstract | CrossRef Full Text | Google Scholar

Coyle, H., Traynor, V., and Solowij, N. (2015). Computerized and virtual reality cognitive training for individuals at high risk of cognitive decline: systematic review of the literature. Am. J. Geriatr. Psychiatry 23, 335–359. doi: 10.1016/j.jagp.2014.04.009

PubMed Abstract | CrossRef Full Text | Google Scholar

Dillon, A., and Morris, M. G. (1996). “Annual review of information science and technologyuser acceptance of new information technology: theories and models,” in Annual Review of Information Science and Technology (Vol. 14) (Medford, NJ: Information Today), 3–32.

Google Scholar

Dohnke, B., Knäuper, B., and Müller-Fahrnow, W. (2005). Perceived self-efficacy gained from and health effects of, a rehabilitation program after hip joint replacement. Arthritis Care Res. 53, 585–592. doi: 10.1002/art.21324

PubMed Abstract | CrossRef Full Text | Google Scholar

Etnier, J. L., Nowell, P. M., Landers, D. M., and Sibley, B. A. (2006). A meta-regression to examine the relationship between aerobic fitness and cognitive performance. Brain Res. Rev. 52, 119–130. doi: 10.1016/j.brainresrev.2006.01.002

PubMed Abstract | CrossRef Full Text | Google Scholar

Feng, Y., and Wang, X. (2012). Antioxidant therapies for Alzheimer’s disease. Oxid. Med. Cell. Longev. 2012:472932. doi: 10.1097/00002093-199803001-00002

PubMed Abstract | CrossRef Full Text | Google Scholar

Ferrucci, R., Mameli, F., Guidi, I., Mrakic-Sposta, S., Vergari, M., Marceglia, S., et al. (2008). Transcranial direct current stimulation improves recognition memory in Alzheimer disease. Neurology 71, 493–498. doi: 10.1212/01.wnl.0000317060.43722.a3

PubMed Abstract | CrossRef Full Text | Google Scholar

Folstein, M. F., Folstein, S. E., and McHugh, P. R. (1975). A practical state method for grading the cognitive state of patients for the clinician. J. Psychiatr. Res. 12, 189–198.

PubMed Abstract | Google Scholar

García-Betances, R. I., Arredondo Waldmeyer, M. T., Fico, G., and Cabrera-Umpiérrez, M. F. (2015a). A succinct overview of virtual reality technology use in Alzheimer’s disease. Front. Aging Neurosci. 7:235. doi: 10.3389/fnagi.2015.00235

PubMed Abstract | CrossRef Full Text | Google Scholar

García-Betances, R. I., Jiménez-Mixco, V., Arredondo, M. T., and Cabrera Umpiérrez, M. F. (2015b). Using virtual reality for cognitive training of the elderly. Am. J. Alzheimers Dis. Other Demen. 30, 49–54. doi: 10.1177/1533317514545866

PubMed Abstract | CrossRef Full Text | Google Scholar

Giovagnoli, A. R., Del Pesce, M., Mascheroni, S., Simoncelli, M., Laiacona, M., and Capitani, E. (1996). Trail making test: normative values from 287 normal adult controls. Ital. J. Neurol. Sci. 17, 305–309. doi: 10.1007/bf01997792

PubMed Abstract | CrossRef Full Text | Google Scholar

Grembowski, D., Patrick, D., Diehr, P., Durham, M., Beresford, S., Kay, E., et al. (1993). Self-efficacy and health behavior among older adults. J. Health Soc. Behav. 34, 89–104. doi: 10.2307/2137237

PubMed Abstract | CrossRef Full Text | Google Scholar

Haider, S., Saleem, S., Perveen, T., Tabassum, S., Batool, Z., Sadir, S., et al. (2014). Age-related learning and memory deficits in rats: role of altered brain neurotransmitters, acetylcholinesterase activity and changes in antioxidant defense system. Age 36:9653. doi: 10.1007/s11357-014-9653-0

PubMed Abstract | CrossRef Full Text | Google Scholar

Harman, D. (1981). The aging process. Proc. Natl. Acad. Sci. U S A 78, 7124–7128. doi: 10.1073/pnas.78.11.7124

PubMed Abstract | CrossRef Full Text | Google Scholar

Hill, N. T. M., Mowszowski, L., Naismith, S. L., Chadwick, V. L., Valenzuela, M., and Lampit, A. (2016). Computerized cognitive training in older adults with mild cognitive impairment or dementia: a systematic review and meta-analysis. Am. J. Psychiatry 174, 329–340. doi: 10.1176/appi.ajp.2016.16030360

PubMed Abstract | CrossRef Full Text | Google Scholar

Kazdin, A. E. (1980). Acceptability of alternative treatments for deviant child behavior. J. Appl. Behav. Anal. 13, 259–273. doi: 10.1901/jaba.1980.13-259

PubMed Abstract | CrossRef Full Text | Google Scholar

Kim, M. J., Han, C. W., Min, K. Y., Cho, C. Y., Lee, C. W., Ogawa, Y., et al. (2016). Physical exercise with multicomponent cognitive intervention for older adults with Alzheimer’s disease: a 6-month randomized controlled trial. Dement. Geriatr. Cogn. Dis. Extra 6, 222–232. doi: 10.1159/000446508

PubMed Abstract | CrossRef Full Text | Google Scholar

Lexell, E. M., Flansbjer, U. B., and Lexell, J. (2014). Self-perceived performance and satisfaction with performance of daily activities in persons with multiple sclerosis following interdisciplinary rehabilitation. Disabil. Rehabil. 36, 373–378. doi: 10.3109/09638288.2013.797506

PubMed Abstract | CrossRef Full Text | Google Scholar

Liu, J., Roussel, C., Lagger, G., Tacchini, P., and Girault, H. H. (2005). Antioxidant sensors based on DNA-modified electrodes. Anal. Chem. 77, 7687–7694. doi: 10.1021/ac0509298

PubMed Abstract | CrossRef Full Text | Google Scholar

Liu, J., Su, B., Lagger, G., Tacchini, P., and Girault, H. H. (2006). Antioxidant redox sensors based on DNA modified carbon screen-printed electrodes. Anal. Chem. 78, 6879–6884. doi: 10.1021/ac0608624

PubMed Abstract | CrossRef Full Text | Google Scholar

Magni, E., Binetti, G., Bianchetti, A., Rozzini, R., and Trabucchi, M. (1996a). Mini-mental state examination: a normative study in Italian elderly population. Eur. J. Neurol. 3, 198–202. doi: 10.1111/j.1468-1331.1996.tb00423.x

PubMed Abstract | CrossRef Full Text | Google Scholar

Magni, E., Binetti, G., Padovani, A., Cappa, S. F., Bianchetti, A., and Trabucchi, M. (1996b). The mini-mental state examination in Alzheimer’s disease and multi-infarct dementia. Int. Psychogeriatr. 8, 127–134. doi: 10.1017/s1041610296002529

PubMed Abstract | CrossRef Full Text | Google Scholar

Marceglia, S., Mrakic-Sposta, S., Rosa, M., Ferrucci, R., Mameli, F., Vergari, M., et al. (2016). Transcranial direct current stimulation modulates cortical neuronal activity in Alzheimer’s disease. Front. Neurosci. 10:134. doi: 10.3389/fnins.2016.00134

PubMed Abstract | CrossRef Full Text | Google Scholar

Marzorati, M., Di Santo, S. G., Mrakic-Sposta, S., Moretti, S., Jesuthasan, N., Caroppo, A., et al. (2016). “Supporting physical and cognitive training for preventing the occurrence of dementia using an integrated system: a pilot study,” in International Conference on Wireless Mobile Communication and Healthcare (Cham: Springer), 367–374.

Google Scholar

Moll van Charante, E. P., Richard, E., Eurelings, L. S., van Dalen, J. W., Ligthart, S. A., van Bussel, E. F., et al. (2016). Effectiveness of a 6-year multidomain vascular care intervention to prevent dementia (preDIVA): a cluster-randomised controlled trial. Lancet 388, 797–805. doi: 10.1016/S0140-6736(16)30950-3

PubMed Abstract | CrossRef Full Text | Google Scholar

Mrakic-Sposta, S., Gussoni, M., Montorsi, M., Porcelli, S., and Vezzoli, A. (2012). Assessment of a standardized ROS production profile in humans by electron paramagnetic resonance. Oxid. Med. Cell. Longev. 2012:973927. doi: 10.1155/2012/973927

PubMed Abstract | CrossRef Full Text | Google Scholar

Mrakic-Sposta, S., Gussoni, M., Montorsi, M., Porcelli, S., and Vezzoli, A. (2014). A quantitative method to monitor reactive oxygen species production by electron paramagnetic resonance in physiological and pathological conditions. Oxid. Med. Cell. Longev. 2014:306179. doi: 10.1155/2014/306179

PubMed Abstract | CrossRef Full Text | Google Scholar

Mrakic-Sposta, S., Gussoni, M., Porcelli, S., Pugliese, L., Pavei, G., Bellistri, G., et al. (2015). Training effects on ROS production determined by electron paramagnetic resonance in master swimmers. Oxid. Med. Cell. Longev. 2015:804794. doi: 10.1155/2015/804794

PubMed Abstract | CrossRef Full Text | Google Scholar

Mrakic-Sposta, S., Vezzoli, A., Malacrida, S., Falla, M., and Strapazzon, G. (2017). “Direct” and “Indirect” methods to detect oxidative stress during acute or chronic high-altitude exposure. High Alt. Med. Biol. 18, 303–304. doi: 10.1089/ham.2017.0067

PubMed Abstract | CrossRef Full Text | Google Scholar

Ngandu, T., Lehtisalo, J., Solomon, A., Levalahti, E., Ahtiluoto, S., Antikainen, R., et al. (2015). A 2 year multidomain intervention of diet, exercise, cognitive training and vascular risk monitoring versus control to prevent cognitive decline in at-risk elderly people (FINGER): a randomised controlled trial. Lancet 385, 2255–2263. doi: 10.1016/S0140-6736(15)60461-5

PubMed Abstract | CrossRef Full Text | Google Scholar

Novelli, G., Papagno, C., Capitani, E., Laiacona, M., Vallar, G., and Cappa, S. F. (1986). Tre test clinici di ricerca e produzione lessicale. Taratura su soggetti normali. Arch. Psicol. Neurol. Psichiatr. 47, 477–506.

Google Scholar

Nunomura, A., Castellani, R. J., Zhu, X., Moreira, P. I., Perry, G., and Smith, M. A. (2006). Involvement of oxidative stress in Alzheimer disease. J. Neuropathol. Exp. Neurol. 65, 631–641. doi: 10.1097/01.jnen.0000228136.58062.bf

PubMed Abstract | CrossRef Full Text | Google Scholar

Rizzo, A. A., and Kim, G. J. (2005). A SWOT analysis of the field of VR rehabilitation and therapy. Presence Teleoper. Virtual Environ. 14, 119–146. doi: 10.1162/1054746053967094

CrossRef Full Text | Google Scholar

Serino, S., Pedroli, E., Tuena, C., De Leo, G., Stramba-Badiale, M., Goulene, K., et al. (2017). A novel virtual reality-Based training protocol for the enhancement of the “mental frame syncing” in individuals with Alzheimer’s disease: a development-of-concept trial. Front. Aging Neurosci. 9:240. doi: 10.3389/fnagi.2017.00240

PubMed Abstract | CrossRef Full Text | Google Scholar

Sherman, D. S., Mauser, J., Nuno, M., and Sherzai, D. (2017). The efficacy of cognitive intervention in mild cognitive impairment (MCI): a meta-analysis of outcomes on neuropsychological measures. Neuropsychol. Rev. 27, 440–484. doi: 10.1007/s11065-017-9363-3

PubMed Abstract | CrossRef Full Text | Google Scholar

Spinnler, H., and Tognoni, G. (1987). Standardizzazione e taratura italiana di test neuropsicologici. Ital. J. Neurol. Sci. 8, 1–120.

Tacchini, P., Lesch, A., Neequaye, A., Lagger, G., Liu, J., Cortés-Salazar, F., et al. (2013). Electrochemical pseudo-titration of water-soluble antioxidants. Electroanalysis 25, 922–930. doi: 10.1002/elan.201200590

CrossRef Full Text | Google Scholar

Talassi, E. M., Guerreschi, M., Feriani, V., Fedi, A., Bianchetti, M., and Trabucchi, M. (2007). Effectiveness of a cognitive rehabilitation program in mild dementia (MD) and mild cognitive impairment (MCI): a case control study. Arch. Geront. Geriat. 44, 391–399. doi: 10.1016/j.archger.2007.01.055

PubMed Abstract | CrossRef Full Text | Google Scholar

Tieri, G., Morone, G., Paolucci, S., and Iosa, M. (2018). Virtual reality in cognitive and motor rehabilitation: facts, fiction and fallacies. Expert Rev. Med. Devices 15, 107–117. doi: 10.1080/17434440.2018.1425613

PubMed Abstract | CrossRef Full Text | Google Scholar

Tönnies, E., and Trushina, E. (2017). Oxidative stress, synaptic dysfunction and Alzheimer’s disease. J. Alzheimers Dis. 57, 1105–1121. doi: 10.3233/jad-161088

PubMed Abstract | CrossRef Full Text | Google Scholar

Vellas, B., Carrie, I., Gillette-Guyonnet, S., Touchon, J., Dantoine, T., Dartigues, J. F., et al. (2014). Mapt study: a multidomain approach for preventing Alzheimer’s disease: design and baseline data. J. Prev. Alzheimers Dis. 1, 13–22.

PubMed Abstract | Google Scholar

Wolf, M. M. (1978). Social validity: the case for subjective measurement or how applied behavior analysis is finding its heart. J. Appl. Behav. Anal. 11, 203–214. doi: 10.1901/jaba.1978.11-203

PubMed Abstract | CrossRef Full Text | Google Scholar