Assessment Practice of Patient-Centered Outcomes in Surgical Neuro-Oncology: Survey-Based Recommendations for Clinical Routine

Abstract

The psycho-oncological burden related to the diagnosis of an intracranial tumor is often accompanied by neurocognitive deficits and changes in character, overall affecting health-related quality of life (HRQoL) and activities of daily living. Regular administration of adequate screening tools is crucial to ensure a timely detection of needs for support and/or specific interventions. Although efforts have been made to assure the quality of neuro-oncological care, clinical assessment practice of patient-reported outcomes (PROs) remains overall heterogeneous, calling for a concise recommendation tailored to neuro-oncological patients. Therefore, this survey, promoted by the German Society of Neurosurgery, was conducted to evaluate the status quo of health care resources and PRO/neurocognition assessment practices throughout departments of surgical neuro-oncology in Germany. 72/127 (57%) of registered departments participated in the study, including 83% of all university hospital units. A second aim was to shed light on the impact of quality assurance strategies (i.e., department certification as part of an integrative neuro-oncology cancer center; CNOC) on the assessment practice, controlled for interacting structural factors, i.e., university hospital status (UH) and caseload. Despite an overall good to excellent availability of relevant health care structures (psycho-oncologist: 90%, palliative care unit: 97%, neuropsychology: 75%), a small majority of departments practice patient-centered screenings (psycho-oncological burden: 64%, HRQoL: 76%, neurocognition: 58%), however, much less frequently outside the framework of clinical trials. In this context, CNOC affiliation, representing a specific health care quality assurance process, was associated with significantly stronger PRO assessment practices regarding psycho-oncological burden, independent of UH status (common odds ratio=5.0, p=0.03). Nevertheless, PRO/neurocognitive assessment practice was not consistent even across CNOC. The overall most commonly used PRO/neurocognitive assessment tools were the Distress Thermometer (for psycho-oncological burden; 64%), the EORTC QLQ-C30 combined with the EORTC QLQ-BN20 (for HRQoL; 52%) and the Mini-Mental Status Test (for neurocognition; 67%), followed by the Montreal Cognitive Assessment (MoCA; 33%). Accordingly, for routine clinical screening, the authors recommend the Distress Thermometer and the EORTC QLQ-C30 and QLQ-BN20, complemented by the MoCA as a comparatively sensitive yet basic neurocognitive test. This recommendation is intended to encourage more regular, adequate, and standardized routine assessments in neuro-oncological practice.

Introduction

The diagnosis of an intracranial tumor confronts patients on the one hand with the burden of an oncological disease, but on the other hand also with neurocognitive deficits and changes in character, which overall affect health-related quality of life (HRQoL) and activities of daily living. Reliable patient-reported outcome measures (PROMs) can facilitate early recognition of psychosocial burden, depression, and anxiety and can lead to adequate support (1, 2). Accordingly, assessment and monitoring of neurocognitive function can play an important role in therapy and disease monitoring (3). Therefore, timely and closely followed patient-reported outcome (PRO) and performance-based assessments seem highly advisable to ensure a comprehensive neuro-oncological care, and have recently attracted increasing interest even beyond the context of clinical trials. However, to date there is no consensus regarding the best clinical and scientific practice of PRO and performance-based assessments in neuro-oncological patients.

Approximately 10 years ago, a standardized certification for neuro-oncology centers was implemented in Germany aiming at standardizing and improving patient care as comprehensively as possible. Since then, many positive developments have been observed in the field of neuro-oncology, driven by enhanced interdisciplinary cooperation. Despite all this, the sole requirement to date is to offer psycho-oncology counseling to at least 10% of brain tumor patients. Accordingly, clinical experience shows that the implemented standard operating procedures linked to certification have not yet reached a satisfactory level in terms of comprehensiveness and detail. For instance, adequate PROMs have not been included in official recommendations, and other important aspects, such as neurocognition, play a subordinate role, since no specifications are required. A fixed screening scheme to identify all types of related support needs would therefore be desirable as a standard operating procedure, even beyond the framework of certified neuro-oncology centers (CNOC; certified by the German Cancer Society [DKG]).

For this purpose and as a first step, we designed a survey, to describe the status quo of different assessment strategies applied throughout neuro-oncological units in CNOC and non-CNOC in Germany, also considering the university status as a potential confounder. Local organizational and health care structures are also considered to unravel interactions between existing structures as well as the clinical and scientific practice to evaluate psycho-oncological burden, HRQoL, and neurocognition in brain tumor patients. To conclude, this work provides a recommendation for a simple and little time-consuming assessment, based on the practical results of this survey and the literature.

Material and Methods

Study Design

The survey was designed by the authors on behalf of the neuro-oncological section of the German Society of Neurosurgery (DGNC) and was sent to all registered neurosurgical centers treating neuro-oncologic patients (i.e., n=127 centers) throughout Germany. The survey was conducted between November 2019 and April 2020. The heads of the neurosurgical departments or (if existing) of the specialized sub-units for neuro-oncological surgery were invited via electronic mail and/or telephone to participate in the survey. To ensure maximum survey response, multiple reminders were placed via electronic mail or phone calls to the departments’ secretaries. If no response was received after at least six reminders, the department was excluded from the study. The survey contained 28 multiple- and single-choice questions divided into four sections, mainly covering the following points (for detailed overview, cf. translated survey in the supplement): (i) center organization (CNOC, university hospital [UH], specialized neuro-oncologic outpatient clinic, caseload); (ii) health care structure (psycho-oncology, neuropsychology, palliative care); (iii) HRQoL assessment (practice and tools); (iv) assessment of psycho-oncological burden, depression, and anxiety (practice and tools); (v) assessment of neurocognition (practice and tools).

Statistical Analysis

Data were stratified by institutional academic level (two levels: UH; others) and by affiliation to a CNOC (two levels: affiliated; not affiliated) to investigate the association of institution type and certification on the health care structure as well as on the practice of PRO assessments.

Differences between groups (stratified by, e.g., CNOC affiliation) with respect to binary outcomes such as the existence of health care structures were analyzed using the Mantel Haenszel Chi-squared test with continuity correction, controlling for the respective confounding co-factor (e.g., UH). In case the Mantel Haenszel test was significant, Fisher’s exact tests were calculated post-hoc for the respective subgroups.

For ordinal or continuous outcome variables, such as the time span between tumor diagnosis and first contact to palliative care, Wilcoxon’s rank sum test with continuity correction was calculated. Associations between ordinal or continuous variables and binary variables (e.g., caseload and UH) were analyzed using point-biserial correlations. To control for the interfering effect of a second significant factor, partial correlations were additionally calculated when appropriate. Statistically significant differences are generally reported as exact p-values. Whenever appropriate, a false discovery rate (FDR) correction (4) was applied (referred to as FDR-corrected throughout the manuscript). The statistical analysis was performed using R (version 3.6.3; R Studio version 1.1.463).

Results

Out of 127 neurosurgical departments (including 36 UHs and 46 CNOCs), 72 departments (56.7%) participated. Four departments (3.1%) declined to participate; the remaining 51 departments did not respond despite being approached at least six times. 14 out of 16 German federal states returned the survey, with a certain overrepresentation of the districts North Rhine-Westphalia (21%) and Bavaria (15%).

Center Organization

30 (42%) of the participating departments were part of UHs, as opposed to 37 (51%) university-affiliated teaching hospitals, and 5 (7%) district hospitals without university affiliation. 35 departments (49%) were part of CNOCs, and 60 departments (86%) declared to run a specialized neuro-oncologic outpatient clinic with a median caseload of 250 neuro-oncological consultations per year (range: 20-3000). This implies that this survey included 83% of all 36 German UHs running a neurosurgical unit and 76% of all 46 German CNOCs. Of note, there was a highly significant relationship between UH and CNOC status with most departments having the status of both (n=24/72) or neither UH nor CNOC (n=31/72; p<0.0001; Table 1).

For this reason, the use of the Mantel-Haenszel test was considered appropriate (cf. Statistical analysis). An overview of the caseloads specifically referring to primary brain tumors (referred to as “caseload” throughout the manuscript) is provided in Table 2. In our sample, the caseload showed a strong, significant correlation with UH (r=0.63; p<0.0001) which remains significant when controlling for the factor CNOC affiliation using a partial correlation approach (r=0.55; p<0.0001). In contrast, the moderate correlation of the caseload with the CNOC affiliation of the department (r=0.37; p=0.002) did not survive when controlled for the UH status (r=0.04; p=0.55).

Of interest, 28 (47%) of the participating departments replied to perform awake neurosurgery on a regular basis, ranging from 32% without to 62% with CNOC affiliation, irrespective of the UH status (Cochrane Mantel-Haenszel test: cOR=3.5, p=0.002) and caseload.

Health Care Structure

Except for departments without UH status nor CNOC affiliation (77%), psycho-oncology services were fully available in all other participating centers (Table 2), as reflected by a moderate, significant correlation of this care structure with caseload size (r=0.26; p=0.03). Across all centers, the availability of psycho-oncological support was higher for inpatients compared to outpatients (i.e., 78% versus 54%, respectively) and was mostly provided by psychologists (79%) and/or by medical staff (27%), and very rarely by pastors (1%).

Accordingly, inpatient palliative care was available in nearly all departments (97% overall), whereas the existence of outpatient palliative care services ranged from 43% to 71% (Table 2). In most centers, the respective services were provided by the hospitals and relatively rarely in collaboration with other institutions (psycho-oncology: 7%; palliative care: 15%, overall). The median time span between tumor diagnosis and first contact with palliative care was 39 weeks (i.e., 9 months), ranging from 1 to 87 weeks (i.e., 20 months), statistically unrelated to CNOC or UH status.

Overall, neuropsychological units/services existed in 75% of participating hospitals, ranging from 55% to 100% depending on department affiliation: there was a statistical trend towards better availability of neuropsychologists in UH (p=0.095) whereas the CNOC affiliation factor had no significant influence (Table 2 and Figure 1). Accordingly, an utmost weak correlation was observed between neuropsychologist availability and caseload (r=0.21; p=0.09).

Figure 1

Assessment of Psycho-Oncological Burden, Depression, and Anxiety

The assessment of psycho-oncological burden, depression, and anxiety is practiced in most of the participating departments (i.e., 64%), more commonly across CNOCs (p=0.03; Table 3), but independent of the caseload (r=0.17; p=0.16). However, this influence of CNOC affiliation on assessment practice was not significant after correcting for exclusively study-related practice, i.e., when considering only assessments outside the context of clinical trials (Figure 2). Overall, the majority of patients (median estimation 80%) is assessed in departments which reported to perform PROMs of psycho-oncological distress, depression, and anxiety. Relatively rarely (20%), the caregivers were included in distress assessments, irrespective of department affiliation and certification (Table 3); however, significantly associated with higher caseloads (r=0.46; p=0.02).

Figure 2

The most common PROM carried out to assess psycho-oncological burden, depression, and/or anxiety was by far the Distress Thermometer (DT; (5) 61% overall), followed by the Hospital Anxiety and Depression Scale (HADS; (6) 33% overall), the Beck Depression Inventory (BDI; (7, 8) 22% overall), and the Hornheider Screening Instrument (HIS; (9) 17% overall). The Basic Documentation for Psycho-Oncology [PO-Bado; (10)], which is an external assessment instrument, was used by four departments (i.e., 11% overall) in addition to at least one of the aforementioned PROMs (Figure 3A). When considering only the 22 departments which perform distress assessments (also) outside the context of clinical trials, the three most frequently used tools were the DT (64%), the HSI (23%), and the HADS (18%), followed by the BDI (14%) and the PO-Bado (9%).

Figure 3

Overall, the assessments were mostly performed by physicians (58%), followed by nurses (56%) and psycho-oncologists (28%), and rarely by students (9%) and case managers (5%) (Figure 3B). To account for the association between CNOC affiliation and assessment practice (in contrast to an utmost minimal association with UH), the descriptive data shown in Figure 3 are stratified by CNOC.

Health-Related Quality of Life Assessment

Although HRQoL assessment is practiced in the vast majority of departments (76%; Table 2), irrespective of their caseloads (r=0.18; p=0.15), the assessment is widely limited to clinical trials. Consequently, the percentage of centers with clinical routine practice in HRQoL assessment outside the context of studies reaches only 34%, statistically independent of their affiliations and caseload (Figure 2). Moreover, screening of brain tumor patients for HRQoL is generally irregular, even in departments that perform such screening (median 50% of patients, overall; see supplemental Table S1).

Figure 4A shows that the most commonly used HRQoL screening instrument was the 30-items quality of life questionnaire of the European Organisation for Research and Treatment of Cancer (EORTC QLQ-C30) accompanied with its brain module (EORTC QLQ-BN20) (15), overall (52%) as well as in CNOC (64% vs. 31% no CNOCs). In contrast, outside CNOC departments, the Short Form Health 36 [SF-36; (16)] was mostly used (50% versus 29% in CNOCs; 36% overall). The shortened version of the SF-36 [i.e., SF-12; (17)] represented the third most frequent HRQoL assessment tool (24% overall; CNOCs: 18%; no CNOCs: 19%); further instruments were named by single centers (cf. Figure 4 legend).

Figure 4

Additional analysis of the subset of 24 departments performing HRQoL assessments other than in the context of clinical trials revealed a similar but more diversified pattern (EORTC: 42%; SF-36: 26%; SF-12: 5%; others: 11%).

In line with the distress assessments, the HRQoL self-reports are again mostly obtained by physicians, followed by nurses; in contrast, case managers are only very rarely involved (Figure 4B).

Neurocognitive Assessment

The overall rate of departments practicing neurocognitive assessments was 58% (Table 2), irrespective of the caseload. However, only 29% of participating departments (also) perform cognitive assessments unrelated to studies (statistically independent of center affiliation, certification, and caseload; Figure 2). Accordingly, the overall median percentage of brain tumor patients undergoing cognitive assessment in each department was low, i.e., 25% (with estimates ranging from 5% to 100%; cf. Supplementary Table S1), indicating that few centers follow regular clinical practice in this regard.

By far, the most commonly used screening instrument for neurocognitive functions was the mini mental status test (MMST) (18) (67% overall), particularly in centers without CNOC affiliation (Figure 5A). Accordingly, the MMST was practiced by seven out of nine (i.e., 78% of the) departments which practice neurocognitive assessment (also) beyond the exclusive context of clinical trials and answered this question.

Figure 5

The Montreal Cognitive Assessment [MoCA; (22)] represented the overall second most used neurocognitive test (33% overall) and reached the level of the MMST in CNOC (Figure 5A). When excluding the departments which practice neurocognitive assessments solely in clinical trials, the test was only named by one of the remaining nine centers.

Similar to the assessments of psycho-oncological burden, depression, and anxiety, as well as HRQoL, the tests were mostly performed by physicians. Nurses and neuropsychologists were also often included in the assessment, especially in CNOC (Figure 5B). Outside the context of clinical trials, the distribution was even between physicians and neuropsychologists (both 53%), followed by nurses (18%), whereas students played no role in collecting neurocognitive screening data (0%).

Regarding the time points of neurocognitive tests, the vast majority of the 35 centers practicing this assessment type investigate patients before (86%) and after (91%) surgery; 57% responded to conduct neurocognitive assessments during follow-up as well. Overall, 26% of the centers performing awake surgery include neurocognitive tests in the intraoperative setting (compared to 36% when considering only centers which practice neurocognitive screenings in clinical routine).

Discussion

This survey investigated available health care structures and PRO as well as neurocognitive assessment practice in German neurosurgical departments, depending on their UH status and CNOC affiliation as well as on their caseloads of primary brain tumor patients. To the best of the authors’ knowledge, this is the first survey on this topic, providing a comprehensive overview due to the inclusion of the majority (57%) of the registered neurosurgical departments. Despite an overall good to excellent availability of relevant health care structures (i.e., referring to psycho-oncology, palliative care, and neuropsychology), the clinical routine assessment of relevant PROs, HRQoL assessment, and neurocognitive functions is limited, especially outside clinical trials. However, CNOC affiliation, representing a specific health care quality assurance process, was associated with significantly stronger PRO assessment practices regarding psycho-oncological burden, depression, and anxiety, independent of UH status.

Assessment of Psycho-Oncological Burden and Adequate Support

Irrespective of tumor entity and prognosis, neuro-oncological patients are at risk for psychological comorbidities (23–25). A screening should be feasible within minutes and results have to be interpreted immediately in order to provide adequate support (26).

In line with our assumption from everyday clinical practice, psychosocial assessment was only carried out by a minority of participating departments outside clinical trials (42%). Although the majority of clinicians attach high importance to screening, its implementation in clinical routine is challenging (27), e.g., due to exhausted workload capacity of qualified staff. This applies particularly to screening instruments developed for cancer patients in general, which might be too complex and time-consuming for brain tumor patients, and thus difficult for them to manage. Even the application of seemingly quick and disease-specific self-report forms can bind significant staff resources, for instance, when patients need assistance in completing the form due to neurocognitive and/or other neurological deficits, or when the consecutive detection of needs requires further steps in patient management. On the other hand, the completion of screening forms by accompanying caregivers on behalf of patients reduces time expenditure for qualified medical personnel but leads to biased assessments. Compared to patients, it is even more difficult to address their relatives as well, reflected by only 20% of caregivers being included in distress assessments despite the generally heavy burden (28, 29). Although not yet widely used in clinical practice (and therefore not included in this survey), several established instruments are available to assess caregiver burden, e.g., the concise 12-items short form of the Zarit Burden Interview [ZBI] (30).

External factors such as certification requirements are leading to faster implementation of screening practice. This is underlined by the fact that psycho-oncological assessment was more common across CNOCs, where not only access to studies and specialized therapy is provided but also the required health care structures for patients are in place. However, implementing quality standards takes time. Looking at the past five years in the certification process, the numbers of initial psycho-oncological counselling have increased very slowly from 11.7% since 2015 (31) to 18.5% (32). Without the control of minimum requirements regarding the rate of patients to be psycho-oncologically assessed, the clinical assessment practice is prone to remain inconsistent.

The German psycho-oncology guideline recommends HADS-D, HSI, DT and PO-Bado, among others, as screening instruments. The BDI should not be regarded as a screening instrument for psycho-oncological distress/burden of disease. Nevertheless, it was indicated by 22% of participants and can be considered as a complementary and comparatively sensitive 21-items instrument to assess depression (as a disease to be medically treated with a considerable prevalence in brain tumor patients) (33). Since self-assessment might not be possible in every neuro-oncological patient, e.g., due to cognitive deficits, an external assessment by the physician can be helpful. If depression is suspected, this will be followed by a psycho-oncological consultation and specific diagnostics and, if necessary, therapy (cf. guideline unipolar depression of the Association of the Scientific Medical Societies in Germany [AWMF]) (34).

Regarding psycho-oncological screening instruments, the PO-Bado (11%) and the HSI (16%) were used rather rarely. The PO-Bado is an external assessment tool (10); hence, for the interpretation of its results, it should be considered that the physicians’ estimations do not necessarily reflect the patients’ perspectives (35). The HSI, a self-assessment screening instrument, as well as the PO-Bado, are widely used screening tools within Germany.

Both do not meet international quality criteria, which makes them rather unsuitable for international comparison.

The two most frequently used screening instruments were the HADS (33%) and the DT (61%). The HADS, an internationally well-established psycho-oncological screening instrument, refers exclusively to anxiety and depression, whereas the DT allows for the assessment of a much wider range of psychosocial problems and needs (36). Goebel and Mehdorn (37) validated the DT in brain tumor patients. Here, patients are considered to potentially carry a clinically relevant burden if the score is ≥ 6. In a previous work by Rapp et al. the relationship between HADS and DT was analyzed in more than 470 patients (26) resulting in the recommendation to consult a psycho-oncologist if the DT score is ≥ 5 and emotional problems ≥ 2.

The predominant use of the DT – at least in surgical neuro-oncology in Germany – might not only mirror the broad international acceptance of its short answer option but be also due to its simple, easily administrable, and non-stigmatizing character. The importance of these characteristics should not be underestimated in respect of the considerable prevalence of cognitive deficits in the target population, which interferes with the completion of long and complex questionnaires (38, 39). To address the specific needs of brain tumor patients, Goebel et al. have recently developed an adapted version of the problem list of the DT [HEAT; (38)], focusing on a more disease-specific needs assessment. This test still needs to be validated but could become a highly valuable PROM for brain tumor patients in the future (cf. Table 4).

Currently, we recommend the DT as a psycho-oncological screening tool for brain tumor patients (Table 4).

Health-Related Quality of Life Assessment

Serving as an independent predictor of therapy compliance and survival (44), HRQoL was the first PROM serving to evaluate new schemes of neuro-oncological therapy (45). In recent decades, its assessment has gained importance as an outcome measure of treatment response, far beyond the use as an endpoint in clinical trials (46, 47). In this regard, it is not surprising that this study showed a predominance of HRQoL assessment compared to other PROs evaluated (76% overall; up to 100% in CNOCs with UH status), although it also consumed relatively costly human resources (including 72% physicians and 58% nurses, overall). However, only 34% of participating departments practice HRQoL assessments outside clinical trials (independent of the institutional status). This demonstrates that the benefits of their use in improving clinical outcome prediction, complementing standard clinical outcomes, and detecting specific support needs are far from being exhausted. This finding might be influenced by (i) the lack of clear recommendations for HRQoL PROMs in current guidelines, (ii) the copyright protection of most common HRQoL PROMs making them less easily accessible, and (iii) logistic reasons related to the increased manpower required to ensure consequent assessment. In agreement with its predominance in European clinical trials, the EORTC QLQ-C30/BN-20 (15) was the most commonly used instrument (42%) in German centers, too. Notwithstanding its excellent quality in terms of internal consistency, content validity, and construct validity (48), as well as its validation for the specific group of neuro-oncological patients, this comparatively long 50-item questionnaire may be hard for patients to cope with, especially when being part of a comprehensive and repeated assessment based on multiple PROMs. This might be one reason why shorter HRQoL assessment tools (i.e., the 36-item SF-36 or its 12-item short form) (16, 17) were ranked second in frequency of use by the departments participating in this survey (SF-36: 26%; SF-12: 5%), especially by departments without CNOC affiliation, which are generally less influenced by specific requirements of clinical trials. Therefore, prospective studies and novel computerized concepts such as the computerized adaptive test version of the EORTC QLQ-C30 [EORTC CAT; (42, 43)] are highly appreciated. In the upcoming version, the authors intend to achieve a maximum PROM quality whilst reducing time needed to complete the questionnaires.

In summary, we presently recommend the EORTC QLQ-C30/BN-20 to assess quality of life in brain tumor patients (Table 4).

Neurocognitive Assessment

The practice of neurocognitive assessment is rather limited (i.e., 58% overall; 29% outside clinical trials) and widely restricted to relatively simplistic dementia screening tools despite an apparently good overall availability of qualified investigators (e.g., almost 100% neuropsychologists, especially in UHs). Recent literature discussed brief cognitive screenings to be insensitive to important cognitive symptoms; thus, rendering them inadequate (49). The vast majority of departments assessing neurocognitive functions reported to use the well-known and easily administrable MMST, even more if outside the context of clinical studies. In contrast to its broad acceptance, the MMST, originally developed for dementia screening, demonstrates relatively low sensitivity regarding the detection of cognitive deficits in brain tumor patients (50). In comparison, another dementia screening test, i.e., the MoCA, which also allows for a relatively time-efficient and well standardized test administration, was reported to perform significantly better in neuro-oncological patients (p<0.0001) (51). This might explain why the MoCA is used relatively frequently in clinical trials, almost overtaking the MMST in departments with considerable study activity (i.e., CNOCs).

More sensitive but also more time-consuming and potentially burdensome, neurocognitive test batteries are apparently very rarely part of clinical assessments in German neurosurgical departments (2 centers, 3% overall), although nowadays generally recommended (52). This is noteworthy since cognitive deficits correlate strongly not only with HRQoL and activities of daily living (53) but also with tumor progression (54) and survival (55). Accordingly, timely detection of neurocognitive deficits using appropriate, sensitive screening instruments appears advisable to enable the responsible physicians to recommend adequate diagnostic, supportive, or therapeutic interventions. Here, a comprehensive but shortened neurocognitive testing instrument could help to improve assessment practice, and thus the detection of cognitive deficits and related support needs. It would need to be tailored to the limited attention span and coping ability of (newly diagnosed) neuro-oncologic patients and sufficiently standardized to be administered by trained nurses or students. Such a neurocognitive test battery including five parallel versions is currently being evaluated for use in glioblastoma patients in the multicentric NOA-19 study (20, 21). The results will help to find an appropriate neurocognitive test strategy for brain tumor patients in the future.

For now, we recommend the MoCA as basic assessment tool for neurocognition (Table 4).

Impact of Quality Assurance Strategies and Personnel Structure

The higher rate of psycho-oncological PRO assessments in the subgroup of CNOC departments demonstrates the potential of quality-assuring instruments to make a change in assessment practice, building the ground for the detection of support needs and subsequent initialization of supporting interventions. Therefore, the integration of further patient-centered outcome assessments into quality assurance strategies (such as standard-operating procedures, guidelines, or controlled certification requirements) seems advisable to achieve optimized health care standards in neuro-oncology also outside clinical studies. In this context, concrete recommendations regarding an ideal time frame for first contact to palliative care units might also be valuable, as early and regular contact with a palliative care team beginning within a few weeks after first tumor diagnosis has been shown to improve HRQoL, symptom burden, and mood in patients of other oncological entities (56, 57). Moreover, advanced care planning (ACP) is clearly appreciated by the vast majority of neuro-oncological patients and is highly dependent on the patient’s general, psychological, and neurocognitive state still being adequate (58, 59). These and other points favoring an integrative palliative care approach argue for earlier involvement of palliative care teams than currently practiced in German departments of surgical neuro-oncology (with a median of 9 months, up to 20 months in our data set).

Another key finding of this work is the imbalance between existing health care structures and available instruments for assessing clinically clearly important PROs and neurocognition on the one hand, and the heterogeneous and incomplete clinical practice of such assessments on the other. As mentioned earlier, one reason for this might be the traditional dependency of PROM and neurocognitive assessments on highly qualified, high-cost personnel like physicians. Especially, the completion of traditional PROMs, i.e., self-report forms to be completed by patients, could be undertaken by less costly staff if appropriate training was to be provided. To which extent this model is transferrable to neurocognitive testing depends on the degree of standardization and the ease of administration of the tests used (which in our view are sufficiently high regarding, e.g., the MMST or the MoCA test).

Strengths and Limitations

This survey, addressing all registered centers of surgical neuro-oncology in Germany to avoid selection bias, draws a rather comprehensive picture of the neuro-oncological PRO practices in the country. Even pursuing this inclusive approach, we achieved an excellent participation rate of 57% (n=72/127, including n=42 non-UHs) compared to recent surveys in the field of neuro-oncology which followed a similar inclusion strategy [e.g., 5%; n=362/7280; (60)]. Other surveys reporting response rates in the range of 36% (61) up to 75% (62) come with the limitation of addressing a highly selected group of centers [e.g., 28 centers across eleven European nations; (62)].

Despite this methodological strength, the survey is not fully representative due to (i) the missing centers, especially regarding non-UHs and non-CNOCs (selection bias), as well as (ii) missing values due to incomplete surveys/responses, and (iii) the at least potential subjectiveness/rater dependency of several survey items (since the data are based on information provided by medical consultants rather than on official/reliable statistics of the respective institutions). Moreover, (iv) the survey was addressed to neurosurgical units as one representative part of integrative neuro-oncological care centers. To provide a more comprehensive overview of assessment practices dependent on the stages of treatment/disease (and on the distinct disciplines involved) was beyond the scope of this work and will be subject to an upcoming survey.

In the present inquiry, the existence of certain institutional and medical structures was surveyed (e.g., presence of a neuropsychologist), whereas the extent to which this (personnel) structure is actively involved in the assessment of brain tumor patients was not. Moreover, it should be emphasized that the existence of health care structures and practice of assessments (to detect neurocognitive and psycho-oncological needs) do not per se lead to improved quality of care – unless followed by adequate interpretation of the outcomes and timely initiation of appropriate measures. The question of which resources and assessment tools, mediated by consecutive interventions/support, have a significant impact on health care quality was beyond the scope of this work and might be further addressed in a prospective study.

Concluding Recommendations

The status quo of PRO and neurocognition assessment in surgical neuro-oncology shows that despite existing care structures, even in CNOCs there are no consistent standard procedures. As a consequence, many patients and caregivers are left alone with their needs and burdens. Widespread adoption of screening tools is essential to implement regular PRO and neurocognitive assessments in clinical practice. Therefore, screening tools are best suited when they bridge the gap between high test quality and practical considerations: tests should be as familiar as possible to the hospital staff, little time-consuming, and easy to perform and to evaluate. With regard to the results of this survey and literature, we hope that our concise recommendation (provided in Table 4) will encourage more regular, appropriate and standardized routine assessments in neuro-oncological practice.

Funding

This work received financial support from the University of Cologne, Faculty of Medicine, to cover the publication costs.

Publisher’s Note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

References

• 1

  PelletierGVerhoefMJKhatriNHagenN. Quality of Life in Brain Tumor Patients: The Relative Contributions of Depression, Fatigue, Emotional Distress, and Existential Issues. J Neurooncol (2002) 57(1):41–9. doi: 10.1023/A:1015728825642

  • CrossRef
  • Google Scholar

• 2

  IJzerman-KorevaarMSnijdersTJde GraeffATeunissenSCCMde VosFYF. Prevalence of Symptoms in Glioma Patients Throughout the Disease Trajectory: A Systematic Review. J Neurooncol (2018) 140(3):485–96. doi: 10.1007/s11060-018-03015-9

  • CrossRef
  • Google Scholar

• 3

  DouwLKleinMFagelSSvan den HeuvelJTaphoornMJAaronsonNKet al. Cognitive and Radiological Effects of Radiotherapy in Patients With Low-Grade Glioma: Long-Term Follow-Up. Lancet Neurol (2009) 8(9):810–8. doi: 10.1016/S1474-4422(09)70204-2

  • CrossRef
  • Google Scholar

• 4

  BenjaminiYHochbergY. Controlling the False Discovery Rate: A Practical and Powerful Approach to Multiple Testing. J R Stat Society: Ser B (Methodological) (1995) 57(1):289–300. doi: 10.1111/j.2517-6161.1995.tb02031.x

  • CrossRef
  • Google Scholar

• 5

  RothAJKornblithABBatel-CopelLPeabodyEScherHIHollandJC. Rapid Screening for Psychologic Distress in Men With Prostate Carcinoma: A Pilot Study. Cancer (1998) 82(10):1904–8. doi: 10.1002/(sici)1097-0142(19980515)82:10<1904::aid-cncr13>3.0.co;2-x

  • CrossRef
  • Google Scholar

• 6

  ZigmondASSnaithRP. The Hospital Anxiety and Depression Scale. Acta Psychiatry Scand (1983) 67(6):361–70. doi: 10.1111/j.1600-0447.1983.tb09716.x

  • CrossRef
  • Google Scholar

• 7

  BeckATWardCHMendelsonMMockJErbaughJ. An Inventory for Measuring Depression. Arch Gen Psychiatry (1961) 4:561–71. doi: 10.1001/archpsyc.1961.01710120031004

  • CrossRef
  • Google Scholar

• 8

  WangY-PGorensteinC. Psychometric Properties of the Beck Depression Inventory-II: A Comprehensive Review. Braz J Psychiatry (2013) 35(4):416–31. doi: 10.1590/1516-4446-2012-1048

  • CrossRef
  • Google Scholar

• 9

  StrittmatterGTilkornMMawickR. How to Identify Patients in Need of Psychological Intervention. Recent Results Cancer Res (2002) 160:353–61. doi: 10.1007/978-3-642-59410-6_40

  • CrossRef
  • Google Scholar

• 10

  Marten-MittagBBookKBuchholdBDinkelAGründoblerBHenrichGet al. The Basic Documentation for Psycho-Oncology Short Form (PO-Bado SF)–An Expert Rating Scale for Distress Screening: Development and Psychometric Properties. Psychooncology (2015) 24(6):653–60. doi: 10.1002/pon.3708

  • CrossRef
  • Google Scholar

• 11

  CellaDFTulskyDSGrayGSarafianBLinnEBonomiAet al. The Functional Assessment of Cancer Therapy Scale: Development and Validation of the General Measure. J Clin Oncol (1993) 11(3):570–9. doi: 10.1200/JCO.1993.11.3.570

  • CrossRef
  • Google Scholar

• 12

  PickardASDe LeonMCKohlmannTCellaDRosenbloomS. Psychometric Comparison of the Standard EQ-5D to a 5 Level Version in Cancer Patients. Med Care (2007) 45(3):259–63. doi: 10.1097/01.mlr.0000254515.63841.81

  • CrossRef
  • Google Scholar

• 13

  MahoneyFIBarthelDW. Functional Evaluation: The Barthel Index. Md State Med J (1965) 14:61–5. doi: 10.1037/t02366-000

  • CrossRef
  • Google Scholar

• 14

  HütterBOGilsbachJM. Grundlagen Und Erste Ergebnisse Zur Methodischen Eignung Des Aachener Lebensqualitätsinventars [Background and First Results About Methodological Characteristics of the Aachen Life Quality Inventory]. Zentralbl Neurochir (2001) 62(2):37–42. German. doi: 10.1055/s-2002-19476

  • CrossRef
  • Google Scholar

• 15

  OsobaDAaronsonNKMullerMSneeuwKHsuMAYungWKet al. The Development and Psychometric Validation of a Brain Cancer Quality-of-Life Questionnaire for Use in Combination With General Cancer-Specific Questionnaires. Qual Life Res (1996) 5(1):139–50. doi: 10.1007/BF00435979

  • CrossRef
  • Google Scholar

• 16

  WareJESherbourneCD. The MOS 36-Item Short-Form Health Survey (SF-36). I. Conceptual Framework and Item Selection. Med Care (1992) 30(6):473–83. doi: 10.1097/00005650-199206000-00002

  • CrossRef
  • Google Scholar

• 17

  WareJEKosinskiMKellerSD. A 12-Item Short-Form Health Survey: Construction of Scales and Preliminary Tests of Reliability and Validity. Med Care (1996) 34(3):220–33. doi: 10.1097/00005650-199603000-00003

  • CrossRef
  • Google Scholar

• 18

  NorrisDRClarkMSShipleyS. The Mental Status Examination. AFP (2016) 94(8):635–41. doi: 10.4135/9781483392271.n315

  • CrossRef
  • Google Scholar

• 19

  KalbeEKesslerJCalabresePSmithRPassmoreAPBrandMet al. DemTect: A New, Sensitive Cognitive Screening Test to Support the Diagnosis of Mild Cognitive Impairment and Early Dementia. Int J Geriatr Psychiatry (2004) 19(2):136–43. doi: 10.1002/gps.1042

  • CrossRef
  • Google Scholar

• 20

  German Clinical Trials Register (DRKS). DRKS00010162 – Test-Retest-Reliability and Tumor-Localisation-Dependent Sensitivity of Neurocognitive Tests in Glioblastoma Patients. A Prospective Multicenter Study. NOA-19 (ReCog-GBM-L). Available at: https://www.drks.de/drks_web/navigate.do?navigationId=trial.HTML&TRIAL_ID=DRKS00010162 (Accessed April 27, 2021).

  • Google Scholar

• 21

  Weiß LucasCGoldbrunnerRHJungkCLoehrMRenovanzMNOA-19 Study Group. Towards an Optimised Clinical Neuropsychological Assessment for Glioblastoma Patients – Concept and Interim Analysis of a Prospective Multi-Centre Study. 71st Annual Meeting of the German Society for Neurosurgery (DGNC). Sine Loco [Digital], June 21-24, 2020 (2020). Duesseldorf: German Medical Science GMS Publishing House. Available at: https://www.egms.de/static/de/meetings/dgnc2020/20dgnc127.shtml (Accessed April 27, 2021). doi: 10.3205/20dgnc127

  • CrossRef
  • Google Scholar

• 22

  NasreddineZSPhillipsNABédirianVCharbonneauSWhiteheadVCollinIet al. The Montreal Cognitive Assessment, MoCA: A Brief Screening Tool for Mild Cognitive Impairment. J Am Geriatr Soc (2005) 53(4):695–9. doi: 10.1111/j.1532-5415.2005.53221.x. Erratum in: J Am Geriatr Soc. 2019 Sep;67(9):1991. doi: 10.1111/jgs.15925.

  • CrossRef
  • Google Scholar

• 23

  GoebelSMehdornHM. Development of Anxiety and Depression in Patients With Benign Intracranial Meningiomas: A Prospective Long-Term Study. Support Care Cancer (2013) 21(5):1365–72. doi: 10.1007/s00520-012-1675-5

  • CrossRef
  • Google Scholar

• 24

  LitofskyNSFaraceEAndersonFMeyersCAHuangWLawsERet al. Depression in Patients With High-Grade Glioma: Results of the Glioma Outcomes Project. Neurosurgery (2004) 54(2):358–66; discussion 366-367. doi: 10.1227/01.neu.0000103450.94724.a2

  • CrossRef
  • Google Scholar

• 25

  MainioAHakkoHNiemeläAKoivukangasJRäsänenP. Depression in Relation to Anxiety, Obsessionality and Phobia Among Neurosurgical Patients With a Primary Brain Tumor: A 1-Year Follow-Up Study. Clin Neurol Neurosurg (2011) 113(8):649–53. doi: 10.1016/j.clineuro.2011.05.006

  • CrossRef
  • Google Scholar

• 26

  RappMSchipmannSHoffmannKWiewrodtRSteigerH-JKampMet al. Impact of Distress Screening Algorithm for Psycho-Oncological Needs in Neurosurgical Patients. Oncotarget (2018) 9(60):31650–63. doi: 10.18632/oncotarget.25833

  • CrossRef
  • Google Scholar

• 27

  MitchellAJLordKSlatteryJGraingerLSymondsP. How Feasible Is Implementation of Distress Screening by Cancer Clinicians in Routine Clinical Care? Cancer (2012) 118(24):6260–9. doi: 10.1002/cncr.27648

  • CrossRef
  • Google Scholar

• 28

  BaumstarckKChinotOTabouretEFarinaPBarriéMCampelloCet al. Coping Strategies and Quality of Life: A Longitudinal Study of High-Grade Glioma Patient-Caregiver Dyads. Health Qual Life Outcomes (2018) 16(1):157. doi: 10.1186/s12955-018-0983-y

  • CrossRef
  • Google Scholar

• 29

  PiilKJakobsenJChristensenKBJuhlerMGuettermanTCFettersMDet al. Needs and Preferences Among Patients With High-Grade Glioma and Their Caregivers - A Longitudinal Mixed Methods Study. Eur J Cancer Care (Engl) (2018) 27(2):e12806. doi: 10.1111/ecc.12806

  • CrossRef
  • Google Scholar

• 30

  BédardMMolloyDWSquireLDuboisSLeverJAO’DonnellM. The Zarit Burden Interview: A New Short Version and Screening Version. Gerontologist (2001) 41(5):652–7. doi: 10.1093/geront/41.5.652

  • CrossRef
  • Google Scholar

• 31

  Onkozert. Jahresbericht [Annual Report] (2015). Available at: https://onkozert.de/wordpress/wp-content/uploads/2018/07/noz_jahresbericht-2015-A3(160721).pdf?v=3698282321. German (Accessed April 26, 2021).

  • Google Scholar

• 32

  Onkozert. Jahresbericht [Annual Report] (2020). Available at: https://www.onkozert.de/wordpress/wp-content/uploads/2020/09/ncc_annualreport-2020-A1_200728.pdf?v=89180600 (Accessed April 26, 2021).

  • Google Scholar

• 33

  RooneyAGCarsonAGrantR. Depression in Cerebral Glioma Patients: A Systematic Review of Observational Studies. JNCI: J Natl Cancer Institute (2011) 103(1):61–76. doi: 10.1093/jnci/djq458

  • CrossRef
  • Google Scholar

• 34

  DGPPNBÄKKBVAWMF. S3 Guideline Unipolar Depression – Long Version, 2nd Edition [S3-Leitlinie/Nationale VersorgungsLeitlinie Unipolare Depression - Langfassung, 2. Auflage] (2015). Available at: https://www.awmf.org/uploads/tx_szleitlinien/nvl-005l_S3_Unipolare_Depression_2017-05.pdf (Accessed April 27, 2021).

  • Google Scholar

• 35

  RenovanzMTsakmaklisHSoebiantoSNeppelINadji-OhlMBeutelMet al. Screening for Distress in Patients With Intracranial Tumors During the First 6 Months After Diagnosis Using Self-Reporting Instruments and an Expert Rating Scale (The Basic Documentation for Psycho-Oncology Short Form - PO-Bado Sf). Oncotarget (2018) 9(57):31133–45. doi: 10.18632/oncotarget.25763

  • CrossRef
  • Google Scholar

• 36

  KeirSTCalhoun-EaganRDSwartzJJSalehOAFriedmanHS. Screening for Distress in Patients With Brain Cancer Using the NCCN’s Rapid Screening Measure. Psychooncology (2008) 17(6):621–5. doi: 10.1002/pon.1271

  • CrossRef
  • Google Scholar

• 37

  GoebelSMehdornHM. Measurement of Psychological Distress in Patients With Intracranial Tumours: The NCCN Distress Thermometer. J Neurooncol (2011) 104(1):357–64. doi: 10.1007/s11060-010-0501-5

  • CrossRef
  • Google Scholar

• 38

  GoebelSKnuthCDammMLindenDCoburgerJRingelFet al. Towards the Targeted Assessment of Relevant Problems: Optimization of the Distress Thermometer for Adult Neuro-Oncological Patients. Psycho-Oncology (2020) 29(12):2057–66. doi: 10.1002/pon.5564

  • CrossRef
  • Google Scholar

• 39

  GuanBWangKShaoYChengXHaoJTianCet al. The Use of Distress Thermometer in Advanced Cancer Inpatients With Pain. Psychooncology (2019) 28(5):1004–10. doi: 10.1002/pon.5032

  • CrossRef
  • Google Scholar

• 40

  EORTC - Quality of Life. Brain (Update of QLQ-Bn20). Available at: https://qol.eortc.org/questionnaire/bn20-update (Accessed April 27, 2021).

  • Google Scholar

• 41

  EORTC - Quality of Life. Item Library. Available at: https://qol.eortc.org/item-library (Accessed April 27, 2021).

  • Google Scholar

• 42

  PetersenMAAaronsonNKArrarasJIChieW-CConroyTCostantiniAet al. The EORTC CAT Core-The Computer Adaptive Version of the EORTC QLQ-C30 Questionnaire. Eur J Cancer (2018) 100:8–16. doi: 10.1016/j.ejca.2018.04.016

  • CrossRef
  • Google Scholar

• 43

  PetersenMAAaronsonNKConroyTCostantiniAGiesingerJMHammerlidEet al. International Validation of the EORTC CAT Core: A New Adaptive Instrument for Measuring Core Quality of Life Domains in Cancer. Qual Life Res (2020) 29(5):1405–17. doi: 10.1007/s11136-020-02421-9

  • CrossRef
  • Google Scholar

• 44

  MauerMStuppRTaphoornMJBCoensCOsobaDMarosiCet al. The Prognostic Value of Health-Related Quality-of-Life Data in Predicting Survival in Glioblastoma Cancer Patients: Results From an International Randomised Phase III EORTC Brain Tumour and Radiation Oncology Groups, and NCIC Clinical Trials Group Study. Br J Cancer (2007) 97(3):302–7. doi: 10.1038/sj.bjc.6603876

  • CrossRef
  • Google Scholar

• 45

  StuppRMasonWPvan den BentMJWellerMFisherBTaphoornMJBet al. Radiotherapy Plus Concomitant and Adjuvant Temozolomide for Glioblastoma. New Engl J Med (2005) 352(10):987–96. doi: 10.1056/NEJMoa043330

  • CrossRef
  • Google Scholar

• 46

  ArmstrongTSDirvenLAronsDBatesAChangSMCoensCet al. Glioma Patient-Reported Outcome Assessment in Clinical Care and Research: A Response Assessment in Neuro-Oncology Collaborative Report. Lancet Oncol (2020) 21(2):e97–103. doi: 10.1016/S1470-2045(19)30796-X

  • CrossRef
  • Google Scholar

• 47

  TaphoornMJBSizooEMBottomleyA. Review on Quality of Life Issues in Patients With Primary Brain Tumors. Oncologist (2010) 15(6):618–26. doi: 10.1634/theoncologist.2009-0291

  • CrossRef
  • Google Scholar

• 48

  DiRisioACHararyMvan WestrhenenANassrEErmakovaASmithTRet al. Quality of Reporting and Assessment of Patient-Reported Health-Related Quality of Life in Patients With Brain Metastases: A Systematic Review. Neurooncol Pract (2018) 5(4):214–22. doi: 10.1093/nop/npy024

  • CrossRef
  • Google Scholar

• 49

  BuneviciusAMillerJParsonsM. Isocitrate Dehydrogenase, Patient-Reported Outcomes, and Cognitive Functioning of Glioma Patients: A Systematic Review. Curr Oncol Rep (2020) 22(12):120. doi: 10.1007/s11912-020-00978-9

  • CrossRef
  • Google Scholar

• 50

  RobinsonLTangETaylorJ-P. Dementia: Timely Diagnosis and Early Intervention. BMJ (2015) 350:h3029. doi: 10.1136/bmj.h3029

  • CrossRef
  • Google Scholar

• 51

  OlsonRAChhanabhaiTMcKenzieM. Feasibility Study of the Montreal Cognitive Assessment (MoCA) in Patients With Brain Metastases. Support Care Cancer (2008) 16(11):1273–8. doi: 10.1007/s00520-008-0431-3

  • CrossRef
  • Google Scholar

• 52

  DurandTBernierM-OLégerITailliaHNoëlGPsimarasDet al. Cognitive Outcome After Radiotherapy in Brain Tumor. Curr Opin Oncol (2015) 27(6):510–5. doi: 10.1097/CCO.0000000000000227

  • CrossRef
  • Google Scholar

• 53

  LiJBentzenSMLiJRenschlerMMehtaMP. Relationship Between Neurocognitive Function and Quality of Life After Whole-Brain Radiotherapy in Patients With Brain Metastasis. Int J Radiat Oncol Biology Physics (2008) 71(1):64–70. doi: 10.1016/j.ijrobp.2007.09.059

  • CrossRef
  • Google Scholar

• 54

  MeyersCAHessKR. Multifaceted End Points in Brain Tumor Clinical Trials: Cognitive Deterioration Precedes MRI Progression. Neuro Oncol (2003) 5(2):89–95. doi: 10.1093/neuonc/5.2.89

  • CrossRef
  • Google Scholar

• 55

  DanielsTBBrownPDFeltenSJWuWBucknerJCArusellRMet al. Validation of EORTC Prognostic Factors for Adults With Low-Grade Glioma: A Report Using Intergroup 86-72-51. Int J Radiat Oncol Biol Phys (2011) 81(1):218–24. doi: 10.1016/j.ijrobp.2010.05.003

  • CrossRef
  • Google Scholar

• 56

  TemelJSGreerJAMuzikanskyAGallagherERAdmaneSJacksonVAet al. Early Palliative Care for Patients With Metastatic Non–Small-Cell Lung Cancer. New Engl J Med (2010) 363(8):733–42. doi: 10.1056/NEJMoa1000678

  • CrossRef
  • Google Scholar

• 57

  FerrellBRTemelJSTeminSAlesiERBalboniTABaschEMet al. Integration of Palliative Care Into Standard Oncology Care: American Society of Clinical Oncology Clinical Practice Guideline Update. J Clin Oncol (2017) 35(1):96–112. doi: 10.1200/JCO.2016.70.1474

  • CrossRef
  • Google Scholar

• 58

  FritzLDirvenLReijneveldJCKoekkoekJAFStiggelboutAMPasmanHRWet al. Advance Care Planning in Glioblastoma Patients. Cancers (Basel) (2016) 8(11):102. doi: 10.3390/cancers8110102

  • CrossRef
  • Google Scholar

• 59

  FritzLZwinkelsHKoekkoekJAFReijneveldJCVosMJDirvenLet al. Advance Care Planning in Glioblastoma Patients: Development of a Disease-Specific ACP Program. Support Care Cancer (2020) 28(3):1315–24. doi: 10.1007/s00520-019-04916-9

  • CrossRef
  • Google Scholar

• 60

  StaartjesVEStumpoVKernbachJMKlukowskaAMGadjradjPSSchröderMLet al. Machine Learning in Neurosurgery: A Global Survey. Acta Neurochir (Wien) (2020) 162(12):3081–91. doi: 10.1007/s00701-020-04532-1

  • CrossRef
  • Google Scholar

• 61

  Vaz SalgadoMATorresJEstebanJGutiérrezJALeyLCarratoA. Survey of Treatment Recommendations for Elderly Patients With Glioblastoma. Clin Transl Oncol (2020) 22(8):1329–34. doi: 10.1007/s12094-019-02260-2

  • CrossRef
  • Google Scholar

• 62

  ArzoineJLevéCPérez-HickAGooddenJAlmairacFAubrunSet al. Anesthesia Management for Low-Grade Glioma Awake Surgery: A European Low-Grade Glioma Network Survey. Acta Neurochir (Wien) (2020) 162(7):1701–7. doi: 10.1007/s00701-020-04274-0

  • CrossRef
  • Google Scholar