AUTHOR=Kenda Martin , Cheng Zhuo , Guettler Christopher , Storm Christian , Ploner Christoph J. , Leithner Christoph , Scheel Michael 

TITLE=Inter-rater agreement between humans and computer in quantitative assessment of computed tomography after cardiac arrest

JOURNAL=Frontiers in Neurology

VOLUME=Volume 13 - 2022

YEAR=2022

URL=https://www.frontiersin.org/journals/neurology/articles/10.3389/fneur.2022.990208

DOI=10.3389/fneur.2022.990208

ISSN=1664-2295

ABSTRACT=Background
Quantitative assessment of head computed tomography (CT) is used to predict neurological outcome after cardiac arrest (CA). The current reference standard is manual analysis of brain regions by an experienced physician. Recently, automated analysis methods have been suggested. There is limited data on the interrater agreement of CT quantification after cardiac arrest. 
Methods
Three human raters with different levels of clinical experience retrospectively assessed the Gray-White-Matter Ratio (GWR) in the head CTs of 95 cardiac arrest patients. GWR was also quantified by a recently published computer algorithm. All raters were blinded to clinical information and outcome. We calculated intraclass correlation (ICC) for interrater agreement on GWR and its underlying regions of interest (ROIs) as well as prognostic performance measures. 
Results
Interrater agreement on GWR was very good (ICC 0.82-0.84) between all three human raters across different levels of expertise and between the computer algorithm and neuroradiologist (ICC 0.83; 95%-CI 0.78-0.88). Despite high overall agreement, considerable deviations in GWR between raters were observed in individual patients. In our cohort, this did not lead to any false poor neurological outcome prediction but a few false good neurological outcome predictions. 
Conclusion
Even though human and computer raters mostly came to the same conclusion in GWR assessment, physicians should be aware that interrater variability could potentially impact prognostic ability and lead to a false prognosis. This underscores the need for further standardization of quantitative CT analysis after cardiac arrest as well as multimodal diagnostics.