The ins and outs of errors in oncology imaging: the DAC framework for radiologists

Abstract

With the increasingly central role of imaging in medical diagnosis, understanding and monitoring radiological errors has become essential. In the field of oncology, the severity of the disease makes radiological error more visible, with both individual consequences and public health issues. The quantitative trend radiology allows to consider the diagnostic task as a problem of classification supported by the latest neurocognitive theories in explaining decision making errors, this purposeful model provides an actionable framework to support root cause analysis of diagnostic errors in radiology and envision corresponding risk-management strategies. The D for Data, A for Analysis and C for Communication are the three drivers of errors and we propose a practical toolbox for our colleagues to prevent individual and systemic sources of error.

1 Introduction

According to an annual complaints report for radiologists in France, “medical error” affected 1 in 20 radiologists. A similar US analysis of complaints analysis filed over one decade showed that oncology errors accounted for the largest proportion accounting from 40% up to 80% of complaints when focusing on complaints with high harm (1, 2).

In this context, it is interesting to explore the mechanisms of “radiological error” by looking more specifically at this most represented field, i.e., cancer diagnostic radiology Mammography and computed tomography were the most concerned modalities probably due to the breast cancer screenings over the world and widespread use of CT scans for generic whole-body analysis (3). Ultrasound, even though it is frequently indicated as a first line examination was not predominantly concerned maybe due to the impossible retrospective analysis unlike other acquired modalities that can be objectively reassessed by a second opinion.

Previous frameworks for root-causing radiological errors have been proposed and the most used classification was published in 2014 by Kim and Mansfield (4). However, its applicability in terms of risk management is not straightforward. Radiological errors being primarily human errors, we propose to update a classification in accordance with the latest neurocognitive knowledge to regroup by underlying concepts and mechanics.

In this paper, firstly, we explore the terminologies of error in experimental medicine together with the current critical place of quantitative imaging in oncology. Then, we detail cognitive mechanisms of decision-making in radiology as applied to oncology to propose an updated error classification supported by signal detection theory. Finally, we detail strategies for managing the risk of error at both the individual and systemic level.

2 The” experimental” medicine and the concept of biomarkers

2.1 The “error” terminology

The terminology related to “error” is deeply embedded in the history of experimental and quantitative medicine. This terminology is critical for understanding scientific literature and for comparing results across different studies (Figure 1).

Figure 1

However, depending on the field -whether in statistics or forensic medicine- the same terms can carry different meanings, often leading to confusion. A frequent source of this confusion is the distinction between “error” and “variability (5, 6). This issue dates back to the early days of quantitative experimentation, which were influenced by Mendel’s mechanistic approach to biology and his application of mathematical principles.

In the context of image analysis, variability refers to differences in interpretation between multiple observers. This is typically measured through indices that reflect the level of agreement or discordance among radiologists. These indices are essential for evaluating the reliability of diagnostic decisions (7).

Since error implies that what is correct is known, the notion of “truth” is inseparable from the notion to which error refers. In the era of quantitative medicine, we have inherited the term “ground truth” or “gold standard”, which is defined by empirical evidence, i.e., information given by direct sensory or experimental observation that is known to be true. This idea of truth aligns with the philosophical notion of “a posteriori knowledge,” knowledge that is based on experience, as opposed to “a priori knowledge,” which is derived from reasoning alone (8).

The relationship between error and experience is fundamental to evidence-based medicine, where decisions are grounded in both experimental results and practical experience (9).

In legal contexts, the term “fault” is used as a type of error, but with a critical distinction: an error is considered a fault only when the reference of the correctness is indisputably known in advance. In radiology, this typically means adherence to established best practices (10, 11).

In statistical and cognitive science, we encounter the term “bias”, which is a systematic cause of error that is mostly unconscious or invisible and a priori. A cognitive bias specifically refers to errors in thinking caused by a distortion in how information is processed.

While these terms -error, variability, fault, and bias- are distinct, they are also interrelated. Figure 1 visually represents the relationships between these concepts and their importance across medical, statistical, and cognitive domains.

2.2 Quantitative imaging as a decision-making tool in cancerology

Medical imaging is a decision-making tool for the cancer patient that not only has to be performant but must also be used properly and at the right time to be efficient.

2.2.1 Imaging biomarkers

The term “imaging biomarker” defines discriminating information contained in a medical image (12, 13). This information is measured by imaging modality and must satisfy characteristics that allow it to be used for decision-making, i.e., accuracy, precision, reliability, and relevance (14). The first three characteristics are also quantifiable. In addition, when studying “error” in radiology, it is essential to continually evaluate the source of information for its relevance and biological plausibility within the clinical context.

2.2.2 Oncology and surrogate endpoints

The oncologic context provides a good example of “a priori discriminating” use of biomarkers (Figure 2). Their use differs depending on cancer stage. It is either a diagnostic question in a symptomatic or asymptomatic patient (i.e., during screening), or a question of the cancer treatment’s efficacy (i.e., during follow-up).

Figure 2

In both cases, the ground truth can only be known a posteriori since it corresponds to histology or surgery (for disease diagnosis), or death (for treatment efficacy).

Oncology imaging biomarkers are “surrogates” endpoints when they can be used instead of clinical outcomes. Indeed, invasive diagnosis is morbid, and death is fatal, therefore imaging biomarker-assisted decision-making attempts to predict disease or treatment ineffectiveness as early as possible with minimal error. The validity of such surrogates is regulatory and based on previously acquired evidence (15).

3 Cognitive decision model in cancer imaging

The decision step turns a virtual error into a real, and potentially harmful entity. Therefore, there is a notion of risk in making a decision and since the impact is substantial when considering oncology imaging, the stakes of this risk are higher.

To understand errors, it is essential to understand how both the tools (i.e., quantitative imaging biomarkers) and context of the decision-making (surrogates) described above are integrated by the individual operator to take his decision.

Modern cognitive science theories allow us to better understand the brain mechanism underlying decision making in these situations of uncertainty.

“Signal theory” and the Bayesian model represent the state-of-the-art of understanding of the decision mechanism, from the sensorial information to the decisional action (16, 17).

To best illustrate schematically the mental stages during the radiological diagnosis process, we consider the mammographic screening use case (Figure 3).

Figure 3

To address this task, the mental activity asks two sub questions which, for the sake of understanding, are represented sequentially but are in fact intertwined i.e.:

3.1 “Given what I see on the mammogram, how plausible is the hypothesis that my patient has cancer?”

In a probabilistic Bayesian model, this “plausibility” is denoted by p(H|D) and corresponds to the probability of having cancer (H) knowing the mammographic information (D). Conversely, the “a posteriori probability of having this mammographic data knowing the hypothesis”, is noted p(D|H) and corresponds to the “likelihood” of the data.

The law of probabilities shows that the plausibility of a given hypothesis is a function of the likelihood but also the “a priori probability” of the hypothesis noted p(H). This so-called “prior” contextualizes the decision making and greatly modifies the outcome. The law is written as:

It can be well understood by general radiologists that have experienced lung parenchymal findings on a coughing patient with a known evolutive cancer. They know that this may correspond to either a metastasis or an inflammation suggesting that the likelihood of the image alone is not discriminating enough. It is the a priori knowledge that allows us to conclude about plausibility (18).

In our practice, this “prior” corresponds to theoretical and experiential knowledge, to the context of the analysis and to the instructions for performing the task (i.e., reading the images) that is related to it. In oncology, for example, the latter read rules correspond to the BIRADS analysis for the mammography or RECIST criteria for an oncologic follow-up. Those rules are framing the context of the task (19).

Also, in processing this first sub question, there are two irreducible variability factors to consider, one concerning the data i.e., imaging biomarker and its signal/noise ratio (external noise), the second concerning the performance of the radiologist observer (internal sensory noise) which is equivalent to a “detector” intrinsic performance (20). The latest participates both in inter-observer variability and intra-observer variability as it varies for a same individual according to mentally perturbating stimuli (e.g., tiredness, stress, external stimuli).

3.2 Given the purpose of the mammography exam, what criterion maximizes its gain?

Signal theory suggests that the observer applies a gain maximization function to set the criterion according to the desired goal. Indeed, in radiology and medicine in general, unfortunately there is often overlap between positive images corresponding to the ill state and so-called negative images corresponding to the non-ill state with a normal distribution of patients. This phenomenon is expressed simply by contingency tables commonly used in medicine to evaluate diagnostic test effectiveness (Figure 4).

Figure 4

In oncology, the physician is torn between a conservative objective (i.e., avoidance of biopsies and death at the cost of missed cancers), and a non-conservative objective (i.e., not missing any cancer, even if invasive diagnostic procedures are required). The first case minimizes false positives (statistical type I error) and the second case minimizes false negatives (statistical type II error).

The decision criterion is determined to maximize the gains according to the objective that the radiologist has set. It is therefore variable according to the individual and the context.

4 The DAC classification of radiological errors

The Kim and Mansfield classification is the most widely accepted classification for error types in radiology (4). Based on a retrospective evaluation, this classification determined 12 error types according to their cause. We feel that this classification is imperfect because it does not fully cover the chronological nature of the causality principle.

We therefore propose a simpler error classification with a simpler approach that can be illustrated with the information flow (Figure 5 and Table 1).

Figure 5

This higher-level classification advantageously integrates the previously detailed processes of “information to decision” through the acquisition of information (D for data), its analysis using cognition (A for analysis) and the communication of this analysis (C for communication). We separate errors according to this DAC approach:

• Error type D: Data or meta-data related errors

• Error type A: Analysis through cognition related errors

• Error type C: Communication of diagnostic results related errors

4.1 Error related to the data (Type D)

The first step of a radiological examination in oncology is to collect the information necessary to understand and frame the problem. It is important not to corrupt this information processing. Unfortunately, as every radiologist knows, in our practice several differential diagnoses share the same image findings (18).

Clinical information is useful to contextualize visual analysis and this meta-information improves the knowledge of what was previously named as the “prior”. This type of information is furthermore necessary as it can indirectly influence the image information when used to adapt the acquisition protocol and field of view. We also mentioned above that it influences the maximization criteria. This step of information processing is therefore critical. Unfortunately, appropriateness of imaging referral is sometimes not justified with impact on unnecessary or wrong examination (21).

In practice, meta-information is often available if the radiology team makes the necessary effort to retrieve it from patients, doctors, and family (Figure 6). Indeed, for our field of interest, it is crucial to make comparisons with historical examinations, to know the surgical interventions already performed and to question the patient if an image is confusing (Figure 7).

Figure 6

Figure 7

In everyday practice, it should be remembered that this type of error would be considered as a fault error since the radiologist does not have an obligation of result but rather an obligation of means. All necessary means must therefore be put in place to recover all the required information for interpretation, even if this is never easy.

The centralized clinical trial illustrates the importance of “prior” information to contextualize image analysis. Indeed, the censorship of some of the site information during blinded independent review often led to mistakes in the selection of targets for RECIST assessment with well-known benign lesion in the liver suspected to be a metastasis (22, 23).

Ultimately, the pixel-information once the examination is performed might also be corrupted if the technic of the examination was not performed correctly. It is important to provide the best technic of acquisition to avoid confusing artefacts for the analysis. If one is not satisfied with the technic of an examination, it should be reperformed with the scope to help a better-informed decision.

4.2 Error related to the analysis (Type A)

This type of error is also named “cognitive error” as it is linked to the cognition framework described previously (Figures 3, 4).

Radiological interpretation corresponds schematically to a visual search task for significant abnormalities in one or several medical images. For didactic purposes, the task can again be broken down chronologically into two steps: detection then characterization (24).

Under-performance can either be a result of under-detection or a properly detected, but misinterpreted finding (25).

4.2.1 Under-detection (Type A1)

During detection, the two determining factors are the visibility of the lesions and the radiologist’s detector performance:

• Concerning the visual stimulus associated with the lesion, its visibility can be estimated by the signal-to-noise ratio. In addition, its cognitive integration can be conducted through two processes:

  • Bottom-up processes depend almost entirely on the information perceived, and therefore little on the assumptions or expectations of the perceiving radiologist.

  • Top-down processes based on integration of the previously learned information on this perceptual information. They are high-level cognitive processes and control the sensory information from knowledge and experience.

• Regarding the performance of the human detector, this depends on the analysis method and the level of attention.

  • The visual analysis strategy varies between radiologists (Figure 8) depending on their knowledge and experience. It has been described that there are significant differences in the exploration of a CT volume or mammograms between radiologists and that these differences correlate with different detection efficiencies (26, 27).

  • Attention to the task is also a factor contributing to variability. It has been documented that radiologists may miss abnormalities that are visible retrospectively, either because of a drop in attention level, or because of a shift in attention. Cognitive biases can interfere with the radiologist’s attention and lead to non-detection errors.

    • ▪ “Satisfaction bias” is well-documented in radiology (28). It refers to a drop in attention after the discovery of an abnormality. Hence, it is responsible for the non-detection of additional abnormalities. In oncology, this is exemplified by detection of one pathological finding but miss detection of multifocal lesions.

    • ▪ “Inattentional blindness” bias is also documented in radiology. It refers to attention locked in a top-down process that prevents the detection of unexpected anomalies (Figure 9). This phenomenon is illustrated by the popular article entitled “The invisible gorilla strikes again[ … ]” and others have reproduced the phenomenon (29, 30).

Figure 8

Figure 9

4.2.2 Misinterpretation (Type A2)

The characterization process involves the radiologist’s judgment. Once a finding is detected, the question of diagnosis arises. Misinterpretation is rarely due to the responsible radiologists’ lack of knowledge. In documented series, this represents around 10% of errors (4, 24, 25). Frequently, it is to do with the functioning of the brain, that uses a heuristic strategy for information processing. These heuristics reduce the brain’s workload at the cost of systematic errors. These cognitive biases are widely documented in radiology and oncology, with anchoring, confirmation and availability biases most frequently encountered (31).

Oncological follow-up is vulnerable to anchoring bias when the analysis is conducted in a sequential manner. In such a setting, we observe that the radiologists tend to confirm the previous measurement result while this measurement operation is reputed to be non-subjective (Figure 10).

Figure 10

4.3 Error related to the communication of report (Type C)

In oncology, radiological evaluation is central for diagnosis and treatment decisions. Tumor follow-up criteria are now used as a decision-making tool not only for clinical trials, but also for routine use.

Communication of results is often delicate because it is aimed at both the patient and the physician, sometimes with a slightly different objective.

The communication medium is the written report. In routine, this error is difficult to trace and the Kim and Mansfield analysis probably underestimates its frequency (4). However, in clinical trials, non-conformity of reports is documented as a frequent deviation with 55% (32).

These type C errors are no less impactful and lead to bad decisions because of incomplete, false, or misunderstood information (Figure 11) (11).

Figure 11

5 Discussion

Radiology errors are addressed to prevent occurrence of adverse events. Since error is statistically embedded in the above predictive decision-making model, the objective is to minimize the individual related sources of error previously outlined.

However, it is necessary to consider radiological interpretation in its global environment to differentiate the control strategy for the individual’s error from a systemic approach focusing on conditions and factors acting on this individual (Figure 12) (33).

Figure 12

According to the pareto principle, in relation to the 3 classes of errors described above, we can propose a toolbox trying to address the most frequent contributors (Table 2). Cognitive type A errors dominate and represent up to 80% in reported series (4, 25).

Preventing cognitive errors typically requires the use of debiasing strategies, which aim to either prevent or correct the initial judgment, often formed through heuristics However, corrective strategies face practical limitations, as they require either revisiting and re-evaluating one’s initial decision or relying on a concurrent external opinion to provide a counterbalance. Both approaches are time-intensive and frequently infeasible in high-pressure environments like clinical practice, where time constraints may not allow for thorough reconsideration. Moreover, merely being “aware” of cognitive biases does not necessarily reduce their impact after they have already occurred. Research has shown that awareness alone often fails to mitigate cognitive errors because biases are deeply ingrained in our cognitive processes and operate subconsciously. Additionally, measuring individual susceptibility to cognitive biases is still an area of ongoing research (34, 35).

Then, even if type D errors are less frequent, the preventive action might be more effective in making all necessary efforts to collect the best clinical information and pixel information.

Also, regular training helps to reduce the type A errors linked to lack of knowledge errors and might also improve the quality of communication (6, 36). Large literature insists from several years on the necessity to improves the quality of reporting by improving communication skills of radiologists and using several tools such as structured reports and multidisciplinary standardized disease lexicon and classifications (37, 38).

Sources of error are multifactorial and dependent on the radiologist’s environment (39). We can consider primary prevention to reduce systemic risk factors and secondary or tertiary prevention for early correction of potential radiologist errors (5) (Table 3).

We initially illustrated the cognitive feedback loop of the mistakes to create knowledge for future diagnostics. To make it possible, the working conditions should promote positive communication regarding errors, which are in fact an opportunity for improvement. Sharing errors during a regular staff meeting or peer review is a good way to dramatize the error in radiology (40, 41). However, these facilities are resource intensive, jeopardizing their feasibility (42).

Poor working conditions have been established as a risk factor for interpretation errors by several authors (43).

The number of images per minute read by radiologists has increased by 7 in this age of hyper-efficiency, driven by the digitalization that occurred in the 2000’s (43, 44). In the event of overwork, the risk of error for the radiologist increases. It has been estimated that a 2-fold increase in examination rate increases the risk of omission errors by 25% (45). We conducted a survey of 35 radiologists in the south of France, and found that 80% of radiologists interpreted >20 CT scans per 4-hour shift; some authors have shown that there is a significant increase in errors when performing more than 20 CAP-CT scans per day (46).

More than 2/3 of errors are caused by cognitive attention bias. It is reported that interconnectivity leads to multiple interruptions in the workflow that affect the radiologist’s attention (47). This multitasking distracts the radiologist, increasing the risk of error. Some authors have shown that radiologists can be interrupted every 4 to 12 minutes.

Workflow in oncology is essential because radiological analysis consists of comparing and measuring lesions repeatedly. Previous measurements must be easily available at the time of analysis by optimized equipment. Moreover, the measurement step is time consuming, and some authors propose a hybrid workflow after the baseline measurement to decrease the examination time for the radiologist without loss of quality (32). The same automatic computation for image post-processing analysis (mainly measurements) in any radiological field could greatly help to reduce workload therefore indirectly reduce attention bias linked to these mentally consuming tasks.

Furthermore, the negative predictive value of artificial intelligence could potentially read the content of images and propose a prioritization of patients with significant radiological abnormalities, allowing more time and attention to be spent on these at-risk patients compared to others, but this means to qualify and build trust into AI-triage systems (48, 49).

It is generally accepted that “two brains are better than one”. An important fact to keep in mind is the importance of the communication of one’s confidence level with second readers as this seems to be a determinant in the application of this adage (50). The second radiological opinion has been shown to be effective in several studies and this paradigm is used in centralized independent imaging readings for clinical trials (23, 51).

The second opinion can also come from a machine. The developments of artificial intelligence in detection and characterization should allow it to compete with a radiologist’s readings in the future.

Also, specifically in oncology, “tumor tracking” software can integrate the analysis rules for follow-up criteria and enable prevention of non-compliance errors during the analysis time i.e., number of targets, minimum size.

More generally, awareness about the risk control necessity to prevent errors is promoted through good practices of quality management. Imaging departments should start to implement quality assurance standards helping them to detect and correct the risks of errors (7).

Audit of their working condition and performance should also benefit to reduce individual errors by unmasking such correctable environmental risk factors (52).

6 Conclusion

The errors detected are only the tip of the iceberg as many of them will not have a significant enough impact to trigger a complaint. Oncology, which deals with a serious disease and regular examinations, is the indication that provides the best insight into the ins and outs of medical imaging errors.

For the sake of understanding, we proposed a threefold classification of mechanisms of error related to the information (D), the cognition (A), and the reporting (C).

However, it is important to understand that even if we tried to systematize it, the error in radiology partially escapes this systematization because it occurs in a complex and non-deterministic world. The DAC classification describes an over-simplified model, still it offers a practical means for risk management to identify and operate on drivers of errors.

The individual factors of errors are dominated by cognitive bias, but debiasing strategies seems more feasible through environmental drivers. The toolbox that we provide are generalist and non-exhaustive. At the individual level, raising awareness of preventable errors and adopting a non-blaming behavior will help to move towards quality driven practices in radiology with benefit from a sort of collective intelligence thanks to more sharing of errors and experiences. AI-machines are a hot topic of discussion regarding error with high promises addressing the quality of care more than the reduction of the radiologist’s workload (53). Humans will still be in the loop for a while and error management will not disappear soon.

References

• 1

  HarveyHBTomovEBabayanADwyerKBolandSPandharipandePVet al. Radiology malpractice claims in the United States from 2008 to 2012: characteristics and implications. J Am Coll Radiol. (2016) 13:124–30. doi: 10.1016/j.jacr.2015.07.013

  • CrossRef
  • Google Scholar

• 2

  RosenkrantzABSiegalDSkillingsJAMuellnerANassSJHricakH. Oncologic errors in diagnostic radiology: A 10-year analysis based on medical malpractice claims. J Am Coll Radiol. (2021) 18:1310–6. doi: 10.1016/j.jacr.2021.05.001

  • CrossRef
  • Google Scholar

• 3

  GoergenSSchultzTDeakinARuncimanW. Investigating errors in medical imaging: lessons for practice from medicolegal closed claims. J Am Coll Radiol. (2015) 12:988–97. doi: 10.1016/j.jacr.2015.03.025

  • CrossRef
  • Google Scholar

• 4

  KimYWMansfieldLT. Fool me twice: delayed diagnoses in radiology with emphasis on perpetuated errors. AJR Am J Roentgenol. (2014) 202:465–70. doi: 10.2214/AJR.13.11493

  • CrossRef
  • Google Scholar

• 5

  BrunoMA. 256 Shades of gray: uncertainty and diagnostic error in radiology. Diagnosis (Berl). (2017) 4:149–57. doi: 10.1515/dx-2017-0006

  • CrossRef
  • Google Scholar

• 6

  BradyAP. Error and discrepancy in radiology: inevitable or avoidable? Insights Imaging. (2017) 8:171–82. doi: 10.1007/s13244-016-0534-1

  • CrossRef
  • Google Scholar

• 7

  AlpertHRHillmanBJ. Quality and variability in diagnostic radiology. J Am Coll Radiol. (2004) 1:127–32. doi: 10.1016/j.jacr.2003.11.001

  • CrossRef
  • Google Scholar

• 8

  DemarestBvan den BergH. Kant’s theory of scientific hypotheses in its historical context. Stud Hist Philos Sci. (2022) 92:12–9. doi: 10.1016/j.shpsa.2022.01.011

  • CrossRef
  • Google Scholar

• 9

  ErturkSMOndategui-ParraSOteroHRosPR. Evidence-based radiology. J Am Coll Radiol. (2006) 3:513–9. doi: 10.1016/j.jacr.2006.01.005

  • CrossRef
  • Google Scholar

• 10

  MonticcioloDLMalakSFFriedewaldSMEbyPRNewellMSMoyLet al. Breast cancer screening recommendations inclusive of all women at average risk: update from the ACR and society of breast imaging. J Am Coll Radiol. (2021) 18:1280–8. doi: 10.1016/j.jacr.2021.04.021

  • CrossRef
  • Google Scholar

• 11

  BerlinL. Radiologic errors and malpractice: a blurry distinction. AJR Am J Roentgenol. (2007) 189:517–22. doi: 10.2214/AJR.07.2209

  • CrossRef
  • Google Scholar

• 12

  Biomarkers Definitions Working, G. Biomarkers and surrogate endpoints: preferred definitions and conceptual framework. Clin Pharmacol Ther. (2001) 69:89–95. doi: 10.1067/mcp.2001.113989

  • CrossRef
  • Google Scholar

• 13

  KesslerLGBarnhartHXBucklerAJChoudhuryKRKondratovichMVToledanoAet al. The emerging science of quantitative imaging biomarkers terminology and definitions for scientific studies and regulatory submissions. Stat Methods Med Res. (2015) 24:9–26. doi: 10.1177/0962280214537333

  • CrossRef
  • Google Scholar

• 14

  SullivanDCObuchowskiNAKesslerLGRaunigDLGatsonisCHuangEPet al. Metrology standards for quantitative imaging biomarkers. Radiology. (2015) 277:813–25. doi: 10.1148/radiol.2015142202

  • CrossRef
  • Google Scholar

• 15

  Administration USFaD. Clinical Trial Endpoints for the Approval of Cancer Drugs and Biologics, Guidance for Industry. In: Industry Gf, editor.: FDA, New Hampshire Ave Silver Spring. (2018).

  • Google Scholar

• 16

  BurgessA. Signal detection theory: A brief history. In: SameiEKrupinskiEA, editors. The Handbook of Medical Image Perception and Techniques. Cambridge: Cambridge University Press (2018) p. 28–48.

  • Google Scholar

• 17

  ChaterNZhuJ-QSpicerJSundhJLeón-VillagráPSanbornA. Probabilistic biases meet the bayesian brain. Current Directions in Psychological Science (2020) 29:506–12. doi: 10.1177/0963721420954801

  • CrossRef
  • Google Scholar

• 18

  EgglinTKFeinsteinAR. Context bias. A problem in diagnostic radiology. JAMA. (1996) 276:1752–5. doi: 10.1001/jama.1996.03540210060035

  • CrossRef
  • Google Scholar

• 19

  TirkesTHollarMATannMKohliMDAkisikFSandrasegaranK. Response criteria in oncologic imaging: review of traditional and new criteria. Radiographics. (2013) 33:1323–41. doi: 10.1148/rg.335125214

  • CrossRef
  • Google Scholar

• 20

  BurgessAEColborneB. Visual signal detection. IV. Observer inconsistency. J Optical Soc America A. (1988) 5:617–27. doi: 10.1364/JOSAA.5.000617

  • CrossRef
  • Google Scholar

• 21

  RyanJWHollywoodAStirlingAGlynnMMacMahonPJBolsterF. Evidenced-based radiology? A single-institution review of imaging referral appropriateness including monetary and dose estimates for inappropriate scans. Ir J Med Sci. (2019) 188:1385–9. doi: 10.1007/s11845-019-02005-8

  • CrossRef
  • Google Scholar

• 22

  IannessiABeaumontHLiuYBertrandAS. RECIST 1.1 and lesion selection: How to deal with ambiguity at baseline? Insights Imaging. (2021) 12:36. doi: 10.1186/s13244-021-00976-w

  • CrossRef
  • Google Scholar

• 23

  SchmidAMRaunigDLMillerCGWalovitchRCFordRWO'ConnorMet al. Radiologists and clinical trials: part 1 the truth about reader disagreements. Ther Innov Regul Sci. (2021) 55:1111–21. doi: 10.1007/s43441-021-00316-6

  • CrossRef
  • Google Scholar

• 24

  WaiteSGrigorianAAlexanderRGMacknikSLCarrascoMHeegerDJet al. Analysis of perceptual expertise in radiology - current knowledge and a new perspective. Front Hum Neurosci. (2019) 13:213. doi: 10.3389/fnhum.2019.00213

  • CrossRef
  • Google Scholar

• 25

  DonaldJJBarnardSA. Common patterns in 558 diagnostic radiology errors. J Med Imaging Radiat Oncol. (2012) 56:173–8. doi: 10.1111/j.1754-9485.2012.02348.x

  • CrossRef
  • Google Scholar

• 26

  GandomkarZTayKBrennanPCMello-ThomsC. Recurrence quantification analysis of radiologists’ scanpaths when interpreting mammograms. Med Phys. (2018) 45:3052–62. doi: 10.1002/mp.12935

  • CrossRef
  • Google Scholar

• 27

  DrewTVoMLOlwalAJacobsonFSeltzerSEWolfeJM. Scanners and drillers: characterizing expert visual search through volumetric images. J Vis. (2013) 13:3. doi: 10.1167/13.10.3

  • CrossRef
  • Google Scholar

• 28

  BerbaumKSFrankenEAJr.DorfmanDDRooholaminiSAKatholMHBarloonTJet al. Satisfaction of search in diagnostic radiology. Invest Radiol. (1990) 25:133–40. doi: 10.1097/00004424-199002000-00006

  • CrossRef
  • Google Scholar

• 29

  WilliamsLCarriganAAuffermannWMillsMRichAElmoreJet al. The invisible breast cancer: Experience does not protect against inattentional blindness to clinically relevant findings in radiology. Psychon Bull Rev. (2021) 28:503–11. doi: 10.3758/s13423-020-01826-4

  • CrossRef
  • Google Scholar

• 30

  DrewTVoMLWolfeJM. The invisible gorilla strikes again: sustained inattentional blindness in expert observers. Psychol Sci. (2013) 24:1848–53. doi: 10.1177/0956797613479386

  • CrossRef
  • Google Scholar

• 31

  BusbyLPCourtierJLGlastonburyCM. Bias in radiology: the how and why of misses and misinterpretations. Radiographics. (2018) 38:236–47. doi: 10.1148/rg.2018170107

  • CrossRef
  • Google Scholar

• 32

  BeaumontHBertrandASKlifaCPatritiSCippoliniSLoveraCet al. Radiology workflow for RECIST assessment in clinical trials: Can we reconcile time-efficiency and quality? Eur J Radiol. (2019) 118:257–63. doi: 10.1016/j.ejrad.2019.07.030

  • CrossRef
  • Google Scholar

• 33

  SinghHGraberMLKissamSMSorensenAVLenfesteyNFet al. System-related interventions to reduce diagnostic errors: a narrative review. BMJ Qual Saf. (2012) 21:160–70. doi: 10.1136/bmjqs-2011-000150

  • CrossRef
  • Google Scholar

• 34

  RichieMJosephsonSA. Quantifying heuristic bias: anchoring, availability, and representativeness. Teach Learn Med. (2018) 30:67–75. doi: 10.1080/10401334.2017.1332631

  • CrossRef
  • Google Scholar

• 35

  GraberMLKissamSPayneVLMeyerANSorensenALenfesteyNet al. Cognitive interventions to reduce diagnostic error: a narrative review. BMJ Qual Saf. (2012) 21:535–57. doi: 10.1136/bmjqs-2011-000149

  • CrossRef
  • Google Scholar

• 36

  HartungMPBickleICGaillardFKanneJP. How to create a great radiology report. Radiographics. (2020) 40:1658–70. doi: 10.1148/rg.2020200020

  • CrossRef
  • Google Scholar

• 37

  HallFM. Language of the radiology report: primer for residents and wayward radiologists. AJR Am J Roentgenol. (2000) 175:1239–42. doi: 10.2214/ajr.175.5.1751239

  • CrossRef
  • Google Scholar

• 38

  BrownPJRossingtonHTaylorJLambregtsDMJMorrisEWestNPet al. Standardised reports with a template format are superior to free text reports: the case for rectal cancer reporting in clinical practice. Eur Radiol. (2019) 29:5121–8. doi: 10.1007/s00330-019-06028-8

  • CrossRef
  • Google Scholar

• 39

  PatelAGPizzitolaVJJohnsonCDZhangNPatelMD. Radiologists make more errors interpreting off-hours body CT studies during overnight assignments as compared with daytime assignments. Radiology. (2020) 297:374–9. doi: 10.1148/radiol.2020201558

  • CrossRef
  • Google Scholar

• 40

  MankadKHoeyETJonesJBTirukondaPSmithJT. Radiology errors: are we learning from our mistakes? Clin Radiol. (2009) 64:988–93. doi: 10.1016/j.crad.2009.06.002

  • CrossRef
  • Google Scholar

• 41

  ChaudhryHDel GaizoAJFriginiLAGoldberg-SteinSLongSDMetwalliZAet al. Forty-one million RADPEER reviews later: what we have learned and are still learning. J Am Coll Radiol. (2020) 17:779–85. doi: 10.1016/j.jacr.2019.12.023

  • CrossRef
  • Google Scholar

• 42

  MaurerMHBronnimannMSchroederCGhadamgahiEStreitparthFHeverhagenJTet al. Time requirement and feasibility of a systematic quality peer review of reporting in radiology. Rofo. (2021) 193:160–7. doi: 10.1055/a-1178-1113

  • CrossRef
  • Google Scholar

• 43

  FormanHPLarsonDBKazerooniEANorbashACroweJKJavittMCet al. Masters of radiology panel discussion: hyperefficient radiology–can we maintain the pace? AJR Am J Roentgenol. (2012) 199:838–43. doi: 10.2214/AJR.12.9648

  • CrossRef
  • Google Scholar

• 44

  BrulsRJMKweeRM. Workload for radiologists during on-call hours: dramatic increase in the past 15 years. Insights Imaging. (2020) 11:121. doi: 10.1186/s13244-020-00925-z

  • CrossRef
  • Google Scholar

• 45

  SokolovskayaEShindeTRuchmanRBKwakAJLuSShariffYKet al. The effect of faster reporting speed for imaging studies on the number of misses and interpretation errors: A pilot study. J Am Coll Radiol. (2015) 12:683–8. doi: 10.1016/j.jacr.2015.03.040

  • CrossRef
  • Google Scholar

• 46

  FitzGeraldR. Radiological error: analysis, standard setting, targeted instruction and teamworking. Eur Radiol. (2005) 15:1760–7. doi: 10.1007/s00330-005-2662-8

  • CrossRef
  • Google Scholar

• 47

  WilliamsLHDrewT. Distraction in diagnostic radiology: How is search through volumetric medical images affected by interruptions? Cognit Res Princ Implic. (2017) 2:12. doi: 10.1186/s41235-017-0050-y

  • CrossRef
  • Google Scholar

• 48

  DembrowerKWahlinELiuYSalimMSmithKLindholmPet al. Effect of artificial intelligence-based triaging of breast cancer screening mammograms on cancer detection and radiologist workload: a retrospective simulation study. Lancet Digit Health. (2020) 2:e468–74. doi: 10.1016/S2589-7500(20)30185-0

  • CrossRef
  • Google Scholar

• 49

  LauritzenADRodriguez-RuizAvon Euler-ChelpinMCLyngeEVejborgINielsenMet al. An artificial intelligence-based mammography screening protocol for breast cancer: outcome and radiologist workload. Radiology. (2022) 304:41–9. doi: 10.1148/radiol.210948

  • CrossRef
  • Google Scholar

• 50

  BahramiBOlsenKLathamPERoepstorffAReesGFrithCD. Optimally interacting minds. Science. (2010) 329:1081–5. doi: 10.1126/science.1185718

  • CrossRef
  • Google Scholar

• 51

  BeaumontHEvansTLKlifaCGuermaziAHongSRChadjaaMet al. Discrepancies of assessments in a RECIST 1.1 phase II clinical trial - association between adjudication rate and variability in images and tumors selection. Cancer Imaging. (2018) 18:50. doi: 10.1186/s40644-018-0186-0

  • CrossRef
  • Google Scholar

• 52

  DondiMPaezDTorresLMarengoMDelaloyeABSolankiKet al. Implementation of quality systems in nuclear medicine: why it matters. An outcome analysis (Quality management audits in nuclear medicine part III). Semin Nucl Med. (2018) 48:299–306. doi: 10.1053/j.semnuclmed.2017.12.001

  • CrossRef
  • Google Scholar

• 53

  KweeTCKweeRM. Workload of diagnostic radiologists in the foreseeable future based on recent scientific advances: growth expectations and role of artificial intelligence. Insights Imaging. (2021) 12:88. doi: 10.1186/s13244-021-01031-4

  • CrossRef
  • Google Scholar