Building an Adverse Outcome Pathway network for COVID-19

1 Introduction

SARS-CoV-2 infection provokes the coronavirus disease 2019 (COVID-19). Patients can experience a range of clinical manifestations, from no symptoms to critical illness and death. Underlying mechanisms of COVID-19 have been extensively investigated and were instrumental for the implementation of rapid therapeutic and preventive measures. However, this unprecedented worldwide investigation of a disease resulted in an overwhelming flux of in vitro, in vivo, clinical and epidemiological data regarding COVID-19 pathogenesis. In this context, the CIAO project (https://www.ciao-covid.net/) was initiated early 2020 by the Joint Research Centre of the European Commission as a crowdsourcing initiative comprising more than 70 scientists from academic, regulatory and industrial organizations worldwide (Nymark et al., 2021). The aim of the project was to comprehensively gather and systematically organize published scientific literature on COVID-19 pathology using the Adverse Outcome Pathway (AOP) framework.

An AOP describes a series of biological Key Events (KEs) that are essential for the initiation and progression of the disease, of which the Molecular Initiating Event (MIE) is the first one (Villeneuve et al., 2014; OECD, 2018). MIE describes the molecular interaction between a stressor and the biological system, responsible for initiating the downstream cascade of KEs, culminating in the manifestation of an Adverse Outcome (AO). The Key Event Relationships (KERs) describe the causal relationship between an upstream and a downstream KE. In essence, the series of selected set of molecular, cellular and tissue level KEs and KERs presented in an AOP are expected to predict the occurrence of an AO at the tissue, organism, individual or population level.

Within the CIAO project, 22 AOPs were partially or fully developed spanning multiple organs such as respiratory systems, nervous system, gastrointestinal tract and liver (Wittwehr et al., 2021; Clerbaux, Amigó, et al., 2022). Around 70 KEs were identified occurring at different levels of biological organization that play a role in initiation, promotion, and clinical manifestation of COVID-19. Around half of these KEs were already described in the online repository platform AOP-Wiki (www.aop-wiki.org), such as the set of harmonized KEs representing inflammation (Villeneuve et al., 2019) re-used to build the inflammatory response of the disease. The remaining >30 KEs were identified and added to the AOP-Wiki during the CIAO project.

Fourteen of those AOPs start with the same MIE, namely, “binding to angiotensin converting enzyme-2 (ACE2)” receptor. For 7 of those, binding to ACE2 leads to viral replication as the virus mainly uses the ACE2 receptor to enter the cell and replicate. Viral replication is essential to drive some outcomes, hence it was proposed to develop a viral hub AOP consisting of SARS-CoV-2 specific KEs depicting its replication steps (Clerbaux et al., 2023).

Regarding AOs, the 22 CIAO AOPs depict pathways leading to 19 outcomes of the multifaceted COVID-19, namely, infection proliferation, thrombosis, microvascular dysfunction, hyperinflammation, thrombo-inflammation, acute respiratory distress, lung fibrosis (x2), decreased lung function, encephalitis, short-term anosmia, stroke, dysgeusia, intestinal barrier disruption, gut dysbiosis, liver fibrosis, liver injury (x2), kidney injury (x2), heart failure and myocardial infraction. The evidence supporting the development of the pathways leading to short-term anosmia, intestinal disorders and gut dysbiosis have been described in detail by the consortium (Clerbaux, Fillipovska, et al., 2022; Clerbaux, Mayasich, et al., 2022; Shahbaz et al., 2022). AOPs related to neurological symptoms in COVID-19 such as stroke, encephalitis and others have been detailed in Hogberg et al., 2022. The 2 AOPs leading to liver injury propose an indirect effect via systemic inflammation or due to hypoxia/thrombosis following binding to ACE2 in the lungs (Vinken, 2021). The 2 AOPs leading to kidney injury were built on the similar rationale (Wittwehr et al., 2021). Those 4 liver and kidney related AOPs were not added into the AOP-Wiki.

In addition, lung fibrosis and decreased lung function were already reported in the AOP-Wiki to be triggered by certain chemicals and nanomaterials (Gerloff et al., 2017; Da et al., 2021). Similarities in responses to SARS-CoV-2 and those nanomaterials or chemicals suggest common pathways leading to lung fibrosis and decrease lung function highlighting mechanistic cross-talk between (nano)toxicology and viral diseases (Gerloff et al., 2017; Kim et al., 2021; Kinaret, del Giudice, and Dario, 2020). Those AOPs were further considered in the project (Clerbaux, Amigó, et al., 2022), resulting in a total of 24 individual linear AOPs linked to COVID-19. From those 24 AOPs, 20 are present in the AOP-Wiki and 11 were included under the CIAO 1.96 Organisation for Economic Co-operation and Development (OECD) AOP program work plan but have, however, not yet completed the OECD reviewing process at this date.

Individual AOPs are pragmatic units describing how defined perturbations of a biological system can lead in a causal manner to a particular AO. However, to reflect the complex biology and interrelatedness of the various processes, a network of AOPs can be generated that can, in turn, guide AOP development (Knapen et al., 2018; Villeneuve et al., 2018). In an AOP network derivation, shared KEs or hub node revealed after AOPs have been developed independently (Knapen et al., 2018).

The objective of this study was to derive a directed network of AOPs for COVID-19. The primary network was used to inform the curation procedure highlighting the inconsistencies in AOP nomenclature and harmonization opportunities. After manual refinement, the final network enabled visualization of the complex interactions of the KEs leading to the various AOs of COVID-19. Based on the challenges encountered in deriving and visualizing a biologically relevant Wiki-derived AOP network for COVID-19 and building further on previous efforts (Wiklund et al., 2023), we propose here recommendations to address issues such as ontology standardization and organ specificity in the AOP-Wiki. We also discuss ways to tackle quality issues in individual AOPs in the AOP-Wiki such as unequal completeness in KE description and limited scope of available evidence provided in KERs. These quality issues are acknowledged as reflecting the need for further implementation of the FAIR principles to improve AOP findability, accessibility, interoperability and re-usability (Wittwehr et al., 2024). Finally, the study highlights the specific challenges of exploiting AOPs for a viral disease such as the need for capturing KEs specific to SARS-CoV-2 replication while complying with the stressor agnostic principle of the framework. Thus, this study proposes recommendations linked to the challenges encountered during development of an AOP network for modeling diseases as well as for assessing toxicity.

2 Methods & materials

2.1 Manual extraction and inclusion of relevant AOPs

AOPs describing KEs involved in COVID-19 were collected manually. This included the individual AOPs developed within the CIAO project and AOPs already available in the AOP-Wiki. The AOPs that did not have a unique AOP-Wiki identifiers (IDs), that were not compliant with the OECD AOP development guidance (OECD, 2018), that lacked KER components and that were incomplete with only KEs or KERs titles in the AOP-Wiki, were excluded from the analysis. All AOPs and associated KEs/KERs included in the network are available in the AOP-Wiki and publicly accessible under a CC BY 4.0 license.

2.2 Systematic and manual pre-processing of the KE and KERs contained in the AOPs

The KEs and KERs contained in the selected AOPs were retranscribed in a spreadsheet (Supplementary Material S1, first sheet). In addition, an expert analysis was also performed to identify KE that describe the same biological perturbation but under different KE titles and manually merge them under a single ID while retaining their original AOP-Wiki IDs in the spreadsheet (Supplementary Material S1, second sheet). To account for KE duplications frequently observed in AOP-Wiki, a systematic way to map KE titles to standard ontology terms derived from various ontologies was used (Supplementary Material S1 1, third sheet). This is a crucial step towards establishing an interoperability across different AOPs, needed for computational generation of a shared node in the network.

2.3 Computational derivation of a primary network

The pre-processed information (S1) was used to generate a directional AOP network using the open-source network visualization program Cytoscape v3.7 through a series of iterative processes. The spreadsheet (S1) was imported through Cytoscape “import network from file” function. Interaction parameters and attributes were defined considering directionality of KEs in the AOPs, including source and target nodes in order to allow the program to identify KERs (i.e., connectivity between downstream to upstream KEs), interaction type (adjacent or nonadjacent), and edge, target and source node attributes (AOP and KE type, e.g., whether a node was a MIE, KE or an AO). The attributes were used to define visualization options, such as colored nodes with the MIE in green, KEs in yellow/orange and the AO in red. The edge (i.e., KER) colors correspond to individual AOPs. The Cytoscape app NetworkAnalyzer was applied to analyze the directed network properties.

The primary network could also be derived from the third party tool Biovista Vizit based on the selected AOPs and expansion of the KEs with a minimum effort on node rearrangement.

2.4 Manual refinement of the primary network

The computationally generated primary network was further refined and organized in a tissue specific manner. This required duplication of some MIEs and KEs as well as integration of missing elements.

3 Results

3.1 Manual extraction of the relevant AOPs

As part of the CIAO project, 22 AOPs have been developed of which 18 were added into the AOP-Wiki and 11 were included under the 1.96 OECD AOP program work plan (Table 1). Binding to ACE2 is the MIE of 14 of those AOPs. Binding to ACE2 leads to viral replication for 7 of them (AOP320, 379, 394, 395, 422, 428 and 430) as the virus mainly uses this receptor to enter the cell and replicate. While for 4 AOPs (AOP381, 383, 385, 428), binding to ACE2 induces ACE2 dysregulation/downregulation depicting the effect on the physiological role in the renin-angiotensin system (RAS) of ACE2 when hijacked by SARS-CoV-2.

Table 1

Table 1. The list of 24 AOPs of which 17 (in bold) were considered for inclusion in the primary network.

In addition to the AOPs developed within the CIAO project, 2 AOPs already present in the AOP-Wiki, namely, AOP173 developed under OECD project 1.32 and AOP302 developed under OECD project 1.87, described pathways that were relevant for COVID-19 (Da et al., 2021; Halappanavar et al., 2023). Taking advantage of the stressor agnostic principle of the framework which bridge knowledge of mechanisms of adversity, those 2 AOPs were further considered for COVID-19.

In total, 24 AOPs were collected, of which 17 were considered for inclusion for the primary network derivation and analysis. The other AOPs were excluded as they were not present in the Wiki or not following the OECD guidance (Table 1).

3.2 Systematic and manual pre-processing of the information contained in the AOPs

Systematic analysis of KEs containing similar KE titles, resulted in the identification of the following KEs as potential duplicates: KE1276 “Lung fibrosis” and KE1458 “Pulmonary fibrosis”. An additional expert judgment was necessary to manually merge 16 additional paired KEs that are similar with regards to the biological perturbations that they describe. A common title was arbitrary given for the purposes of network analysis (Table 2).

Table 2

Table 2. Harmonization of KE titles of duplicated KEs.

3.3 Computational derivation of a primary network

The pre-processed information was used to generate a directional AOP network using Cytoscape through a series of iterative processes (Figure 1). Nodes were colored in green for MIE, yellow for KEs and red for AO. The edge (i.e., KER) colors differentiate individual AOPs.

Figure 1

Figure 1. Computational derivation of a primary network using Cytoscape.

In addition to the network generated by Cytoscape, a network generated by Biovista Vizit as a live editable graph (Figure 2) is provided. For clarity, circles were drawn around the harmonized KEs (blue circles in Figure 2).

Figure 2

Figure 2. Network of COVID-19 AOPs as generated by Biovista Vizit tool.

The use of an automated tool for visualization makes the high-level inspection of the graph easier. The Vizit graph can be divided into two groups of AOPs: the ones that have ACE2 Binding KE-and associated downstream and upstream KEs (purple circles and red box) and others that do not. At the top of the graph, 2 KEs are not connected; KE1874 “Blood brain barrier disruption” and KE1875 “Cerebrovascular disease (stroke)”, both of which are part of AOP395 “Binding of Sars-CoV-2 spike protein to ACE2 receptors expressed on pericytes leads to stroke”.

3.4 Manual refinement of primary network

3.4.1 Identification of missing KEs and emergence of a new AOP

The primary network highlighted the fact that cell death, widely acknowledged as an essential and crucial step in disease pathogenesis (Deshpande and Zou, 2021; Li et al., 2022; Tojo, 2023; Yuan et al., 2023) was not present in any of the AOPs, hence in the network. Therefore, the KE “cell death” KE1825 present in the Wiki was added as a new node connecting “increased coronavirus production” (KE1847 from AOP430) to “increased, secretion of proinflammatory mediators (KE 1496 from AOP392). With this, a new AOP emerged: “Binding of SARS-CoV-2 to ACE2 leads to hyperinflammation via cell death” which was added to the AOP-Wiki (AOP468).

3.4.2 Merging of two identical AOPs

The primary network also highlighted that AOP382 and AOP319 described overlapping KEs of a same pathway involving RAS up to lung fibrosis and have been therefore merged into one AOP (AOP319) as such: ACE2 downregulation > Ang II accumulation > Ang II-AT1R activation > ROS regeneration > Collagen deposition > lung fibrosis.

3.4.3 Duplication of tissue specific MIEs and KEs

During manual refinement of the primary network, a need for visualizing tissue-specificity for the MIEs and early KEs was identified.

First, the MIE describing “binding to ACE2” (MIE1739) and the early viral KEs, namely, KE1738 (SARS-CoV-2 cell entry) and KE 1847 (Increased SARS-CoV-2 production) were replicated. Replication of the KEs (i.e., nodes in the network) was accompanied by creation of a new tissue specific KER annotated with the relevant AOP-belonging of the KE. This step was important to distinguish between the entry routes. For example, viral entry through sustentacular cells (nasal), through pulmonary cells (lungs), or through enterocytes (gut) had to be differentiated, as the downstream KEs and AOs will be different depending on the organ. Not duplicating tissue specific MIEs would have led to a network that is not biologically meaningful. For example, without that manual refinement, binding to ACE2 in enterocytes would lead to anosmia. In addition, a new previously undefined MIE, referring to ‘secondary’ routes of viral entry, was added to reflect potential proliferation of the virus to other organs via the systemic circulation (Kirtipal et al., 2022).

Second, the KE 1854 (Dysregulation of ACE2 expression and activity) was replicated to differentiate the effect of ACE2 dysregulation in enterocytes versus in lungs for the same rationale as described above.

3.4.4 Network-guided identification of missing KERs

The addition of ‘increased cell death’ and “secondary entry routes” resulted in identification of 4 new KERs (one to four in Table 3). Besides, to capture the clinical outcomes in liver and kidney, AO344 “liver injury” and AO759 “increased kidney failure” were added as downstream KEs of thrombosis and hyperinflammation (Vinken, 2021) leading to the creation of 4 extra KERs (five to eight in Table 3). Based on additional mechanisms proposed in CIAO publications (Clerbaux, Mayasich, et al., 2022; Hogberg et al., 2022), 3 extra KERs were added linked to neurological syndromes and 7 related to gastrointestinal symptoms (9–18 in Table 3).

Table 3

Table 3. AOP network-driven identification of new KERs.

All those 18 KERs were added to the network as new directed edges connecting the nodes representing the upstream and downstream KEs. The majority of these KERs are however currently empty in the AOP-Wiki. The recently described pragmatic “unit approach” for development of single KER could be applied for these KERs in the future (Svingen et al., 2021).

3.5 Final network

AOPs are living documents that can be constantly updated as new information becomes available. However, AOP381 and AOP383 were still empty at the time of final network and were then removed. The final COVID-19 AOP network included a total of 14 AOPs: the 13 highlighted in bold in Table 1 and the newly emerged from primary network refinement (AOP468).

3.5.1 Visualization

The network was refined to distinguish between the different viral entry routes and consequent organ/system specific responses (nasal and brain, pulmonary, gut and systemic) (Figure 3). In addition, the network was divided into four distinct phases of disease progression involving biological events at the molecular, cellular and tissue levels, within each phase to address partially the challenge of the temporality of the disease not well-captured in AOPs.

(i) Disease onset phase - viral infection phase involving binding of SARS-CoV-2 to a cellular receptor, viral entry, antagonism of the antiviral response, replication and release of new virions as well as ACE2 dysregulation.

(ii) Early cellular and tissue damage.

(iii) Immune and vascular response phase - including the hyperinflammation response.

(iv) Late adverse outcome phase: pulmonary failure, brain, gut and other organs injury.

Figure 3

Figure 3. COVID-19 AOP network involving 14 individual adverse outcome pathways. Green nodes indicate molecular initiating events (MIEs), Orange nodes are key events (KEs) and Red nodes are adverse outcomes (AOs). Transparent boxes depict KEs including MIEs clustered by the disease phase and tissue specificity. The color code for KER belonging indicates which AOP a specific KER (edge in the network) belongs to.

3.5.2 Topology

In total, 69 KEs formed the network with a clustering coefficient of 0.159. The average number of neighbors for each KE was 2.551, which indicates that each individual KE is connected (direct or indirect) with approximately two other KEs on average. The most highly connected KE was Increased, secretion of proinflammatory mediators (KE1496, KE87) with 14 connections creating a bow-tie topology to the network.

3.5.3 Clustering

A bird’s-eye view version of the network was generated using the node clustering function in Cytoscape to cluster MIEs/KEs (nodes in the network) in line with overarching biological mechanisms and organ-specificity (Figure 4). Clustered nodes were collapsed by activating the expand/contract-function and visualizing it as a cluster including the descriptions of the core mechanisms. For example, the MIEs and KEs associated with the disease onset phase in the lungs were clustered and described under ACE2 binding and viral replication in pulmonary cells. The bird’s eye view supported a new level of insight in terms of expert-driven identification of gaps and discrepancies. For example, central KERs and KEs that had not been previously described in the 13 AOPs were identified, as described further below.

Figure 4

Figure 4. Bird’s-eye view of the COVID-19 AOP network providing insight into the interconnection between the various stages and tissue-specificities of the disease.

3.6 Brief description of the COVID-19 KEs and KERs

We aim here to briefly present the different KEs and KERs integrated in the network. The references and associated evidence used to support their essentiality in COVID-19 pathogenesis are documented in their AOP-Wiki IDs or are documented by references from the literature when KERs pages are empty.

3.6.1 Disease onset phase

3.6.1.1 Binding to receptor, viral entry, replication, and release of new virions

In the respiratory tract, SARS-CoV-2 spike proteins bind to the ACE2 receptor expressed at high levels on airway epithelial cells, alveolar epithelial cells, vascular endothelial cells and macrophages (KE1739, Binding to ACE2). Upon binding, the S protein subunits undergo sequential cleavage mediated by proteases and conformational changes that result in viral entry into the cells (KER2056). Many proteases, such as transmembrane serine protease 2 (TMPRSS2) have been identified to aid the virus with its cellular entry (KE1738, SARS-CoV-2 cell entry). Once inside of a cell, the virus undergoes replication (KER2496). The SARS-CoV-2 virus has evolved with a repertoire of proteins that bind and block proteins in the interferon (IFN) cascade primarily inhibiting the expression of host antiviral proteins, enabling replication of the virus (KE 1901, IFN-1 antiviral response, antagonized). When the antiviral response is antagonized, the viral RNA is translated, replicated, transcribed and the genomic RNA is packaged (KER2497) and the new SARS-CoV-2 virions are assembled (KE1847, Increased SARS-CoV-2 production). Subsequently, the newly formed virions are ready to infect an adjacent cell or another human via respiratory droplets, repeating the viral replication and infection cascade (KE 1939, Viral Infection, proliferated).

The virus can bind to any cell that expresses the ACE2 receptor on its surface along the path of the respiratory tract and, when distributed systemically, can bind to ACE2 positive cells in other organs (KE1739). ACE2 is also expressed on sustentacular cells of the olfactory neuroepithelium and enterocytes in the small intestine (KE1739). Hence, sustentacular cells could be infected as well as enterocytes via the olfactory and gastrointestinal routes respectively. And this has been represented in the network.

3.6.1.2 ACE2 dysregulation

In addition to viral infection, the occupation of ACE2 receptors by the viral protein has been shown to dysregulate ACE2 expression and activity (KE1740, KE1787 and KE 1854, ACE2 Dysregulation). This leads to increased Angiotensin II (Ang II) (KE1752, KE1743 and KE1741, Increased, Angiotensin II) and increased expression and activity of its receptor Ang II type 1 (KE1852 - KE1851 Increased Ang II type I receptor (AT1R)), involved in RAS.

In the gastrointestinal tract, ACE2 also has a RAS-independent function in modulating dietary amino acid transport (KE 1854, (enterocytes)) and therefore, ACE2 dysregulation might interfere with intestinal homeostasis locally (Clerbaux et al.).

3.6.2 Early cellular and tissue damage

3.6.2.1 Cell death and damage response in pulmonary cells

In the lungs, SARS-CoV-2 is a cytopathic virus that causes extensive cell death (KE1825 Increase, cell death). KER linking SARS-CoV-2 replication to cell death has been created (KER2488).

Downregulation of ACE2 activity leads to activation of the tissue factor-dependent coagulation and inhibition of fibrinolysis (KE 1866, fibrinolysis, decreased) by Ang II (new KER in network). Internalization of ACE2 and hypofibrinolysis can lead to reduced degradation of bradykinin (KER2352), raising bradykinin levels and activities (KE 1867, Bradykinin system, hyperactivated), which could explain many of the symptoms associated with COVID-19, including vasodilation, hypotension, vascular permeability and hyperinflammation (KER2357).

Other KEs, such as KE1672 (Inhibition of lung surfactant function) and KE 1913 (Endothelial dysfunction), accentuate the damage and possibly set the stage for the severity of the ensuing pro-inflammatory response.

3.6.3 Olfactory damage and blood-brain barrier disruption

In the olfactory epithelium, SARS-CoV-2 infection induces a decrease of sustentacular cell numbers (KE 1870, KER2545), which provide functional and structural support to the olfactory epithelium. The death of these supporting cells can lead to downregulation of olfactory sensory neurons (KE 1871, KER2362) and to degeneration in the structure and function of the olfactory epithelium (KE 1872, KER2363).

In addition to the nasal epithelium, there are other secondary routes of SARS-CoV-2 entry into the central nervous system (Hogberg et al., 2022). The invasion and infection of central nervous system lead to local immune response activation resulting in a cytokine storm, potentially provoking disruption of the blood brain barrier (BBB) (KE 1874, BBB disruption).

3.6.3.1 Gastrointestinal damage

Under certain conditions, SARS-CoV-2 actively infects enterocytes potentially disrupting the intestinal barrier integrity (KE 1931, Intestinal barrier, disruption, KER2546), inducing a local inflammatory response in the gastrointestinal tract and fueling the systemic hyperinflammation (Clerbaux, Mayasich, et al., 2022). In addition, presence of SARS-CoV-2 in the gut has been shown to alter the gut microbiota (KE 1954, Gut microbiota alteration), potentially as a consequence of a disrupted intestinal barrier or because of the dysregulation of intestinal ACE2 (Clerbaux, Fillipovska, et al., 2022).

3.6.4 Immune and vascular phase

Increased secretion of soluble factors including cytokines and chemokines (KE1496, Increased secretion of pro-inflammatory mediators) is the most common node in the network that is connected to several other KEs (Figure 1). Upstream KEs known to lead to release of pro-inflammatory mediators are dysregulated RAS, increased pro-inflammatory signaling (KE1172, Increase activation, Nuclear factor Kappa B (NFκB).

In the other direction, pro-inflammatory factors recruit immune cells (KER1703) including macrophages, monocytes and lymphocytes to the sites of infection (KE1497, KE1750, Increased recruitment of leukocytes). Immune cells further amplify the secretion of cytokines and chemokines, creating a pro-inflammatory environment (KER2761) leading to the pathological state of hyperinflammation (KE1866 Hyperinflammation) (KE2354). In addition, release of pro-inflammatory cytokines (KE1496) increases platelet activation (KE 1914) which increases neutrophil activation (KE915) increasing Platelet-neutrophil interactions (KE 1857) inducing Thromboinflammation (KE 1916). (KER2457 and KER2458).

During this phase, the endothelial injury leads to deregulation of thromboinflammatory processes, resulting in thrombus formation KE 1846 (Thrombosis and Disseminated intravascular coagulation), vascular injury and vascular dysfunction.

3.6.5 Late adverse outcomes

3.6.5.1 Pulmonary outcomes

Acute respiratory distress syndrome (ARDS, AO1748, Increased risk of respiratory failure) is one of the predominant AOs of the target organ in COVID-19. In survivors, Increased collagen deposition (KE68) leads to Pulmonary fibrosis (AO1458, AO1276). Hyperinflammation and lung tissue injury lead to Loss of alveolar capillary membrane integrity (KE1498), Reduced tidal volume (KE1677) and Alveolar collapse (KE1673) resulting in decreased lung function (AO1250).

3.6.5.2 Brain injuries

Several neurological outcomes are commonly observed in COVID-19, including short-term anosmia (AO 1873), neurodegeneration (KE352), and encephalitis (AO 1841). Effects on the nervous systems could be the results of primary viral infection in the tissues or secondary to the target organ response in lungs (Kirtipal et al., 2022).

3.6.5.3 Gastrointestinal disorders and multi-organ failure

Some COVID-19 patients also presented gastrointestinal symptoms such as diarrhea, abdominal discomfort, nausea and/or vomiting (AO 1932) and gut dysbiosis (KER2532).

As the disease spreads systemically, other AOs including liver injury (AO344) and kidney failure (AO759) were added as a consequence of hyperinflammation or thrombosis (Vinken, 2021; Wittwehr et al., 2021).

4 Discussion

With the CIAO project, for the first time, a viral disease was structured within AOPs. Conceptually, a network graph can quickly capture information and an appropriate layout can be instrumental in understanding complex interdependent data (Newman, 2003). Visualization tools used in this study provided semi-automatic processes to import AOP-Wiki information and construct a primary network, highlighting inconsistencies. The final AOP network developed in this study enables visualization of the interrelations between the many perturbed biological events along the four phases of the disease and via the different entry routes of the virus for better understanding of the complex disease. However, generating a COVID-19 relevant AOP network underlined challenges in application of the framework to multi-organ complex pathogenesis involving many routes of exposure.

4.1 Challenges and recommendations to derive biologically relevant AOP networks

4.1.1 Network quality depends on the information in the Wiki

A necessary caution to consider when interpreting AOP networks is that derivation of networks is limited by the scope of knowledge captured in the AOP-Wiki. The nodes and connections reflected in an AOP network are only those for which AOP descriptions have been developed (Villeneuve et al., 2018).

Besides, gaps in the individual AOPs, unequal quality of the KEs description or of the evidence provided for KERs still represent an important limitation in building biologically relevant AOP networks. There is no explicit quality assurance for AOPs in the AOP-Wiki, except when they have gone through a review process according to the OECD guidance (OECD, 2018). AOPs on the OECD work plan benefit from coaching that guide the authors in developing AOP consistent with the framework guidance (OECD, 2018) making those AOPs compliant for future scientific review. The process ensures the development of high quality AOPs fit for use in the research and regulatory science context.

In the present study, the developed AOPs have not yet completed the OECD process, hence they were unequal in their quality and completeness. In general, field information inside KE pages was more complete than in KER pages. A workshop was organized to guide CIAO participants in their efforts to allocate COVID-19 related evidence to KERs while KEs have to contain general and methodology information. The issue of missing or incomplete terms was a major point of focus for the CIAO Ontology Group. In the AOP-Wiki, the authors are required to fill-in several fields/entries, and inevitably, a great majority of those are left empty or filled-in by generic (e.g., " Cell” or " Organ”) terms. Therefore, a bug tracking system, such as the one described below, would allow for warning both the AOP authors and developers against such issues when identified. A concern often raised by AOP authors is that they are not always able to identify a specific term in the current ontology set in the AOP-Wiki. In such cases, the selected ontology needs to be amended with the required information. A suitable alternative for the AOP-Wiki community is to reach out to ontology providers (e.g., the UBERON experts) that are open to recommendations, in order to dictate new terms or required fixes in the respective ontologies.

4.1.2 Importance of ontology harmonization

This study has underlined the importance of terminology harmonization in AOP-Wiki for network building. For example, AO1276 Lung fibrosis and AO1458 Pulmonary fibrosis did not converge as one node in the computationally generated network. This necessitated renaming of the AO title manually to enable connectivity in the network (Supplementary Material S1). Similarly, additional KE titles required manual harmonization before their inclusion in the network. This issue is not specific to COVID-19 and has been encountered in the AOP community since its onset (Ives et al., 2017). The importance of using standardized terminologies to describe the main concepts in an AOP and to avoid fragmentation in the way terms are described, will enable AOPs to be more interoperable with other domains, such as the regulatory world or PubMed. Most importantly, standardization and harmonization of terms will enable the communication in machine-readable format, important for downstream applications, such as reasoning and network visualization.

As discussed previously, the initial efforts focused on automatically identifying consolidation opportunities at the KE title level. However, owing to the lack of KE title standardization, this effort had to be mostly conducted manually. While this is not an easy-to-tackle problem, the CIAO Ontology group assessed the feasibility of adopting the terms from Medical Dictionary for Regulatory Activities (MedDRA), as a first step towards standardization and increasing the interoperability with the regulatory world. MedDRA consists of standardized medical terminology, developed since late 1990s by the International Council for Harmonization of Technical Requirements for Pharmaceuticals for Human Use (ICH), to facilitate sharing of regulatory information internationally for medical products used by humans. MedDRA is also the standard terminology for Adverse Event reporting in the FDA’s Adverse Event Reporting System (FAERS), as well as in other pharmacovigilance databases (e.g., Vigibase). For these reasons, it was considered as a suitable terminology for AO titles as it can better describe stressor-induced AOs observed in clinical settings. The CIAO Ontology Group’s attempt to map AO titles to MedDRA overall resulted in good coverage, as the majority of terms were successfully mapped to MedDRA Preferred Terms, e.g., KE1841 (Encephalitis). In other cases, AO titles were mapped to the most closely related MedDRA term, e.g., KE1875 (Cerebrovascular disease (stroke) was mapped to MedDRA PT Cerebrovascular accident.

Towards a more automated way to tackle such inconsistencies, an additional recommendation was made for KE titles to be automatically generated and composed of terms that exist in the KE components table, a field which, by design, allows for ontology-based annotations of KEs. This is the case of the KE components’ Process and Object fields, both of which rely on a set of ontologies, such as Gene Ontology (GO). The end goal would be to have a structured KE title recommendation as we have for AOPs (MIE leads to AO).

During the adaptation of Biovista Vizit for the AOP-Wiki data, an automatic mapper for multiple ontologies was used as part of the import process. The results for KEs were not as successful as the manual curation since only one pair of CIAO-related KEs were identified as potential duplicates across the whole AOP-Wiki. The reason for this is that the KE titles are not structured and thus the chance to correctly map them to a controlled vocabulary is small. However, the application of the same procedure to the rest of the AOP elements (Stressors, Biological Objects, Biological Process, Cells, Organs), was a much more encouraging process since it identified a number of duplications, invalid entries and other inconsistencies in the AOP-wiki terminology.

The quality issues underlined the need for a tighter implementation of the FAIR principles, for improving findability, accessibility, interoperability and reusability of the AOPs (Wittwehr et al., 2024). First, we propose a continuous integration (CI) system based on the ontology mapper approach, that will test the current state of the AOP-Wiki or any newly submitted entry for known common issues and notify the AOP author or even refuse the entry. The full output of the CI could be published in a specific location inside the AOP-Wiki and possibly, by AOP, KE or even by author, so the users and the developers are aware of the common quality issues that have been automatically detected.

A second recommendation could be the development of a bug tracking system (BT), similar to “GitHub Issues”, that will help users and developers report quality issues, keep track of them and their dependencies and finally resolve them.

However, having in place mechanisms such as CI and BT for detecting, reporting, discussing and organizing quality issues, is not enough. Any such initiative should be coupled by a policy of addressing quality issues. For conflict resolution, one could envision a committee that can have a final decision on how a particular issue should be fixed or which solution should be applied. Finally, it should be possible for engaged individuals within the AOP-Wiki community (e.g., AOP-Wiki curators) to perform complex refactoring, e.g., deduplicating/merging content, so that such issues are readily resolved. Another recommendation could be to develop a systematic way to identify duplicate KEs extracted from the AOP-Wiki and apply it as a filter before providing the primary network.

Finally, one could learn that when embarking on a large-scale AOP-aligned collaboration, harmonization of ontology and controlled vocabulary is something that is best brought about as early as possible, and certainly not assumed as implicit.

4.1.3 Need to account for organ specificity

No additional parameters such as taxonomic space, life stages, sex applicability were used to filter and restrict the network. By contrast the primary derived network visually highlighted the need for manual duplication of some KEs to account for organ specificity. Prominent example is KE1739 (Binding to ACE2), which occurs in lung cells and is also the MIE in sustentacular cells and enterocytes. Accounting for tissue specificity was essential for the representation of systemic or cross tissue effects in the AOP network. As such, according to the AOP principles, KEs are not specific to tissue or cell types. These principles have been mainly driven by the practical consideration of reusing KEs to prevent the duplication of effort when defining those KEs. However, if the downstream effects of a KE are dependent on the cell-type, tissue, or organ where the KE occurs, it may be more appropriate to define individual KEs more narrowly and look for means of sharing information across the separate KEs. As more emphasis is placed on defining KEs using biological ontologies, the options for information sharing across KEs are dramatically improved. The need to account for organ specificity has also driven the CIAO Ontology Group to consider and recommend the combination of all Biological context fields in one table, allowing more than one row (cell, tissue or even subcellular location) per KE, within a single KE page.

Taken together, addressing such challenges would not only allow for the sustainability and re-use of the CIAO-developed AOPs, but also pave the way for the development of novel AOPs addressing emerging health issues that could benefit from the collaborative aspect of the AOPs.

4.2 Challenges and recommendations specific to viral AOP networks

4.2.1 Need to describe the SARS-CoV-2 specific infection steps

When modeling COVID-19 via AOPs, the stressor agnostic principle was challenged (Clerbaux et al., 2023). Depicting the biology of the virus was essential to capture the mechanisms underlying the disease onset and progression. Viral replication and viral load are well-established parameters correlated with poor prognosis of the disease (Brosseau et al., 2022) and most approved antivirals act on viral proteins specifically. Hence, stressor-specific KEs describing the replication cycle of SARS-CoV-2 were proposed to serve as a series of linked KEs, a reusable unit that can be integrated into the several AOPs that required SARS-CoV-2 replication to mediate the outcomes (Clerbaux et al., 2023). As such, only the initial three linked KEs (KE1738, KE 1901, KE 1847) are specific to SARS-CoV-2 while the downstream KEs in the different AOPs follow the stressor agnostic principle. In addition, the MIE (KE1739: Binding to ACE2) is stressor-agnostic which maintains the MIE open to inclusion of evidence from other stressors, such as SARS-Co-V or ACE2 modulators. In line, binding to ACE2 does not only lead to SARS-CoV-2 cell entry as exemplified with ACE2 dysregulation downstream event.

4.2.2 Discriminating physiological versus pathological inflammation in AOPs

The network showed that the diverse AOs are interrelated and share common signaling pathways and associated molecules, at the core of which is the inflammatory response, playing a central role in initiating a cascade of events locally and systemically spanning over several tissues and culminating in a variety of AOs. The complexity of the disease process and conflicting information on the timing and levels of expression of several pro/anti-inflammatory mediators, makes it challenging to draw a clear picture of the immune response onset and sequential progression towards severity. Many of these events occur in parallel, creating physiological havoc, whereby host defense mechanisms assume host destruction roles. In order to tentatively capture this temporal and quantitative aspect of the immune response, the network was organized visually into the four phases of the disease: disease onset (viral replication), early cellular and tissue damage; immune and vascular response and late adverse outcomes. In addition, to discriminate between physiological and pathological inflammation, the AO hyperinflammation (KE1868) was created which defines per se a pathological situation. However, this might not be sufficient and a quantification aspect of the AOPs might be needed along with a combination of relevant markers.

4.2.3 Utility of an AOP network for a viral disease

Responses to stressor exposure are more complex than the simple one-biological-perturbation to one-adverse-outcome portrayed in individual AOPs. AOP network can aid in understanding and in predicting effects of more realistic complex exposure scenarios. As for toxicology, such network underlined the gaps in our understanding of the pathways, which can guide future needed research. In the context of a viral disease, such a network could provide a structure to integrate multi-omics data from patients into a comprehensive understanding of the pathways at play, notably regarding the inflammatory response. The identification of potential biomarkers could also be of interest, especially for discriminating pathological inflammation. Such network can also serve to capture the influence of the modulating factors such as age, sex, pre-existing comorbidities in order to understand why some patients are more vulnerable than others (Clerbaux, Albertini, et al., 2022). However, such utilities have still to be operated. Finally, the AOP framework was inherently interdisciplinary which was needed to tackle such a complex disease and to support a collaborative large-scale crowdsourced effort (Carusi et al., 2023).

5 Conclusion

This study provided insight into the complex processes and interconnections between perturbed key biological events in COVID-19, as well as the reusability of the knowledge and framework from another field, e.g., toxicology. The study highlighted the challenges in developing an AOP network, such as the need to adopt globally the use of controlled vocabularies and ontologies, the need to consider organ specificity, the gaps in the individual AOPs, the quality of the current KEs and KERs and the challenges specific to applying this framework to a viral disease. The value and impact of the effort presented here stem from how the gathered knowledge about the disease is stored within AOPs in a manner that supports its reuse and uptake within other areas of research, including toxicology and any potential future pandemics caused by other types of pathogens. The current work demonstrates the potential of the AOP framework to support interdisciplinary research while lessons learnt aim to serve ongoing or future AOP-aligned initiatives.

Data availability statement

The original contributions presented in the study are included in the article/Supplementary Material, further inquiries can be directed to the corresponding authors.

Author contributions

PN: Conceptualization, Data curation, Methodology, Supervision, Writing–original draft, Writing–review and editing, Visualization, Validation. L-AC: Conceptualization, Data curation, Formal Analysis, Methodology, Project administration, Supervision, Writing–original draft, Writing–review and editing, Validation. M-JA: Conceptualization, Writing–review and editing. CA: Conceptualization, Visualization, Writing–review and editing, Validation. FB: Conceptualization, Writing–review and editing. GB: Conceptualization, Writing–review and editing, Validation. SC: Conceptualization, Writing–review and editing. FG: Conceptualization, Writing–review and editing. DJ: Conceptualization, Writing–review and editing. EL: Conceptualization, Visualization, Writing–review and editing. LM-C: Conceptualization, Writing–review and editing. MM: Conceptualization, Writing–review and editing. SM: Conceptualization, Writing–review and editing. HM: Writing–review and editing. YK: Conceptualization, Writing–review and editing. MS: Conceptualization, Writing–review and editing. ST: Conceptualization, Writing–review and editing. VV: Conceptualization, Writing–review and editing. SE: Conceptualization, Writing–review and editing. SH: Conceptualization, Data curation, Methodology, Writing–original draft, Writing–review and editing.

Funding

The author(s) declare that financial support was received for the research, authorship, and/or publication of this article. This work was supported by funding from the JRC Exploratory project CIAO (Modelling the Pathogenesis of COVID-19 using the Adverse Outcome Pathway Framework, https://www.ciao-covid.net/). DJ would like to acknowledge funding from the US National Science Foundation (EF-2133763). PN also acknowledges funding from The Swedish Fund for Research without Animals (grants F2021-0005 and F2022-0003). SH acknowledges funding received from Health Canada’s Genomics Research and Development Initiative. ST acknowledges funding received from Japan Agency for Medical Research and Development (AMED) Grant Number JP21mk0101216, JP22mk0101216, JP23mk0101216, Japan Society for the Promotion of Science (JSPS) KAKENHI Grant Number 21K12133.

Acknowledgments

The authors would like to acknowledge all CIAO partners for discussions supporting this work, especially (in no particular order) Jorid Sorli, Julija Filipovska, Anna Bal-Price, Francesca Pistollato, Jenny Waspe, Merlin Mei, Clemens Wittwehr. Authors also acknowledge Julie Buick and Dr Nikolai Chepelev of Health Canada for serving as internal reviewers and for their valuable comments.

Conflict of interest

Authors CA, EL and VV were employed by Biovista. Author GB was employed by Impact Station.

The remaining authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Supplementary material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fsysb.2024.1384481/full#supplementary-material

References

Brosseau, L. M., Escandón, K., Ulrich, A. K., Rasmussen, A. L., Roy, C. J., Bix, G. J., et al. (2022). Severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) dose, infection, and disease outcomes for coronavirus disease 2019 (COVID-19): a review. Clin. Infect. Dis. 75 (1), E1195–E1201. doi:10.1093/cid/ciab903

PubMed Abstract | CrossRef Full Text | Google Scholar

Carusi, A., Filipovska, J., Wittwehr, C., and Clerbaux, L.-A. (2023). CIAO: a living experiment in interdisciplinary large-scale collaboration facilitated by the adverse outcome pathway framework. Front. public health 11, 1212544. doi:10.3389/fpubh.2023.1212544

PubMed Abstract | CrossRef Full Text | Google Scholar

Clerbaux, L. A., Albertini, M. C., Amigó, N., Beronius, A., Bezemer, G. F. G., Coecke, S., et al. (2022). Factors modulating COVID-19: a mechanistic understanding based on the adverse outcome pathway framework. J. Clin. Med. 11 (15), 4464. doi:10.3390/jcm11154464

PubMed Abstract | CrossRef Full Text | Google Scholar

Clerbaux, L.-A., Amigó, N., Amorim, M. J., Bal-Price, A., Batista Leite, S., Beronius, A., et al. (2022). COVID-19 through adverse outcome pathways: building networks to better understand the disease - 3rd CIAO AOP design workshop. Altex 39 (2), 322–335. doi:10.14573/altex.2112161

PubMed Abstract | CrossRef Full Text | Google Scholar

Clerbaux, L. A., Filipovska, J., Nymark, P., Chauhan, V., Sewald, K., Alb, M., et al. (2023). Beyond chemicals: opportunities and challenges of integrating non-chemical stressors in adverse outcome pathways 41, 233–247. doi:10.14573/altex.2307061

PubMed Abstract | CrossRef Full Text | Google Scholar

Clerbaux, L. A., Fillipovska, J., Muñoz, A., Petrillo, M., Coecke, S., Amorim, M. J., et al. (2022). Mechanisms leading to gut dysbiosis in COVID-19: current evidence and uncertainties based on adverse outcome pathways. J. Clin. Med. 11 (18), 5400. doi:10.3390/jcm11185400

PubMed Abstract | CrossRef Full Text | Google Scholar

Clerbaux, L. A., Mayasich, S. A., Muñoz, A., Soares, H., Petrillo, M., Albertini, M. C., et al. (2022). Gut as an alternative entry route for SARS-CoV-2: current evidence and uncertainties of productive enteric infection in COVID-19. J. Clin. Med. 11 (19), 5691. doi:10.3390/jcm11195691

PubMed Abstract | CrossRef Full Text | Google Scholar

Da, E., Vogel, U., Hougaard, K. S., Pérez-Gil, J., Zuo, Y. Y., and Sørli, J. B. (2021). An adverse outcome pathway for lung surfactant function inhibition leading to decreased lung function. Curr. Res. Toxicol. 2 (May), 225–236. doi:10.1016/j.crtox.2021.05.005

PubMed Abstract | CrossRef Full Text | Google Scholar

Deshpande, R., and Zou, C. (2021). Programmed cell death in SARS-CoV-2 infection: a short review, 223–228.

CrossRef Full Text | Google Scholar

Gerloff, K., Landesmann, B., Worth, A., Munn, S., Palosaari, T., and Whelan, M. (2017). The adverse outcome pathway approach in nanotoxicology. Comput. Toxicol. 1, 3–11. doi:10.1016/j.comtox.2016.07.001

CrossRef Full Text | Google Scholar

Halappanavar, S., Sharma, M., Solorio-Rodriguez, S., Wallin, H., Vogel, U., Sullivan, K., et al. (2023). Substance interaction with the pulmonary resident cell membrane components leading to pulmonary fibrosis OECD Series on Adverse Outcome Pathways, Paris: OECD Publishing, 33. doi:10.1787/10372cb8-en

CrossRef Full Text | Google Scholar

Hogberg, H. T., Lam, A., Ohayon, E., Shahbaz, M. A., Clerbaux, L. A., Bal-Price, A., et al. (2022). The adverse outcome pathway framework applied to neurological symptoms of COVID-19. Cells 11 (21), 3411. doi:10.3390/cells11213411

PubMed Abstract | CrossRef Full Text | Google Scholar

Ives, C., Campia, I., Wang, R. L., Wittwehr, C., and Edwards, S. (2017) Creating a structured AOP knowledgebase via ontology-based annotations. Appl. In Vitro Toxicol.

Google Scholar

Kim, Y. J., Park, C. G., Lim, S. R., Jun, I., and Lee, Y. O. (2021). Advanced adverse outcome pathways potentially bridging pathogenesis of COVID-19. Available at: https://aopwiki.org/.

Google Scholar

Kinaret, P. A. S., Giudice, G. del, and Dario, G. (2020). Covid-19 acute responses and possible long term consequences: what nanotoxicology can teach us. Nano Today 35 (Dec), 100945. doi:10.1016/j.nantod.2020.100945

PubMed Abstract | CrossRef Full Text | Google Scholar

Kirtipal, N., Kumar, S., Kumar, S., Dhar, V., Gireesh Babu, K., Malý, P., et al. (2022). Understanding on the possible routes for SARS CoV-2 invasion via ACE2 in the host linked with multiple organs damage. Infect. Genet. Evol. 99 (February), 105254. doi:10.1016/j.meegid.2022.105254

PubMed Abstract | CrossRef Full Text | Google Scholar

Knapen, D., Angrish, M. M., Fortin, M. C., Katsiadaki, I., Leonard, M., Margiotta-Casaluci, L., et al. (2018). Adverse outcome pathway networks I: development and applications. Environ. Toxicol. Chem. 37 (6), 1723–1733. doi:10.1002/etc.4125

PubMed Abstract | CrossRef Full Text | Google Scholar

Li, X., Zhang, Z., Wang, Z., Gutiérrez-castrellón, P., and Shi, H. (2022). Cell deaths: involvement in the pathogenesis and intervention therapy of COVID-19. Signal Transduct. Target. Ther. 7 (4), 186. doi:10.1038/s41392-022-01043-6

PubMed Abstract | CrossRef Full Text | Google Scholar

Newman, M. E. J. (2003). The structure and function of complex networks. SIAM Rev. Soc. Ind. Appl. Math. 45 (2), 167–256. doi:10.1137/s003614450342480

CrossRef Full Text | Google Scholar

Nymark, P., Sachana, M., Leite, S. B., Sund, J., Krebs, C. E., Sullivan, K., et al. (2021). Systematic organization of COVID-19 data supported by the adverse outcome pathway framework. Front. Public Health 9 (1), 638605–638612. doi:10.3389/fpubh.2021.638605

PubMed Abstract | CrossRef Full Text | Google Scholar

OECD (2018). User’s handbook supplement to the guidance document for developing and assessing AOPs. Available at: http://www.oecd.org/officialdocuments/publicdisplaydocumentpdf/?cote=env/jm/mono(2016)12&doclanguage=en.

Google Scholar

Shahbaz, M. A., De Bernardi, F., Alatalo, A., Sachana, M., Clerbaux, L. A., Muñoz, A., et al. (2022). Mechanistic understanding of the olfactory neuroepithelium involvement leading to short-term anosmia in COVID-19 using the adverse outcome pathway framework. Cells 11 (19), 3027. doi:10.3390/cells11193027

PubMed Abstract | CrossRef Full Text | Google Scholar

Svingen, T., Villeneuve, D. L., Knapen, D., Panagiotou, E. M., Draskau, M. K., Damdimopoulou, P., et al. (2021). A pragmatic approach to adverse outcome pathway development and evaluation. Toxicol. Sci. 184 (2), 183–190. doi:10.1093/toxsci/kfab113

PubMed Abstract | CrossRef Full Text | Google Scholar

Tojo, K., Yamamoto, N., Tamada, N., Mihara, T., Abe, M., Nishii, M., et al. (2023). Early alveolar epithelial cell necrosis is a potential driver of COVID-19-induced acute respiratory distress syndrome. ISCIENCE 26 (1), 105748. doi:10.1016/j.isci.2022.105748

PubMed Abstract | CrossRef Full Text | Google Scholar

Villeneuve, D. L., Angrish, M. M., Fortin, M. C., Katsiadaki, I., Leonard, M., Margiotta-Casaluci, L., et al. (2018). Adverse outcome pathway networks II: network analytics. Environ. Toxicol. Chem. 37 (6), 1734–1748. doi:10.1002/etc.4124

PubMed Abstract | CrossRef Full Text | Google Scholar

Villeneuve, D. L., Crump, D., Garcia-Reyero, N., Hecker, M., Hutchinson, T. H., LaLone, C. A., et al. (2014). Adverse outcome pathway (AOP) development I: strategies and principles. Toxicol. Sci. 142 (2), 312–320. doi:10.1093/toxsci/kfu199

PubMed Abstract | CrossRef Full Text | Google Scholar

Villeneuve, D. L., Landesmann, B., Allavena, P., Ashley, N., Bal-Price, A., Corsini, E., et al. (2019). Representing the process of inflammation as key events in adverse outcome pathways. Adv. Ecol. Res. 60, 1–24. doi:10.1093/toxsci/kfy047

PubMed Abstract | CrossRef Full Text | Google Scholar

Vinken, M. (2021). COVID-19 and the liver: an adverse outcome pathway perspective. Toxicology 455 (March), 152765. doi:10.1016/j.tox.2021.152765

PubMed Abstract | CrossRef Full Text | Google Scholar

Wiklund, L., Caccia, S., Pípal, M., Nymark, P., and Beronius, A. (2023). Development of a data-driven approach to adverse outcome pathway network generation: a case study on the EATS-modalities. Front. Toxicol. 5 (5), 1183824–1183912. doi:10.3389/ftox.2023.1183824

PubMed Abstract | CrossRef Full Text | Google Scholar

Wittwehr, C., Amorim, M. J., Clerbaux, L. A., Krebs, C., Landesmann, B., Macmillan, D. S., et al. (2021). Understanding COVID-19 through adverse outcome pathways – 2nd CIAO AOP design workshop. Altex 38 (2), 351–357. doi:10.14573/altex.2102221

PubMed Abstract | CrossRef Full Text | Google Scholar

Wittwehr, C., Clerbaux, L. A., Edwards, S., Angrish, M., Mortensen, H., Carusi, A., et al. (2024). Why adverse outcome pathways need to Be FAIR 41(1), 50–56. doi:10.14573/altex.2307131

CrossRef Full Text | Google Scholar

Yuan, C., Ma, Z., Xie, J., Li, W., Su, L., Zhang, G., et al. (2023). The role of cell death in SARS-CoV-2 infection. Signal Transduct. Target. Ther. 8, 357. doi:10.1038/s41392-023-01580-8

PubMed Abstract | CrossRef Full Text | Google Scholar