Diagnosis and classification of portosystemic shunts: a machine learning retrospective case-control study

Diagnosis of portosystemic shunts (PSS) in dogs often requires multiple diagnostic tests, and available clinicopathologic tests have limitations in sensitivity and specificity. The objective of this study was to train and validate a machine learning model (MLM) that can accurately predict the presence of a PSS utilizing routinely collected demographic data and clinicopathologic features. Dogs diagnosed with PSS or control dogs tested for PSS but had the condition ruled out (non-PSS) were identified. Dogs were included if a complete blood count and serum chemistry panel were available from PSS diagnostic testing. Dogs with a PSS were subcategorized as having a single intrahepatic PSS, a single extrahepatic PSS, or multiple extrahepatic PSS. An extreme gradient boosting (XGboost) MLM was trained with data from 70% of the cases, and MLM performance was determined on the test set, comprising the remaining 30% of the case data. Two MLMs were created. The first was designed to predict the presence of any PSS (PSS MLM), and the second to predict the PSS subcategory (PSS SubCat MLM). The trained PSS MLM had a sensitivity of 94.3% (95% CI 90.1–96.8%) and specificity of 90.5% (95% CI 85.32–94.0%) for dogs in the test set. The area under the receiver operator characteristic curve (AUC) was 0.976 (95% CI; 0.964–0.989). The mean corpuscular hemoglobin, lymphocyte count, and serum globulin concentration were most important in prediction classification. The PSS SubCat MLM had an accuracy of 85.7% in determining the subtype of PSS of dogs in the test set, with variable sensitivity and specificity depending on PSS subtype. These MLMs have a high accuracy for diagnosing PSS; however, the prediction of PSS subclassification is less accurate. The MLMs can be used as a screening tool to increase or decrease the index of suspicion for PSS before confirmatory diagnostics such as advanced imaging are pursued.


Introduction
Portosystemic shunts (PSS) are vascular anomalies that allow blood within the portal circulation to bypass the liver and enter the systemic circulation.This results in the clinical syndrome of hepatic encephalopathy, reduced metabolization of hepatically cleared medications, and accumulation of blood ammonia that can result in urate urinary calculi.The clinical presentation of dogs with PSS is variable and can range from subclinical to a severe, life-threatening condition.The overall incidence of congenital PSS is 0.18% (1).However, some breeds are overrepresented, with around 3% of Havanese and Yorkshire terriers being diagnosed with the condition (1)(2)(3).These vascular anomalies are subcategorized as congenital due to inappropriate attenuation of fetal vasculature or acquired when they arise due to portal hypertension.Congenital PSS are further subclassified into extrahepatic PSS, more commonly diagnosed in toy breed dogs, and intrahepatic PSS, more commonly diagnosed in large breed dogs (4,5).
The clinical presentation of dogs with PSS can be vague and variable, with signs including failure to thrive, gastrointestinal signs, or neurologic signs (6,7).Clinical signs could easily be attributed to other disease processes, making recognizing this condition challenging in some cases.Other hepatic diseases, including portal venous hypoplasia (PVH) and chronic hepatitis, can mimic the clinical signs and clinicopathologic findings often associated with PSS, such as anemia, decreased blood urea nitrogen concentrations, elevated liver enzymes, hypoalbuminemia, and hypoglycemia (5, [7][8][9][10]. Diagnostic tools to discriminate between PSS and other disease conditions include measurement of paired fasting and post-prandial serum bile acid concentrations and abdominal imaging studies.Elevated serum bile acid concentrations are a sensitive marker for hepatic insufficiency but cannot differentiate between PSS and other pathology, such as PVH (11).Ultrasonographic identification of PSS requires advanced technical training, and sensitivity depends on the operator's skill level (12,13).Additional imaging tests such as nuclear scintigraphy and computed tomography may be needed to confirm or exclude the presence of a shunt, and dogs often undergo a series of tests for a final diagnosis.Other biomarkers have been investigated to help discriminate between PSS and other liver pathologies, including serum ammonia concentrations protein C; however, these assays lack specificity for diagnosing a PSS (14, 15).
Machine learning techniques have been increasingly utilized to enhance diagnostic and predictive capabilities in the veterinary field.Some previous applications of machine learning in veterinary medicine include the diagnosis of hypoadrenocorticism, leptospirosis, and babesiosis in dogs, as well as lameness in cows (16)(17)(18)(19).Predictable patterns in clinicopathologic data have been previously observed in dogs with PSS (6,7,20).This study aimed to evaluate a gradient-boosted tree machine learning model (MLM) as a predictive tool to detect PSS in dogs utilizing demographic features and features from routinely conducted blood work, the complete blood count and serum biochemistry panel.We hypothesize that MLMs when trained with clinicopathologic data from dogs with PSS or dogs tested for PSS that had the disease ruled out, can accurately discriminate between dogs that have a PSS and those that have non-PSS disease.Further, we hypothesize that MLMs can accurately classify dogs based on the subtype of PSS.

Patient selection and ground truth determination
A retrospective case-control study, selecting dogs from the electronic medical record system of the University of California-Davis Veterinary Medical Teaching Hospital (VMTH) was performed.Records were searched from 2000 to 2020 to identify potential cases (PSS group) and controls (non-PSS group).
Dogs with PSS were identified by (1) searching for "shunt" or "PSS" in the clinical diagnosis field (2) searching for dogs with advanced imaging procedures of the abdomen (contrast CT scan, dual-phase contrast CT scan, nuclear scintigraphy of the liver) or (3) searching for dogs that underwent a PSS related surgical procedure (percutaneous intrahepatic shunt coil embolization procedure, intrahepatic shunt surgical repair, multiple or single extrahepatic shunt attenuation, or laparoscopic liver biopsy).After a medical record review (AZ), dogs were included as cases if PSS were identified on ≥1 imaging modality or during surgical intervention.If conflicting results were obtained regarding the presence of a PSS on multiple imaging modalities, the dog was excluded.Dogs were subcategorized in this group based on imaging into three groups; single intrahepatic portosystemic shunt, single extrahepatic portosystemic shunt, or multiple extrahepatic shunts based on imaging studies or surgical exploration results based on a wholistic medical record review (AZ).
Approximately equal number of dogs with non-PSS disease were identified as controls by searching for dogs that had postprandial serum bile acid concentrations measured (Veterinary Diagnostic Laboratory, University of California-Davis).Medical records were reviewed by a single author (AZ), and dogs were included if a PSS was not identified on at least one of abdominal ultrasound, contrast CT scan, dual-phase contrast CT scan, or transrectal nuclear scintigraphy of the liver.The clinical diagnosis of the non-PSS dogs was reviewed by a single author (KR), and the primary clinical diagnosis was categorized into the following categories: gastrointestinal (including non-PSS hepatic or pancreatic disease), cardiac, immune-mediated, genitourinary, neurologic, respiratory, infectious, neoplasia, trauma and primary surgical, or undetermined.
To be included, PSS and non-PSS dogs had to have a complete blood count (Advia 120; Siemens) and serum chemistry panel (Hitachi 917 years 2000-2009; Cobas c501c/6,000 years 2009-2020) performed (Veterinary Diagnostic Laboratory, University of California-Davis) contemporaneously with the diagnostic investigation of PSS and before surgical intervention for PSS attenuation for dogs with congenital PSS.Medical records for PSS and non-PSS dogs were reviewed, and breed, sex, age (days), weight (kg), results of complete blood count, and serum biochemistry at the time of diagnosis at the VMTH were recorded.Results of abdominal imaging, surgical interventions performed, and results of liver histopathology were also recorded.

Feature preprocessing
Serum bilirubin and gamma-glutamyltransferase have a lower level of detection >0.Values below the detection limit were recorded at 0.01 below the lower end of assay detection.Dog breeds were grouped into toy, herding, hound, non-sporting, sporting, terrier, working, foundation stock service, mix breed, or other.One-hot encoding was utilized for categorical variables.

Machine learning model training
Classification models were trained with patient features and clinicopathologic parameters (Table 1).The data set was randomly split into a 70% training set and a 30% test set and two models were trained, one that predicts the presence of any PSS (PSS MLM) and a second that predicts the subclassification of PSS (PSS SubCat MLM).First, a gradient boosted tree (xgboost 1.7.6,Python3) was trained to discriminate between PSS and non-PSS dogs (PSS MLM) (21).The model was trained using 10-fold crossvalidation, and hyperparameters were tuned using a Bayesian search algorithm optimized based on AUC-ROC.The tuned hyperparameters included gamma, learning rate, maximum depth, the number of estimators, regularization alpha, and regularization lambda (Supplementary Table S1).A second gradient boosted tree (PSS SubCat MLM) was trained to classify animals into the following categories: single intrahepatic portosystemic shunt, single extrahepatic portosystemic shunt, multiple extrahepatic shunts, or non-PSS disease.Due to class imbalance, the synthetic minority over-sampling technique (SMOTE) from imblearn (0.10.1) library was applied to the PSS SubCat MLM training data set (22,23).Model code utilized to build these models is publicly available (https://github.com/MakanFar/pss_classification).

Machine learning model performance evaluation and statistical analysis
The 30% test set was used to evaluate MLM performance.Model prediction results are reported as sensitivity and specificity compared to the labeled classification.Receiver operator characteristic (ROC) plots were generated, and the area under the curve (AUC) was calculated.For the PSS SubCat model, a one-versus-rest multiclass ROC was utilized.The 95% confidence intervals (95% CI) of sensitivity, specificity, and AUC were calculated using the Wilson-Brown method (Prism v.9.2.0; GraphPad).Population characteristics were tested for normality.They did not meet normality criteria, and are presented as the median and interquartile range.Comparisons between continuous clinicopathologic variables were performed with a Wilcoxon-Rank Sum test, and p-values were adjusted for multiple comparisons using a Bonferroni correction.
Feature importance in the PSS MLM was assessed using the gain metric.Features with a high score were considered to have higher gain and, therefore, more impact on the MLM prediction.

Study population demographics
In the PSS group, 1,149 dogs were assessed for eligibility.Of these, 274 were excluded because of a lack of definitive diagnosis of a PSS.From the remaining 875 cases, 223 were excluded for lacking one or both of CBC and serum biochemistry tests.A total of 652 dogs with PSS were included.Of these dogs, 421 had a single extrahepatic shunt, 175 had a single intrahepatic shunt, and 56 had multiple extrahepatic shunts.Table 2 summarizes the method of PSS diagnosis for dogs in this group.In the non-PSS group, 969 dogs were assessed for eligibility.One hundred and fifty were excluded for lack of confident exclusion of a portosystemic shunt.A further 230 were excluded for lacking one or both of CBC and serum biochemistry tests, leaving 589 control dogs for analysis.Dogs in the control group had a PSS ruled out by nuclear scintigraphy in 66, computed tomography in 1, and the remaining 522 dogs had a PSS ruled out with abdominal ultrasound.Some dogs had multiple diagnostics performed (Table 2).Within the non-PSS group, the category of clinical diagnosis consisted of 228 dogs with GI disease, 171 with neurologic disease, 54 with neoplasia, 35 with genitourinary disease, 32 with immune-mediated disease, 12 with respiratory disease, 9 with infectious, 7 with cardiac disease, 1 with primary surgical disease, and 40 with an undetermined disease process.Seven of the dogs with GI diseases were diagnosed with PVH after a review of the histopathology of liver biopsy.
The breed group distribution was significantly different between the PSS control group (p < 0.0001), with more toy breed dogs represented in the PSS group (Table 3).Dogs in the PSS group were younger (p < 0.0001) and had a smaller body weight (p < 0.0001) compared to control dogs.There was no difference in sex distribution between groups (Table 3).Weight measurements were missing for 26 (4.3%)PSS dogs and 21 (9.2%) non-PSS dogs.Age was missing for 4 (0.7%) PSS dogs and 21 (3.6%) non-PSS dogs.

Clinicopathologic findings
Clinicopathologic differences between dogs with PSS and non-PSS control dogs are summarized in Table 4. Individual clinicopathologic values that were missing in >2% of cases included total calcium missing in 274 dogs (22.1%) and red blood cell distribution width in 47 dogs (3.8%).

PSS prediction
The PSS MLM classification model predicted a diagnosis of a PSS correctly in 345/373 (92.5%) dogs in the test set, correctly classifying 183/194 dogs within the PSS group, yielding a sensitivity of 94.3% (95% CI 90.1-96.8%)(Table 5).The classifier correctly classified 162/179 of the non-PSS dogs as not having a PSS, resulting in a specificity of 90.5% (95% CI 85.32-94.0%).The positive and negative likelihood ratios are 9.9 and 0.07, respectively.This model has an AUC of 0.976 (95% CI; 0.964-0.989)(Figure 1).Of the features utilized to train the PSS classification model, MCH, lymphocyte count, and serum globulin concentration were identified as the most important predictors of classification (Figure 2).

Discussion
MLMs trained with routinely collected patient data can accurately predict the presence and subtype of PSS.These MLMs can identify patterns with dog signalment, hematologic, and biochemical parameters that differentiate between PSS and non-PSS disease with an accuracy similar to traditional methods of PSS detection, including ultrasonography, serum post-prandial bile acids, and plasma ammonia concentrations (8,12,24).Application of these MLMs may provide actionable diagnostic information utilizing readily available laboratory parameters that can augment clinical decision-making.
Measurement of post-prandial serum bile acids or plasma ammonia concentrations are commonly used biomarkers to screen for PSS in dogs.Post-prandial serum bile acids are a sensitive marker of PSS (92-93%), but specificity (67%) is lacking (8,24).Elevated plasma ammonia concentrations have a sensitivity of 85-100% and a specificity of 84-89% (8,24).Limitations to applications of these assays include the need for a 12 h fast before performing a bile acids test, the liable nature of ammonia, the need for reference laboratory testing, and the associated costs.Compared to these biomarkers, the MLM trained to detect the presence or absence of a PSS has equivalent sensitivity (94%) with superior specificity (90%), without the practical limitations of biomarker testing.Further, serum bile acids and plasma ammonia concentration testing cannot differentiate between subcategories of PSS.In contrast, a second  Feature importance for the MLM was determined in this study.Understanding the most contributive features allows for model transparency, confidence during clinical decision-making, and may expose physiologic features of PSS that were not previously recognized (25).The most contributive features in this study were MCH, lymphocyte count, and serum globulin concentrations.
The MCH is a calculated value representing the hemoglobin amount per red blood cell.A low MCH has been previously associated with the presence of PSS (26,27).It has been suggested that this may be due to impaired iron utilization and abnormal iron metabolism in dogs with PSS.MCH increases and decreases proportionally to cell volume, and indeed dogs with PSS in this group had lower MCV than non-PSS dogs; however, MCH was more strongly associated with the presence or absence of PSS than MCV.This indicates that MCH is impacted by factors beyond cell size in dogs with PSS.In people, MCH is an independent predictor of iron deficiency, length of hospital stay of people with acute pancreatitis, and premature discontinuation of antiplatelet therapy in people receiving cardiovascular stents indicating this hematologic parameter provides additional information to the more traditionally utilized hemoglobin and MCV (28)(29)(30).
Dogs with PSS in our study had higher lymphocyte counts than dogs without PSS.The mechanism of difference is unknown, but normal to high lymphocytes have been noted previously in dogs and cats with PSS (31,32).This may reflect age, as the PSS group was younger than the non-PSS group, and lymphocyte counts decrease with age (33).Similarly, serum globulin concentration was associated with PSS status, but may reflect changes in age in the population as globulin concentrations increase with age (34).However, these findings were independent of age in our model, indicating other mechanisms may be contributing.
The MLMs presented here were trained on dogs with a high pre-test probability of a PSS.All dogs in non-PSS group had postprandial bile acid concentrations determined, signifying the attending clinician suspected hepatic insufficiency, PSS, or some other hepatic disease.Therefore, the application of these MLMs would only be appropriate for dogs with a high suspicion of PSS rather than as a screening tool broadly applied to a population with a low pre-test probability of a PSS.This study's control population included dogs tested for a PSS with post-prandial serum bile acid concentrations but had the disease ruled out with at least one imaging modality.This includes many dogs that had a PSS ruled out using an abdominal ultrasound rather than the gold standard, nuclear scintigraphy.The sensitivity and specificities for the detection of PSS with abdominal ultrasound varies widely, ranging from 47 to 85% and 67 to 100%, respectively (6,12,13,35).In our institution, highly skilled ultrasonographers interpreting the study, and the holistic assessment of the patient by the attending clinician refuted the presence of a PSS.However, a limitation of this study is the lack of definitive diagnosis in many non-PSS dogs.Additionally, the non-PSS dogs represent at heterogenous group of disease processes as demonstrated by the varied categories of disease in this group, but was predomidated by dogs with primary gastrointestinal disease or neurologic disease.This highlights the ability of this model to identify dogs with PSS in dogs with varied clinical presentations.
A further limitation of this study is the unbalanced data set and the limited number of dogs with some subsets of PSS.Most dogs with PSS were dogs with a single, extrahepatic PSS, followed by dogs with a single, intrahepatic PSS, and the fewest number with multiple, acquired, extrahepatic shunts, consistent with the epidemiology of the disease (20,36).To address this, a method of over-sampling categories with fewer dogs was utilized that can enrich the data set (22,23).Future studies should include larger groups of dogs with acquired, extrahepatic shunts to enrich the training set used to subcategorize types of PSS.These MLMs are trained with blood work performed from a single clinical laboratory over a 20 year time period.This represents a wide period of time necessary to amass the patient numbers necessary to train MLM, yet also introduces limitations, including variations in clinicopathologic methodologies over this period.Indeed we noted that some dogs had missing clinicopathologic variables that were not included on the blood work panels during some time periods.However, we utilized models in this study designed to handle missing values in features without impacting performance (37,38).Future studies should include clinicopathologic data from multiple clinical laboratories with varied instrumentation to determine the generalizability of these models.
The project adopts an open-source framework to facilitate the integration of this method into clinical workflows.This design   In conclusion, MLMs can provide accurate prediction of the presence of PSS and subcategories of disease using routinely collected clinicopathologic and signalment data.These MLMs have similar sensitivity to traditional biomarkers used for detecting PSS in dogs with improved specificity.

FIGURE 1
FIGURE 1Receiver operator characteristic (ROC) curve for the portosystemic shunt machine learning model on the test set data.

FIGURE 2 Feature
FIGURE 2 Feature importance for machine learning model.All features used for model training and the corresponding score that represents the gain and importance in model prediction are listed from those with highest importance to those with the lowest importance.Alanine Transaminase (ALT), Aspartate Transaminase (AST), Alkaline Phosphatase (ALP), Blood Urea Nitrogen (BUN), Gamma-glutamyl Transferase (GGT), Mean Corpuscular Hemoglobin (MCH), Mean Corpuscular Hemoglobin Concentration (MCHC), Mean Corpuscular Volume (MCV), Mean Platelet Volume (MPV), Red Blood Cell Distribution Width (%).

TABLE 1
Model features utilized to predict presence of portosystemic shunt.

TABLE 2
Method of PSS status determination.

TABLE 3
Demographic description of dogs with and without PSS.
MLM was trained to predict the subcategory of PSS and had an overall accuracy of 86%.This study did not include a comparison of serum bile acid concentrations, plasma ammonia concentrations, and MLM performance, as many of the PSS dogs did not have this biomarker testing performed at our institution.

TABLE 4
Clinicopathologic findings of dogs with and without PSS.

TABLE 5
Machine learning model prediction compared to the ground truth of dogs in the test set.
choice promotes transparency and allows collaborative enhancements, where clinicians and developers can explore the codebase.The proposed non-invasive diagnosis solution could be integrated into electronic medical record systems, allowing clinicians to utilize these clinical decision-support tools in real time.