Local-NPDR: A Novel Variable Importance Method for Explainable Machine Learning and False Discovery Diagnosis for Ocean Worlds Biosignatures

Lily A Clough¹Jonathan Major²Lauren M Seyler³Victoria Da Poian⁴Bethany Theiling⁴Brett A McKinney^1*

• ¹Computer Science, University of Tulsa, Tulsa, OK, United States
• ²University of South Florida, Tampa, United States
• ³Stockton University, Galloway, United States
• ⁴NASA Goddard Space Flight Center, Greenbelt, United States

Explainable machine learning (ML) is important for biosignature prediction on future astrobiology missions to minimize the risk of false positives due to geochemical biotic mimicry and false negatives due to environmental factors that mask biosignatures. ML models often use feature importance scores to provide insights into model prediction mechanisms by quantifying each variable's contribution to the prediction. Global variable importance methods aggregate information across all training samples and therefore do not provide interpretation for the classification of a single sample. In contrast, local variable importance scores quantify the contribution of variables to the classification of a single sample and can therefore help explain why the sample was predicted to be in a certain class and diagnose whether it is a false prediction. We present a new local variable importance method that handles nonlinearity, statistical interactions, and includes penalized feature selection. Our approach represents a local version of Nearest-neighbor Projected Distance Regression (NPDR) feature selection. We evaluate local-NPDR on complex simulated data and real data from a study of carbon and oxygen isotopic biosignatures using laboratory-generated ocean world analogue brines. The ability of local-NPDR to differentiate between true and false predictions is compared with other common local importance methods. To diagnose individual false predictions, we use the concordance between global-and local-NPDR scores, allowing the mechanisms of a true or false prediction to be explained. We illustrate the capacity of local-NPDR to integrate scientific explanations of single-sample ML predictions to support a more comprehensive framework for biosignature detection.