Finding the Needle in the Haystack -An Interpretable Sequential Pattern Mining Method for Classification Problems

Alexander Grote^*Anuja HariharanChristof Weinhardt

• Karlsruhe Institute of Technology (KIT), Karlsruhe, Germany

The analysis of discrete sequential data, such as event logs and customer clickstream data, is hindered by the vast number of possible sequential patterns, making it difficult to identify meaningful sequences and extract valuable insights. To address this challenge, we propose a novel feature selection algorithm that combines unsupervised sequential pattern mining with supervised machine learning to identify and correlate meaningful sequences with classification objectives. Unlike existing approaches in interpretable machine learning, our algorithm determines important sequential patterns during the mining process, eliminating the need for subsequent classification to estimate their importance, and compared to existing interesting measures utilised in significant pattern mining, we provide an inherently interpretable and local classspecific interesting measure r1 . We evaluate our algorithm on three diverse datasets, including churn prediction, malware sequence analysis, and a synthetic dataset, which vary in size, application area, and feature complexity. Our results show that our algorithm achieves comparable classification performance to existing feature selection algorithms while maintaining interpretability and reducing computational resources. Our work provides a practical and efficient approach for practitioners to uncover important sequences in classification problems, offering an alternative to existing interpretable machine learning solutions and paving the way for new research opportunities in sequential data analysis.