Post hoc explanations for RNNs using state transition diagrams for time series data

In the age of pervasive Artificial Intelligence (AI), the black-box nature of deep neural networks, especially Recurrent Neural Networks (RNNs), remains a significant concern, particularly in domains necessitating transparent decision-making. While current explainability and interpretability methods offer insights into model decisions, many treat deep networks as obscure entities, with explanations remaining largely model-agnostic. This study pivots towards the domain of time series analysis, where RNNs have demonstrated prowess due to their capacity to handle sequential data. Yet, their intrinsic complexity poses interpretability challenges, making real-world adoption hesitant. Given the European Union’s General Data Protection Regulation (GDPR) mandates around interpretability, there’s an intensified need to demystify such models. Our research introduces an innovative approach, leveraging deterministic finite state automata (FSA) to shed light on RNN’s internal workings. By mapping activated internal states of RNNs onto FSAs, we aim to provide a more interpretable and human-understandable representation. Preliminary methodologies involve the utilization of the k-means clustering algorithm to derive FSAs from various RNN architectures. Our findings could pave the way for more transparent AI deployments, especially in critical sectors requiring a clear understanding of AI decision-making processes.