Neuro-Symbolic Temporal Point Processes

TL;DR

The paper tackles explainability in irregular-event modeling by introducing NS-TPP, a neural-symbolic temporal point process that induces temporal logic rules in a differentiable framework. It represents predicates and rules as embeddings, uses a sequential covering algorithm to grow the rule set, and grounds rules to construct a differentiable intensity function $\lambda^*(t)$ that is learned via maximum likelihood. Key contributions include a unified neural-symbolic feature construction, robustness to noise through embedding-based grounding, and significant efficiency and accuracy gains over state-of-the-art baselines on synthetic and real datasets (e.g., substantial speedups and improved rule recovery). The work has practical implications for high-stakes domains like healthcare and autonomous systems, offering interpretable explanations for complex event dynamics while maintaining strong predictive performance.

Abstract

Our goal is to $\textit{efficiently}$ discover a compact set of temporal logic rules to explain irregular events of interest. We introduce a neural-symbolic rule induction framework within the temporal point process model. The negative log-likelihood is the loss that guides the learning, where the explanatory logic rules and their weights are learned end-to-end in a $\textit{differentiable}$ way. Specifically, predicates and logic rules are represented as $\textit{vector embeddings}$, where the predicate embeddings are fixed and the rule embeddings are trained via gradient descent to obtain the most appropriate compositional representations of the predicate embeddings. To make the rule learning process more efficient and flexible, we adopt a $\textit{sequential covering algorithm}$, which progressively adds rules to the model and removes the event sequences that have been explained until all event sequences have been covered. All the found rules will be fed back to the models for a final rule embedding and weight refinement. Our approach showcases notable efficiency and accuracy across synthetic and real datasets, surpassing state-of-the-art baselines by a wide margin in terms of efficiency.

Figures (4)

• Figure 1: Overview of our neural-symbolic framework for temporal logic induction. The framework begins with preparing fixed predicate embeddings. During the forward pass, rule embeddings scan these predicate embeddings to identify optimal compositional matches. A modified attention-like block then integrates these matches with observed events to generate static neural-symbolic features. Temporal relations are then incorporated to create neural-symbolic temporal features given the already selected predicate pairs. We will use the same matching idea to obtain the temporal neural-symblic features. The final neural-symbolic features by combing static and temporal parts will be used to compute the intensity function and the likelihood. We will adopt MLE to learn all the rule embedding parameters and other continuous model parameters (rule weights and base term) in a differentiable way. The overall rule learning scheme employs the sequential covering algorithm to learn rules progressively until no further rules can be added. Finally, all the rule embeddings and other model parameters will be refined to optimize the likelihood.
• Figure 2: Performance on Different Datasets at Various Repetition Times. The upper panel showcases the accuracy achieved by different groups within varying sample sizes for each repetition. The lower panel details the time efficiency across datasets, noting that time measurements, presented in seconds, have been log-normalized for clarity.
• Figure 3: Comparison Results of Running Time and Accuracy with TELLER and CLUSTER.
• Figure 4: The MAE and Variance of learned rules' weight