Automata Learning of Preferences over Temporal Logic Formulas from Pairwise Comparisons

TL;DR

This paper formalizes the properties of characteristic samples and develops an algorithm that guarantees to learn, given a characteristic sample, the minimal PDFA equivalent to the true PDFA from which the sample is drawn.

Abstract

Many preference elicitation algorithms consider preference over propositional logic formulas or items with different attributes. In sequential decision making, a user's preference can be a preorder over possible outcomes, each of which is a temporal sequence of events. This paper considers a class of preference inference problems where the user's unknown preference is represented by a preorder over regular languages (sets of temporal sequences), referred to as temporal goals. Given a finite set of pairwise comparisons between finite words, the objective is to learn both the set of temporal goals and the preorder over these goals. We first show that a preference relation over temporal goals can be modeled by a Preference Deterministic Finite Automaton (PDFA), which is a deterministic finite automaton augmented with a preorder over acceptance conditions. The problem of preference inference reduces to learning the PDFA. This problem is shown to be computationally challenging, with the problem of determining whether there exists a PDFA of size smaller than a given integer $k$, consistent with the sample, being NP-Complete. We formalize the properties of characteristic samples and develop an algorithm that guarantees to learn, given a characteristic sample, the minimal PDFA equivalent to the true PDFA from which the sample is drawn. We present the method through a running example and provide detailed analysis using a robotic motion planning problem.

Figures (11)

• Figure 1: A Garden example in which the robot is tasked with pollinating the flowers of the garden.
• Figure 2: A pdfa with states $\{00,10,01,11\}$ and alphabet $\Sigma = \{a,b\}$ with the ranking function presented by a directed graph (right).
• Figure 3: A preference sample for alphabet $\Sigma = \{a, b\}$. The lacking of an edge between two words means the sample does not include any comparisons between those two words.
• Figure 4: Top) The indifference graph for the sample in Figure \ref{['fig:sample']}. Bottom) The partition $O_S$ and the relation $R_s$. .
• Figure 5: Steps of our algorithm for pnfa inferring for the sample in Figure \ref{['fig:sample']}. The partial order learned by our algorithm is shown in Figure \ref{['fig:learn_pref_graph']}.

Theorems & Definitions (31)

• Definition 1
• Definition 2
• Definition 3: pdfa extended from rahmani2023probabilistic
• Example 1
• Definition 4
• Definition 5
• Example 2: cont.
• Lemma 1
• Proposition 1
• Theorem 1