Lower Bounds for Greedy Teaching Set Constructions

TL;DR

The paper investigates the limits of the natural greedy teaching-set construction for bounding the best-case teaching dimension $\operatorname{TS}_{\min}$ of concept classes with finite VC dimension $d$, addressing the Recursive Teaching Dimension conjecture. It proves a tight lower bound for $k=1$ via a rectangle-based construction, showing Greedy$(\cdot,1)$ yields $\Omega(\log(|\mathcal{C}|))$-sized teaching sets. For larger $k$, the authors extend the lower-bound phenomenon to $k \ge 2$, demonstrating $\Omega(\log(\log(|\mathcal{C}|)))$ lower bounds using a product-structured class with head/tail components, and further show these lower bounds persist up to $k \le \lceil c d \rceil$ for a small constant $c>0$. Together, the results suggest that resolving $\operatorname{TS}_{\min} = O(d)$ may require higher-order interactions or global class structure beyond the basic greedy approach. This work thus delineates a phase transition in the greedy method’s effectiveness and motivates alternative strategies for achieving near-linear dependence on $d$ in teaching dimension bounds.

Abstract

A fundamental open problem in learning theory is to characterize the best-case teaching dimension $\operatorname{TS}_{\min}$ of a concept class $\mathcal{C}$ with finite VC dimension $d$. Resolving this problem will, in particular, settle the conjectured upper bound on Recursive Teaching Dimension posed by [Simon and Zilles; COLT 2015]. Prior work used a natural greedy algorithm to construct teaching sets recursively, thereby proving upper bounds on $\operatorname{TS}_{\min}$, with the best known bound being $O(d^2)$ [Hu, Wu, Li, and Wang; COLT 2017]. In each iteration, this greedy algorithm chooses to add to the teaching set the $k$ labeled points that restrict the concept class the most. In this work, we prove lower bounds on the performance of this greedy approach for small $k$. Specifically, we show that for $k = 1$, the algorithm does not improve upon the halving-based bound of $O(\log(|\mathcal{C}|))$. Furthermore, for $k = 2$, we complement the upper bound of $O\left(\log(\log(|\mathcal{C}|))\right)$ from [Moran, Shpilka, Wigderson, and Yuhudayoff; FOCS 2015] with a matching lower bound. Most consequentially, our lower bound extends up to $k \le \lceil c d \rceil$ for small constant $c>0$: suggesting that studying higher-order interactions may be necessary to resolve the conjecture that $\operatorname{TS}_{\min} = O(d)$.

Figures (2)

• Figure 1: The arrangement of the point sets $V_1 \cup V_2 \cup \dots$ in the two-dimensional plane. The black horizontal dashed lines delineate the "vertical ranges" of these sets. The red rectangles denote concepts in $\mathcal{C}_i^{(\text{up, 1})}$ and $\mathcal{C}_i^{(\text{down, 1})}$, whereas the blue rectangles denote concepts in $\mathcal{C}_i^{(\text{up, 2})}$ and $\mathcal{C}_i^{(\text{down, 2})}$. For example, observe how the red rectangle in Level $i$, which belongs to $\mathcal{C}_i^{(\text{up, 1})}$, is enlarged to include all the points in $V_1 \cup \dots \cup V_{i-1}$, and yields the corresponding blue rectangle from $\mathcal{C}_i^{(\text{up, 2})}$.
• Figure 2: Example of a concept in $\mathcal{C}_i$ for \ref{['theorem:general-k-lower-bound']}. Given the prefixes for $T_i$, the values are deterministically expanded/contracted for other $T_j$. At most $k$ values of $j \in \{1,\dots,i-1\}$ may have $H_j$ be different from $H_{j+1}$; in this concept a change occurred from $H_{i-1}$ to $H_{i}$.

Theorems & Definitions (10)

• Theorem 1: Rectangles Lower Bound for $k=1$
• Claim 2: A level dominates all lower levels
• Lemma 3
• Theorem 4: Lower Bound for $k \ge 2$
• Claim 5
• Claim 6
• Claim 7: $\mathcal{C}_{i}$ dominates $\mathcal{C}_1,\dots,\mathcal{C}_{i-1}$
• Lemma 8
• Claim 9: Few Head Points
• Claim 10: Few Tail Points