DDRprog: A CLEVR Differentiable Dynamic Reasoning Programmer

TL;DR

The paper tackles the nondifferentiability of learning with branching, discrete logic in dynamic architectures by introducing the DDR framework, which interleaves program prediction and execution and adds a differentiable forking mechanism for branching. It applies DDRprog to CLEVR VQA and DDRstack to Reverse Polish Notation expression evaluation, demonstrating that structural supervision and stack-aware branching yield improved subtask consistency and generalization in compact models. The results show competitive CLEVR performance with strong gains on counting and comparison tasks and robust generalization for long RPN expressions, where standard LSTMs fail. Overall, the work provides a practical approach to integrating discrete logical structure into differentiable neural reasoning, enabling more reliable handling of complex data structures and tasks.

Abstract

We present a novel Dynamic Differentiable Reasoning (DDR) framework for jointly learning branching programs and the functions composing them; this resolves a significant nondifferentiability inhibiting recent dynamic architectures. We apply our framework to two settings in two highly compact and data efficient architectures: DDRprog for CLEVR Visual Question Answering and DDRstack for reverse Polish notation expression evaluation. DDRprog uses a recurrent controller to jointly predict and execute modular neural programs that directly correspond to the underlying question logic; it explicitly forks subprocesses to handle logical branching. By effectively leveraging additional structural supervision, we achieve a large improvement over previous approaches in subtask consistency and a small improvement in overall accuracy. We further demonstrate the benefits of structural supervision in the RPN setting: the inclusion of a stack assumption in DDRstack allows our approach to generalize to long expressions where an LSTM fails the task.

Figures (5)

• Figure 1: Prior work reads the question and predicts a program composed of functional modules (denoted f1, f2, etc.); in contrast our model interleaves module prediction and module execution. Whereas IEP suffers an important nondifferentibility (red X), our model provides an end-to-end differentiable gradient path.
• Figure 2: The baseline LSTM fails to learn the expression parse tree underlying reverse Polish notation; it therefore fails to generalize to expressions longer than seen at training time. In contrast, our DDRstack architecture cleanly incorporates supervision of this stack based tree representation and solves the generalization task.
• Figure 3: Visualization of the DDRprog architecture. This configuration answers "How many things are red or spheres?" by predicting $[filter\_red$, $fork$, $filter\_sphere$, $union$, $count]$
• Figure 4: Visualization of the DDRstack architecture with $n=1$. This particular configuration evaluates the [NUM][NUM][OP] formatted expression [0.4, 0.8, /], which is 0.4/0.8=0.5. NUM tokens are embedded before being passed to the LSTM. OP tokens are used as an index to select the corresponding cell. LSTM predictions at each OP token are used to predict intermediate losses (only one for $n=1$).
• Figure 5: Left: Training curves for DDRstack (train/val overlapping, 17k parameters) and the LSTM128 baseline (255k parameters) on RPN10. Right: Generalization performance of DDRstack and the LSTM baseline to RPN30 after training on RPN10.