Learning the Language of Dynamical Systems
Submitted to NeurIPS 2026
A transformer that learns to generate dynamical systems from observed trajectories, representing them as circuits in a typed compositional language whose combinators form a traced monoidal category. Trained by next-token prediction over the grammar and refined with mutation-MCTS, a 23M-parameter model recovers explicit circuit structure on synthetic and classical systems — a step toward foundation models for physics.
Accepted for L4DC LBR 2026
We train a foundation model to discover circuit structures from observed trajectories, using grammar-constrained decoding to guarantee syntactic validity and a staged pipeline — from supervised imitation through reward-ranked fine-tuning to AlphaZero-style search over circuit tokens.
PDF ↓A framework that compiles mathematical equations into typed compositional circuits and executes them via JAX. Unlike implicit approaches that learn black-box dynamics, Asgard maintains explicit mathematical structure throughout — enabling formal composition, algebraic rewriting, and interchangeable semantics.
PDF ↓Why does next-token prediction learn language so well, yet fail for dynamical systems? We propose an explanation rooted in the smoothness of the syntax-to-semantics map — and what it means for building foundation models in structure-sensitive domains.
PDF ↓An agent-based approach that extends the categorical proof system with simulation-guided reasoning. The agent interleaves strict symbolic rewrites with simulation-informed approximate rewrites to discover approximate closed-form solutions for systems where exact solutions do not exist.
PDF ↓A structural correspondence between signal-flow circuits for dynamical systems and sequential programs — where finding a closed-form solution corresponds to eliminating loops. We show trace elimination is undecidable in general, formalising the intuition that "most ODEs don't have closed-form solutions".
PDF ↓