The Expressivity Boundary of Probabilistic Circuits: A Comparison with Large Language Models

ArXi:2605.12940v1 Announce Type: cross Probabilistic Circuits (PCs) are deep generative models that exact and efficient probabilistic inference. Yet in autoregressive language modeling, PCs still lag behind Transformer-based large language models (LLMs), suggesting an important expressivity gap. In this work, we compare PCs and LLMs under a unified autoregressive formulation.