Closing the Approximation Gap in Simulation-free Latent SDEs

ML BAY INF Generative Models
从含噪声的观测数据中恢复动力学系统,是神经科学、物理学等众多科学领域反复出现的挑战。潜在随机微分方程(Latent SDEs)通过将系统建模为一个不可观测的状态变量来应对该问题:该状态变量遵循一个可学习的随机微分方程演化,并生成所观测到的数据。变分推断(VI)为此类潜在SDE提供了一种可高效优化的目标函数。传统VI算法通过对时间进行离散化并借助数值模拟来评估该目标函数,从而在建模保真度与计算开销之间做出权衡。近期兴起的一类算法——“无需模拟的变分推断”(simulation-free VI),则绕开了这一权衡:它不再以漂移项(drift)参数化后验分布,而是直接以瞬时边缘分布(instantaneous marginals)对后验进行参数化。本文指出,现有无需模拟的VI算法虽提升了计算效率,却为此付出了代价:其参数化形式将近似后验限制在一类比基于模拟方法所能表达的SDE更小的子集中,从而损害了后验推断质量与模型参数学习效果。为此,我们提出了Helmholtz-SDE——一种全新的无需模拟的VI算法:它通过在满足一组预设边缘分布约束的所有路径测度(path laws)上进行优化,弥合了上述能力鸿沟。实验表明,Helmholtz-SDE对动力学过程的重建显著优于以往所有无需模拟的方法,尤其在后验不确定性较高时提升最为明显;同时,其性能可媲美基于模拟的VI方法,而运行时间却仅为后者的极小一部分。
Recovering dynamical systems from noisy observations is a recurring challenge across scientific domains, including neuroscience and physics. Latent stochastic differential equations (SDEs) address this by modeling the system as an unobserved state that evolves according to a learnable SDE and generates the observations. Variational inference (VI) provides a tractable objective for fitting latent SDEs. Traditional VI algorithms evaluate this objective by numerical simulation over a time discretization, trading fidelity for computational cost. A recent class of algorithms, simulation-free VI, sidesteps this tradeoff by parameterizing the posterior through its instantaneous marginals rather than its drift. In this work, we show that the efficiency of existing simulation-free VI algorithms comes at a price: their parameterizations restrict the approximate posterior to a subset of the SDEs available to simulation-based methods, degrading posterior inference and parameter learning. We propose Helmholtz-SDE, a simulation-free VI algorithm that closes this gap by optimizing over path laws compatible with a prescribed collection of marginals. Helmholtz-SDE recovers dynamics more faithfully than prior simulation-free methods, with the largest gains under high posterior uncertainty. It further matches the performance of simulation-based VI at a fraction of the runtime.
许愿