One Basis to Animate Them All: Gaussian Blendshape Distillation for Real-Time Avatars

CV 3DGS HPMR AI Systems and Hardware MCPKD EDIO
2026年10月01日
三维高斯化身模型虽能实现快速渲染,但其实时动画往往受限于高昂的神经网络推理开销。我们针对这一瓶颈问题展开研究,发现预训练化身模型的动画可被精确近似为一组与身份无关的混合形状(blendshapes)的线性组合。基于这一发现,我们提出了GALA(Gaussian Animation via Linear Approximation,基于线性近似的高斯动画方法)——一种知识蒸馏方法:它摒弃了逐帧执行的繁重神经解码过程,转而采用一个轻量级的系数预测器配合线性混合操作来生成动画。为进一步提升重建保真度并降低内存占用,我们提出在渲染感知型度量准则和严格内存预算约束下,采用分块局部主成分分析(block-local PCA)构建混合形状基底。本方法仅需训练一个浅层多层感知机(MLP)网络以预测混合形状系数,且无需对原始化身模型进行任何重训练,即可适配多种动画架构。我们在三类差异显著的化身模型上验证了GALA的有效性,涵盖面部表情动画与含服装动力学的全身动画两大任务。实验表明,该蒸馏方法不仅对未参与训练的新身份具有良好的泛化能力,更可将CPU端动画推理开销最高降低三个数量级,同时几乎完整保留原始渲染质量。本方法所取得的优异效果,有力印证了学习所得化身表征中普遍存在的共享线性结构,使其得以在移动设备上以高达60帧/秒的实时帧率,实现高效率与高精度兼具的动画生成。项目主页:https://ramazan793.github.io/gala/
3D Gaussian avatars support fast rendering, however, their real-time animation is often challenged by the costly neural inference. We address this bottleneck and show that the animation of pretrained avatar models can be closely approximated by a linear combination of identity-independent blendshapes. Building on this finding, we introduce GALA (Gaussian Animation via Linear Approximation), a distillation method that replaces per-frame heavy neural decoding with a shallow coefficient predictor and a linear blend. To improve fidelity and reduce memory requirements, we propose to construct the basis using block-local PCA under a rendering-aware metric and a memory budget. Our method learns a shallow MLP network to predict blendshape coefficients and applies to various animation architectures without retraining original models. We validate GALA by accelerating the inference of three distinct avatar models for 3D animation of facial expressions and full-bodies with clothing dynamics. Across these models, our distillation generalizes to held-out identities and reduces CPU animation cost by up to three orders of magnitude while preserving most of the rendering quality. Excellent results of our method confirm the shared linear structure of learned avatar representations and enable highly efficient and accurate animation at frame rates reaching up to 60fps on mobile devices. Project page: https://ramazan793.github.io/gala/
许愿