Symmetry in Neural Network Parameter Spaces

现代深度学习模型具有高度的超参数化特性，从而导致大量参数配置产生相同的输出。参数空间中的冗余有很大一部分可以通过对称性来解释——这些对称性是指不会改变网络功能的变换。这些对称性塑造了损失函数的景观，并限制了学习的动力学过程，为理解优化、泛化和模型复杂性提供了一个新的视角，这与现有的深度学习理论互为补充。本文综述了参数空间对称性的相关研究。我们总结了现有文献，揭示了对称性与学习理论之间的联系，并指出了这一新兴领域中的空白和机遇。

Modern deep learning models are highly overparameterized, resulting in large sets of parameter configurations that yield the same outputs. A significant portion of this redundancy is explained by symmetries in the parameter space--transformations that leave the network function unchanged. These symmetries shape the loss landscape and constrain learning dynamics, offering a new lens for understanding optimization, generalization, and model complexity that complements existing theory of deep learning. This survey provides an overview of parameter space symmetry. We summarize existing literature, uncover connections between symmetry and learning theory, and identify gaps and opportunities in this emerging field.