Beyond the Permutation Symmetry of Transformers: The Role of Rotation for Model Fusion

B Zhang, Z Zheng, Z Chen, J Li - arxiv preprint arxiv:2502.00264, 2025 - arxiv.org
Symmetry in the parameter space of deep neural networks (DNNs) has proven beneficial for
various deep learning applications. A well-known example is the permutation symmetry in …