Large language models have achieved remarkable success in recent years, primarily due to
the implementation of self-attention mechanisms. However, traditional Softmax attention …