ORIGINAL RESEARCH article
Front. Artif. Intell.
Sec. Machine Learning and Artificial Intelligence
Volume 8 - 2025 | doi: 10.3389/frai.2025.1628943
This article is part of the Research TopicEmerging Optimization, Learning and Signal Processing for Next Generation Wireless Communications and NetworkingView all 3 articles
Spectral Momentum Integration: Hybrid Optimization of Frequency and Time Domain Gradients
Provisionally accepted
- Quanzhou Normal University, Quanzhou, China
Select one of your emails
You have multiple emails registered with Frontiers:
Notify me on publication
Please enter your email address:
If you already have an account, please login
You don't have a Frontiers account ? You can register here
We propose Spectral Momentum Integration (SMI), an optimization enhancement that processes gradients in both frequency and time domains. SMI applies the Fast Fourier Transform to selectively filter gradient frequency components before blending them with original gradients using an adaptive scheduling mechanism. Experiments on a character-level language model demonstrate that SMI can achieve inference acceleration while maintaining model performance. Our approach integrates with existing optimizers without modifying model architecture, though it introduces computational overhead and hyperparameter complexity. While our current validation is limited to small-scale experiments, SMI provides a proofof-concept for incorporating frequency-domain processing into neural network optimization, suggesting potential for broader applications pending large-scale validation.
Keywords: deep learning, optimization, Fast Fourier Transform, Gradient processing, Spectral filtering, Inference acceleration
Received: 16 Jun 2025; Accepted: 16 Jul 2025.
Copyright: © 2025 Huang, Chen and Zheng. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
* Correspondence: Musheng Chen, Quanzhou Normal University, Quanzhou, China
Disclaimer: All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article or claim that may be made by its manufacturer is not guaranteed or endorsed by the publisher.