HeuFouFT Preprint Uses Metaheuristics for Task-Guided Fourier Fine-Tuning
arXiv preprint from 2026-10-05 introduces HeuFouFT, a framework that searches for optimal frequency coordinates in Fourier fine-tuning via metaheuristic optimizers and…
- HeuFouFT builds a coarse intensity map from lightweight probes to initialize GA-SA, PSO and Cuckoo Search optimizers, then applies Random Forest filtering to retain only the top 30% of candidates for proxy fine-tuning.
- On the E2E dataset with GPT-2-Medium, all variants outperform random-uniform FourierFT, Gaussian band-pass FourierFT and LoRA across five metrics; the PSO version beats LoCA on four metrics using 37.6% fewer trainable spectral coefficients.
- Once coordinates are selected the method requires only 15-18% of full fine-tuning FLOPs; the preprint is abstract-only and code is publicly released.
Limitations of Fixed Spectral Allocation
Standard Fourier fine-tuning applies uniform sampling or Gaussian band-pass filters to choose which frequency coordinates receive gradients. These task-agnostic rules do not adapt to specific downstream needs.
The preprint shows that such static approaches can waste a limited spectral budget under memory and compute constraints. HeuFouFT replaces them with a search guided directly by task performance.
Task-Guided Search Framework
HeuFouFT first generates a coarse intensity map using lightweight block-level probes. This map seeds three metaheuristic optimizers: Genetic Algorithm with Simulated Annealing, Particle Swarm Optimization, and Cuckoo Search.
During search a Random Forest regressor filters each population, keeping only the top 30% of candidates for proxy fine-tuning. The selected coordinates are then frozen for the final adaptation stage.
Results on E2E with GPT-2-Medium
All three HeuFouFT variants outperform random-uniform FourierFT, Gaussian band-pass FourierFT and LoRA across five metrics on the E2E dataset using GPT-2-Medium.
The PSO variant further surpasses the strongest baseline LoCA on four metrics while employing 37.6% fewer trainable spectral coefficients and requiring only 15-18% of the FLOPs of full fine-tuning.
Evidence Limits and Availability
This coverage is based solely on the arXiv preprint abstract dated 2026-10-05; the full paper and independent replication are not available in the supplied evidence.
The authors state that public code is released, supporting further experimentation with coordinate search for spectral adaptation in parameter-efficient tuning.
