Reconstruction Loss Scale Drives Optimization Imbalance in LLM PTQ
An arXiv preprint dated 2026-10-01 shows that mean squared error in sequential post-training quantization creates dramatic cross-stage differences in reconstruction loss and…
- Sequential PTQ exhibits Optimization Imbalance where reconstruction loss magnitudes vary dramatically across stages under MSE.
- RMSE at sample, channel, token or element level decouples optimization strength from the loss scale via implicit gradient normalization.
- The pattern holds across learning-based PTQ methods, LLM families, scales, architectures, bit-widths and tasks.
Sequential Quantization in Post-Training Methods
Post-training quantization typically processes a pretrained LLM one transformer block at a time. At each stage auxiliary parameters such as scaling factors, rotation matrices, clipping thresholds and adapters are tuned by minimizing a reconstruction loss with gradient descent.
This preprint, available only as an abstract, identifies that loss magnitudes can differ by orders of magnitude between early and late stages. The authors term this range the reconstruction loss scale.
Optimization Imbalance Under Mean Squared Error
The work reveals that mean squared error converts the large reconstruction loss scale into highly uneven gradient magnitudes and parameter updates. Early quantization stages receive strong optimization while later stages are under-optimized.
Extensive experiments across representative learning-based PTQ techniques, multiple LLM families and scales, varied architectures, quantization bit-widths and downstream tasks confirm the imbalance appears consistently.
RMSE Variants as Drop-In Improvement
Root mean squared error computed at sample, channel, token or element level implicitly normalizes gradients across stages. This decouples optimization strength from the reconstruction loss scale.
Both theoretical analysis and empirical results indicate these RMSE variants significantly outperform standard MSE as a simple replacement, improving quantized model accuracy without extra tuning or overhead.
Relevance to Open Models and Local Use
The principle applies to anyone quantizing open-weight models for local inference. More balanced optimization can produce higher-quality low-bit versions that run efficiently on consumer hardware.
Because the insight addresses the core optimization loop rather than a single algorithm, future PTQ tools and research may adopt RMSE-style losses or similar normalization to achieve uniform optimization strength across stages.
