TR-PTQ Reformulates Transformer Quantization for Integer-Only Inference
arXiv preprint identifies LayerNorm scales and GELU approximations as main quantization error sources in transformers. TR-PTQ uses shared Taylor Region primitives for fully…
- Learned scale parameters in normalization layers and compounded GELU approximations drive most accuracy loss, while SoftMax is robust to aggressive integer quantization.
- TR-PTQ enables integer-only operations via shared Taylor Region exponential and logarithm functions, performing division and square roots in the log-domain.
- The calibration-free method reports less than 1.5 percent absolute accuracy degradation across benchmarks, though only the abstract is available.
Identifying Structural Error Sources
The preprint challenges the view that quantization degradation in transformers stems primarily from limited numerical precision. Instead it pinpoints specific structural issues.
Learned scale parameters within normalization layers and the accumulated approximations inside GELU activations are shown to be the dominant contributors. SoftMax, by comparison, tolerates aggressive quantization without major issues.
This analysis reframes the problem toward targeted mathematical reformulation rather than simply increasing bit width.
Unified Integer-Only Formulation
TR-PTQ introduces a shared Taylor Region approach for exponential and logarithm primitives. This allows computationally intensive operations such as division and square roots to run entirely with standard integer arithmetic in the log domain.
The technique is paired with an outlier-aware optimization for LayerNorm parameters that requires no calibration data. Together these steps remove any need for floating-point hardware support in transformer nonlinearities.
As a post-training method it can be applied to existing models without retraining, simplifying deployment where hardware resources are constrained.
Reported Accuracy and Limitations
The authors state that TR-PTQ keeps absolute accuracy degradation below 1.5 percent on standard vision and language benchmarks. This preprint was submitted on October 7 2026; only the abstract is available, so exact implementation details, full results, and independent reproductions are not yet public.
The work highlights how reformulating specific operations can maintain performance while enabling efficient integer-only inference. Practitioners working with open-weight models on resource-limited devices may find the direction useful once the full paper appears.
