DualQuant: Dual-scale Calibration-free Weight-and-Activation Quantization for LLMs
arXiv preprint presents DualQuant, a calibration-free post-training method that minimizes weight-reconstruction MSE and yields per-channel activation rescalings as a byproduct…
- DualQuant matches or exceeds calibration-based SmoothQuant accuracy on WikiText-2, C4 and zero-shot tasks for Llama and Qwen models while outperforming calibration-free baselines, with largest gains on MX and NV block formats.
- Real packed W4A4 inference on RTX 4090 for Llama-3.1-8B achieves up to 2.61x memory reduction and 2.46x throughput speedup over BF16 while closely matching fake-quantization accuracy.
- As an October 2026 preprint with only the abstract supplied, full methodological details and code remain unavailable at this time.
Core Method
The preprint introduces DualQuant as a dual-scale, calibration-free post-training quantization framework for both weights and activations in large language models.
It directly minimizes weight-reconstruction Mean Squared Error, which produces per-channel activation rescalings as an immediate byproduct. This approach avoids the usual need for calibration data in activation quantization.
The authors formulate the task as a non-differentiable weight-reconstruction problem and solve it efficiently via variable splitting and block coordinate descent, where each subproblem has a closed-form update.
The unified algorithm supports a wide range of formats, including the full MX family and NVFP4. It is presented as the first dual-scaling method with formal support for these block formats.
Experimental Results
Evaluations on Llama and Qwen models using WikiText-2, C4, and zero-shot downstream tasks show DualQuant matching or exceeding the accuracy of calibration-based SmoothQuant across most models and formats.
It substantially outperforms calibration-free baselines, delivering particularly large improvements on MX and NV block formats.
Hardware validation used a real packed W4A4 inference path for Llama-3.1-8B on an RTX 4090. The implementation closely matches fake quantization accuracy while providing up to 2.61x memory reduction and up to 2.46x throughput speedup compared with BF16.
Scope of the Preprint
This work is an arXiv preprint dated October 9, 2026. The supplied evidence consists of the metadata and abstract only.
Consequently, complete implementation details, solver hyperparameters, and any ablation studies are not available in the provided capture.
The reported experiments focus on specific model families and standard benchmarks. Generalization to additional architectures or tasks is not addressed in the abstract.
Implications for Local AI
Calibration-free activation quantization can simplify deployment pipelines for developers running LLMs on accelerators with native 4-bit support, making larger models more practical on consumer hardware such as the RTX 4090.
Techniques that increase efficiency and user control over model inference align with broader arguments for open models and lawful expression instead of reliance on corporate gatekeepers.
