Task-Phase-Conditioned Expert Routing Is Not a Reliable Proxy for MoE Quantization Sensitivity
arXiv preprint shows that routing statistics across prompt, reasoning and response phases fail to improve mixed-precision quantization of expert FFN tensors over uniform…
- Routing-derived composite sensitivity scores failed to outperform uniform bit allocation across flat, steep and random regimes at fixed average bits-per-weight budgets.
- Aggressive low-bit settings caused clear quality degradation regardless of routing guidance, attributed to mandatory downgrades and KLD tail effects.
- The study argues that routing-behavior statistics remain an unvalidated proxy for true per-layer quantization cost in MoE models.
What the Preprint Investigated
The authors examined whether per-layer, per-task-phase expert-routing statistics could reliably guide mixed-precision quantization of routed-expert feed-forward network tensors in Mixture-of-Experts language models.
They built a routing-capture and profiling pipeline for the Qwen-3.6-35B-A3B model using ik_llama.cpp. A composite sensitivity score was derived from routing entropy, decisiveness, coefficient of variation, Jaccard similarity, phase fragility and top-1 expert strength.
Coordinate descent tuned layer-wise bit allocations under a fixed average bits-per-weight budget. The pipeline works for standard top-k MoE architectures.
Experimental Results
Across roughly 4.0 bpw flat, 2.7 bpw steep and 2.7 bpw random-control conditions, routing-guided reallocation did not surpass uniform baselines within each quality tier.
In the flat regime the guided approach was statistically neutral. In the steep low-bit regime, random reallocation performed within one standard error of the routing-guided version, indicating the loss stems from structural constraints and Kullback-Leibler divergence tail injection rather than routing insight.
The aggressive-budget condition produced a decisive quality drop exceeding four standard errors, replicated across independent weight vectors.
Core Argument
The preprint concludes that routing-behavior statistics are an unvalidated proxy for actual per-layer quantization sensitivity.
Local inference users seeking better quantization of MoE models should therefore treat such routing-derived metrics with caution until stronger validation appears.
Limitations of the Evidence
This report is based solely on the arXiv abstract dated 2026-10-14; the full paper is unavailable.
Results are specific to one model and profiling setup. Generalization to other MoE architectures, libraries or techniques is unknown.
