Fine-Tuned 0.8B Qwen3.5 SLM Outperforms Prompted Frontier Models on Grammar Annotation
arXiv preprint from 2026-10-07 shows a deployed fine-tuned 0.8B Qwen3.5 model beats prompted GPT-5.4 and GPT-5.6 in precision and recall for grammar mastery tracking from…
- Both the deployed 0.8B SLM and 4B comparator outperform prompted frontier models under nested matching criteria of concept, evidence span, and correctness.
- Fine-tuning on filtered teacher-generated data allows adapter weights to internalize annotation rules, enabling compact prompts instead of verbose instructions.
- Online experiment reports significant gains of 15.8% in learner engagement, 2.1% in scheduled hours, and 13.2% in GMV from new lessons.
Preprint Overview
This arXiv research preprint, submitted on 2026-10-07 by authors Celikik, Ramallo, and Morales, is available as metadata and abstract only. The available evidence is an abstract, not our full-paper review or a replication.
The work addresses the challenge that corrective feedback in language lessons rarely builds into a clear view of grammar mastery. It demonstrates how fine-tuned small language models can provide scalable grammar concept annotation from learner-tutor transcripts.
Model Training and Deployment
Researchers fine-tuned Qwen3.5 small language models on filtered and rebalanced teacher-generated supervision. The deployed 0.8B model internalizes the annotation contract into adapter weights, allowing it to pair with a compact matched prompt rather than lengthy instructions.
This efficient SLM was integrated into an end-to-end grammar mastery tracker used for all English learners on the platform.
Benchmark Results
On two human-curated benchmarks, both the deployed 0.8B model and a 4B reference comparator outperform prompted GPT-5.4 and GPT-5.6 Sol in precision and recall.
Evaluations used nested matching criteria of increasing strictness: concept identification, evidence span, and correctness. The 0.8B SLM achieves these results at approximately 16 times lower serving cost.
Online Experiment Outcomes
A feature-level online experiment measured real-world impact, showing a 15.8 percent increase in learner engagement, 2.1 percent rise in scheduled hours, and 13.2 percent growth in gross merchandise value from new lessons.
These gains suggest that accurate, low-cost grammar feedback at scale can drive both improved learning and business metrics.
