Preprint Distills Production LLM Signals Into Interpretable Features for Resume-Vacancy Matching
Researchers describe a two-part system that converts opaque LLM outputs into named recruiter-actionable matching dimensions. An offline LLM labeler creates data while a…
- The student model shows 95.79 percent operational agreement with recruiter feedback on a production subset though this remains non-blinded agreement rather than independent evaluation.
- The architecture limits the LLM to offline labeling only and uses a lightweight distilled bi-encoder for low-latency online requests.
- Both the labeler prompts and the student model improve iteratively through ongoing recruiter feedback and retraining on updated labels.
Addressing the Need for Explainable Matching
Recruiters require transparent reasons for candidate-vacancy fit beyond a single relevance score. The preprint outlines a method to generate named interpretable matching dimensions that recruiters can review and adjust.
An earlier production stage relied on an LLM-based labeler refined through recruiter input. The updated design restricts this LLM to offline labeling while a distilled model serves live traffic.
Distillation Approach
The first component is an LLM labeler whose prompts and dimension definitions evolved from past recruiter feedback. It now produces labels offline only.
The second component is a distilled bi-encoder trained on those labels. This compact model runs efficiently on CPU and powers every online request.
The system iterates by revising prompts based on new feedback and retraining the bi-encoder on refreshed labeled data.
Scale and Observed Agreement
Training used 168772 labeled vacancy-resume pairs. Recruiters can confirm or revise the predicted dimensions creating an ongoing feedback loop.
On 927 recruiter-recorded values from a production subset the deployed model matched the recorded decisions in 888 cases for 95.79 percent agreement. The authors note this is operational non-blinded agreement not a blinded human study.
Evidence Limitations
This report draws solely from the arXiv preprint abstract and metadata. Full methodological details exact dimension definitions backbone specifics and broader results are not available in the supplied evidence.
The preprint is dated 2026-10-02 and remains under peer review. Readers should treat reported performance as preliminary pending the complete paper.
