COMPASS Score Estimates Model Compatibility Before Alignment or Merging
arXiv preprint presents COMPASS, a diagnostic using k-NN overlap to predict compatibility of pretrained model pairs for alignment or merging, offering a low-cost filter before…
- COMPASS decomposes compatibility into pair-specific k-NN overlap ease and a paradigm-level ceiling based on training approaches.
- The preprint claims strong correlation with downstream success on 72 cross-modal alignments and 392 model merges.
- It prescribes actions such as prioritizing high-scoring pairs, investing compute on promising ones, switching paradigms or deprioritizing poor matches.
The Compatibility Challenge
Combining pretrained models through alignment or merging is common for multimodal systems and task-specific variants. Yet compatibility is typically confirmed only after running the full process.
This trial-and-error approach demands substantial compute, posing challenges for researchers and resource-limited teams.
The COMPASS Diagnostic
The October 13 2026 preprint introduces COMPASS, a theoretically motivated score that estimates compatibility prior to any alignment or merging.
It leverages the observation that representational convergence is most robust at local neighborhood levels, decomposing compatibility into two axes: pair-specific ease via k-NN overlap between encoders and a paradigm-level ceiling reflecting maximum possible overlap given training methods.
Guidance and Prescriptions
COMPASS ranks candidate pairs and suggests actions: prioritize pairs with high agreement and compatible paradigms, invest compute in pairs with compatible paradigms but low initial overlap, switch pretraining paradigms when they limit agreement, or deprioritize pairs weak on both axes.
Preprint Evidence
According to the abstract, the authors applied COMPASS to 72 cross-modal alignment experiments and 392 task-arithmetic model merges. The estimates reportedly correlate with downstream success, with average Pearson correlations of 0.91 on alignment and 0.70 on merging.
This is an arXiv preprint based on metadata and abstract only. The full paper and independent replication are required to verify the reported results and methodology.
Utility for Model Development
For those working with open models, the approach offers a cheap pre-filter to guide resource allocation toward more promising combinations and reduce failed experiments.
Such predictive tools can inform decisions around alignment and merging where model behavior, including refusal patterns, influences practical usability and user control.
