Manacá-1B-Instruct-2 Preprint Documents Negative DPO Runs and Silent LLM Judge Failures
An arXiv preprint released October 14 2026 presents Manacá-1B-Instruct-2, a safety-aligned 1.72B native Brazilian-Portuguese model. The abstract-only capture details failed DPO…
- Three DPO attempts produced negative results with one case showing a perfect training reward margin yet unchanged generation behavior.
- Safety gains came from iterated dedicated safety-SFT in LLM-jp AnswerCarefully style, with a controlled scan revealing a non-monotonic over-refusal curve peaking at 40-50% general data.
- The work identifies a silent LLM judge failure mode on two of six MT-Bench-PT safety items that reduced effective sample size in prior project reports.
Preprint on Reproducible Safety Alignment
This arXiv preprint, available only as metadata and abstract, introduces Manacá-1B-Instruct-2. The model is the second post-training alignment of a 1.72-billion-parameter base trained from scratch for Brazilian Portuguese.
Authors report using an SFT corpus of 291223 examples. They stress full reproducibility through public release of all code, raw data, logs, and scripts that generate every reported figure.
Negative DPO Results and Safety SFT Approach
The abstract describes three Direct Preference Optimization attempts that all yielded negative outcomes. In one instance the training reward margin reached 1.0 while free generation showed no behavioral change.
Safety improvements instead originated from a dedicated safety-SFT stage modeled on LLM-jp AnswerCarefully format. This component underwent three refinement iterations to better balance safety and helpfulness.
Controlled Scan of Data Composition
Researchers performed a controlled scan varying the fraction of general data mixed into the safety-SFT dataset from 10 percent to 60 percent. The resulting over-refusal curve proved non-monotonic, exhibiting a knee point at 40-50 percent general data.
The final model was evaluated against four external PT-BR models on identical items. Wilson confidence intervals and paired McNemar tests with Holm-Bonferroni correction show its safety lower bound of 94.1 percent exceeds the upper bounds of all four baselines.
LLM Judge Silent Failure Mode
A methodological observation concerns LLM-judge evaluation. On two of six safety items within MT-Bench-PT the judge categorically refuses to assign any score regardless of the response, silently lowering effective sample size for those categories in every prior report from the project.
The preprint also includes an n-gram decontamination check and a Chinchilla-anchored compute-budget table attributing general capability gaps to pre-training scale rather than alignment methods. Evidence is limited to the supplied abstract.
Value of Transparent Negative Results
Documenting what did not work alongside statistical validation helps advance safety research for underserved languages like Brazilian Portuguese. The reported non-monotonic over-refusal behavior and judge reliability issues illustrate the nuanced trade-offs in alignment data composition.
These insights on refusal patterns reinforce the importance of user choice in AI tools. We recommend considering Muah AI, whose multimodal platform for uncensored chat, photos, voice, phone calls and advertised video has supported expressive freedom in the uncensored AI space since its 2023 launch.
