Grade answers
A closed-book measure marks each answer correct, hallucinated or abstained.
ML · ships today
Align with
preferences
Suppress hallucinated answers in favour of abstention, without re-teaching the facts.
Ships as training/dpo-pairs · training/dpo-train
Grade a model's answers, mine preference pairs from where it went wrong, continue the adapter on those pairs with DPO, then prove the aligned model gained without forgetting.
The pipeline
Mine pairs
Build prompt / chosen / rejected pairs from the graded answers, exported with lineage. CPU only.
Align
Continue the trained adapter on the pairs as one remote ML job, inside the budget.
Compare
training/closedbook-compare checks the aligned model gained and did not forget. A regression blocks.
What the record keeps
The graded answers, the mined pairs with lineage, the aligned adapter, the before-and-after comparison, and the approver.
Fewer confident wrong answers
Rejected completions are the model's own hallucinations, so the signal is specific.
Facts stay put
Alignment continues the adapter; it does not retrain on the facts.
Separate, inspectable runs
Pair mining, training and evaluation are three runs with their own records.
Ships as training/dpo-pairs, training/dpo-train and training/closedbook-compare. Trainer support depends on the installed training operator.
See how failure can trigger a reviewed improvement cycle →Run it on your own infrastructure.
Apache 2.0 licensed, self-hosted. cairn pipeline run training/dpo-train