← Research
PaperPreprint · forthcoming

Anchored SFT: KL-regularized fine-tuning without catastrophic drift

Supervised fine-tuning anchored to a frozen reference through KL regularization — combining the stability of SFT with RL-style behavior, and eliminating the adapter drift that plagues long fine-tuning runs.

FoundationFlow Research·2026·9 min readfine-tuningalignment
reference behaviorplain SFT · driftsASFT · anchored
Figure 1 — KL anchoring holds the tuned model near reference behavior (orange) where unconstrained SFT drifts (grey).

Abstract

Supervised fine-tuning improves task performance but can induce catastrophic drift — the model over-fits the tuning set and loses general capability. Anchored SFT (ASFT) adds a KL-regularization term against a frozen reference model, keeping the tuned policy close to the base distribution while it learns. The result combines SFT's stability and data efficiency with RL-style behavioral control — no reward model, no rollout infrastructure.

1 · Motivation

Teams fine-tune to specialize a model, then discover it has regressed on everything else — instruction-following, safety behavior, reasoning it had before. The failure mode is distributional: unconstrained SFT is free to move the policy arbitrarily far from the base model, and on small or narrow datasets it does exactly that.

The fix is to make "stay close to the reference" an explicit objective, not a hope.

2 · Method

ASFT augments the supervised loss with a KL penalty against a frozen copy of the base model. Two modes cover the range from conservative to adaptive:

  • asft — anchored. A fixed KL weight holds the tuned policy near the reference throughout training, maximizing stability.
  • dft — dynamic. The anchor strength adapts over the run, loosening where the data supports it and tightening where drift risk is high.
  • Presets. quick_test, balanced, high_quality, and enterprise pick sensible KL schedules and hyperparameters so operators don't hand-tune from scratch.

ASFT composes with PEFT/LoRA and Flash Attention 2, and integrates with checkpoint scheduling for fault-tolerant long runs.

Specialize the model without forgetting what it knew.
A KL anchor turns "don't drift" from a hope into a training objective.

3 · Results

Across fine-tuning runs, ASFT preserves general capability that unconstrained SFT erodes, while still absorbing the target task — the anchored policy tracks reference behavior on held-out general evals (Figure 1) even as task accuracy climbs. The dynamic dft mode recovers additional task performance where the data is rich, without the drift that an equivalently aggressive plain-SFT run would incur.

4 · Deployment

ASFT is a first-class training objective in FlowServe's fine-tuning engine. Every ASFT (and SFT) run is auto-cataloged with full lineage — base model, job, dataset, LoRA rank — so adapters never drift out of catalog or orphan. Outputs are tagged (ASFT / DFT / SFT) and registered for one-click serving through the same control plane.

Reproducing these numbers

KL schedules interact with dataset and base model. We tune and measure them during a pilot; see Request a pilot to run ASFT on your own data and hardware.

Cite this work

FoundationFlow Research. "Anchored SFT: KL-regularized fine-tuning without catastrophic drift." Preprint, 2026.
Keep reading
PaperForthcoming

DDTree: block-diffusion drafting for speculative decoding

A block-diffusion drafter proposes candidate token trees that the target model verifies in a single parallel pass — reaching near-draft-model latency while preserving full-model output distribution.

PaperForthcoming

TurboQuant: 1–4 bit KV-cache quantization for memory-bound LLM inference

Aggressive, runtime-toggleable KV-cache compression up to 16× that treats memory bandwidth as the first-class bottleneck of quantized serving — trading footprint for negligible accuracy loss.