Notes from the lab.
The techniques inside FlowServe — quantization, speculative decoding, anchored fine-tuning, hardware-agnostic scheduling — written up in the open. Papers, engineering deep-dives, and reproducible benchmarks.
All writing
Anchored SFT: KL-regularized fine-tuning without catastrophic drift
Supervised fine-tuning anchored to a frozen reference through KL regularization — combining the stability of SFT with RL-style behavior, and eliminating the adapter drift that plagues long fine-tuning runs.
DDTree: block-diffusion drafting for speculative decoding
A block-diffusion drafter proposes candidate token trees that the target model verifies in a single parallel pass — reaching near-draft-model latency while preserving full-model output distribution.
TurboQuant: 1–4 bit KV-cache quantization for memory-bound LLM inference
Aggressive, runtime-toggleable KV-cache compression up to 16× that treats memory bandwidth as the first-class bottleneck of quantized serving — trading footprint for negligible accuracy loss.
Get new research in your inbox.
Preprints, benchmarks, and engineering notes as we publish them. No marketing.