Preprint · 2026

Rethinking Classifier-Free Guidance in On-Policy Diffusion Distillation

Bingnan Li, Haozhe Wang, Haozhong Xiong, Fangtai Wu, Jinpeng Yu, Yang Shi, Jiaming Liu^✉, Ruihua Huang

Qwen Business Unit, Alibaba

^✉Corresponding: jmliu1217@gmail.com

The problem

Naive CFG-based OPD

Supervises only the CFG-composed velocity \(v_{\mathrm{cfg}} = \gamma v^{+} + (1-\gamma) v^{-}\).

supervised: the composed velocity only

\(\ell_{\mathrm{naive}} = \lVert v_{\mathrm{cfg}}^{T} - v_{\mathrm{cfg}}^{S}\rVert_{2}^{2}\)

⚠Negative Branch Asymmetry. The teacher–student errors on the two branches stay unconstrained and can cancel — infinitely many degenerate solutions.

Our fix — PDM

Branch-aware OPD

Decouple supervision onto the positive prediction \(v^{+}\) and the CFG direction \(d = v^{+} - v^{-}\).

supervised: the positive prediction v⁺ and the direction d

\(\ell_{\mathrm{PDM}} = \lVert v_T^{+} - v_S^{+}\rVert_{2}^{2} + \lambda\lVert d_T - d_S\rVert_{2}^{2}\)

✓Identifiable supervision. No cross-branch cancellation, so distillation stays stable and effective across guidance scales.

Why

Negative Branch Asymmetry

Let \(e^{+}, e^{-}\) be the teacher–student error on the positive / negative branch. Matching only \(v_{\mathrm{cfg}}\) pins down just one combination of them.

\(e^{+} = v_T^{+} - v_S^{+} \qquad e^{-} = v_T^{-} - v_S^{-}\)teacher − student prediction error, per branch \(\gamma\, e^{+} + (1-\gamma)\, e^{-} = 0\) Every \((e^{+}, e^{-})\) on this line is equally optimal — the errors cancel and stay unconstrained. That ambiguity is NBA.

Cost

Sensitivity to guidance scale

Naive OPD PDM (Ours)

FID ↓

FVD ↓

Naive OPD is highly sensitive to the inference guidance scale γ — it only fits the training scale. PDM stays low and stable across scales.

Naive CFG-based OPD supervises only the composed velocity, so teacher–student errors on the positive and negative branches can cancel — the ambiguity we term Negative Branch Asymmetry (NBA). Positive–Direction Matching (PDM) instead constrains the positive prediction and the CFG conditional direction separately, yielding identifiable supervision that remains stable across inference guidance scales.

Abstract

On-policy distillation (OPD) adapts diffusion models by querying a teacher along trajectories generated by the current student, but how it should behave under classifier-free guidance (CFG), a default component of modern diffusion systems, remains poorly understood.

Existing OPD methods naturally extend velocity matching to the CFG-composed prediction, directly matching teacher and student guided velocities. We show that this objective is underconstrained at the branch level: positive- and negative-branch errors can compensate in the guided prediction. Through two contrasting cases, we find that naive matching works when both branch errors decrease jointly, but fails when the composed objective induces antagonistic branch-error dynamics, improving the positive branch at the expense of the negative branch. We term this behavior Negative Branch Asymmetry (NBA). To address NBA, we introduce Positive–Direction Matching (PDM), a branch-aware OPD objective that separately constrains the positive prediction and the CFG conditional direction. We apply PDM to dense-to-sparse video control, where naive guided matching is highly sensitive to inference guidance scales, while branch-aware supervision enables more robust and effective knowledge transfer.

Contributions

What this paper shows

When is composed matching safe?

CFG-composed OPD is under-identified at the branch level — yet the ambiguity stays benign under shared negative conditioning, where shared-parameter updates reduce both branch errors jointly. We identify exactly when it turns harmful.

Negative Branch Asymmetry

When the teacher's negative branch holds privileged information, joint error reduction breaks: the composed objective improves the positive branch at the expense of the negative branch. These antagonistic dynamics — NBA — hide at the training scale and surface as guidance-scale sensitivity.

Branch-aware OPD, applied

Positive–Direction Matching (PDM) supervises the positive prediction and the CFG direction separately (with IBM as a branch-aware foil), and delivers quantitative gains on dense-to-sparse video control across pose, depth, and scribble.

Background

On-policy distillation meets CFG

In diffusion on-policy distillation (OPD), the student first rolls out its own denoising trajectory; at every state it visits, a teacher supplies a local velocity target and the student is updated to match it.

But modern diffusion models run with classifier-free guidance (CFG): the velocity actually used at inference is a mix of a positive (conditional) and a negative (unconditional) branch.

\[ v_{\mathrm{cfg}} = \gamma\, v^{+} + (1-\gamma)\, v^{-} \]

CFG composition

Existing OPD methods simply match this composed velocity. Writing the per-branch teacher–student errors as \(e^{+} = v_T^{+} - v_S^{+}\) and \(e^{-} = v_T^{-} - v_S^{-}\), the naive per-state loss is a single squared term:

\[ \ell_{\mathrm{naive}} = \bigl\lVert\, \gamma\, e^{+} + (1-\gamma)\, e^{-} \,\bigr\rVert_{2}^{2} \]

Eq. 5

The full training objective averages this per-state loss \(\ell\) over the student-visited states and timesteps.

Why it can be benign

In standard text-to-image distillation, teacher and student share the same negative conditioning. Shared-parameter updates can then reduce both branch errors jointly, so composed matching stays effective — the branch-level ambiguity exists, but does no harm.

The problem

Negative Branch Asymmetry

Now let the teacher's negative branch receive privileged information the student lacks — a reference image, or dense control distilled into a sparse-control student. Its target can no longer be reproduced from the student's negative input.

Yet the naive loss still constrains only the composed discrepancy. At any supervised state, zero guided loss requires just one linear relation:

\[ \gamma\, e^{+} + (1-\gamma)\, e^{-} = 0 \quad\Longleftrightarrow\quad e^{+} = \frac{\gamma-1}{\gamma}\, e^{-} \]

Eq. 6–7

— an entire line of non-zero \((e^{+}, e^{-})\) pairs: the two errors can compensate without changing the guided prediction.

Our method

Positive–Direction Matching

Instead of supervising the merged velocity, PDM supervises two quantities that together determine the guided prediction at any scale: the positive prediction \(v^{+}\) and the CFG conditional direction \(d_M = v_M^{+} - v_M^{-}\).

\[ \ell_{\mathrm{PDM}} = \lVert e^{+}\rVert_{2}^{2} + \lambda\,\lVert d_T - d_S\rVert_{2}^{2} = \lVert e^{+}\rVert_{2}^{2} + \lambda\,\lVert e^{+} - e^{-}\rVert_{2}^{2} \]

Eq. 10

The first term anchors the positive prediction; the second preserves the conditional direction. For any \(\lambda > 0\), zero loss now requires \(e^{+} = e^{-} = 0\) — the compensation freedom of guided-only matching is eliminated, and one branch can no longer improve at the other's expense.

A branch-aware baseline — Independent Branch Matching (IBM)

IBM matches the two branches directly, \(\ell_{\mathrm{IBM}} = \gamma^{2}\lVert e^{+}\rVert_{2}^{2} + (\gamma-1)^{2}\lVert e^{-}\rVert_{2}^{2}\) — the per-branch weights of the expanded naive objective with the cross-branch interaction term removed. Like PDM it has a unique branch-level zero-loss solution for \(\gamma > 1\); it separates the benefit of branch-level supervision from PDM's particular positive–direction parameterization.

Efficient supervision for video. Full-trajectory OPD is especially expensive for video — one teacher evaluation per denoising state. For the dense-to-sparse application, the student still completes the full rollout, but only the first K visited states receive teacher supervision:

\[ \mathcal{L}_{\mathrm{OPD}}^{(K)} = \mathbb{E}_{\{x^{S}_{t_k}\}\sim p_{\theta}}\!\left[ \frac{1}{K}\sum_{k=0}^{K-1} w(t_k)\, \ell\bigl(x^{S}_{t_k},\, t_k\bigr) \right] \]

Eq. 12

A moderate horizon (K = 8 of 50) keeps distillation quality while cutting training cost by ~6× versus full-trajectory supervision. This design is used only for the video experiments; the image-domain studies follow their standard protocols.

Diagnosis · two contrasting settings

When does NBA emerge?

We contrast two image-domain distillation settings that differ in exactly one thing: the negative conditioning. One shares it between teacher and student; the other gives the teacher privileged information on the negative branch.

Text-rendering distillation shared negatives → benign

SD3.5-Medium · OCR-specialized teacher → same-backbone student · γ_train = 4.5

Teacher and student receive the same prompt on the positive branch and the same null-text condition on the negative branch — the regime prior OPD work operates in.

\[\begin{array}{ll} c_T^{+} = y & \hspace{2.2em} c_S^{+} = y \\[4pt] c_T^{-} = \varnothing & \hspace{2.2em} c_S^{-} = \varnothing \end{array}\]

Reference-conditioned distillation privileged negatives → NBA

FLUX.2-klein · reference-conditioned teacher → text-only student · γ_train = 2

The teacher sees a reference image r whose appearance the text prompt cannot pin down — and keeps it even on its negative branch. The text-only student cannot reproduce that negative target.

\[\begin{array}{ll} c_T^{+} = (y,\ \textcolor{#d1495b}{r}) & \hspace{2.2em} c_S^{+} = y \\[4pt] c_T^{-} = (\varnothing,\ \textcolor{#d1495b}{r}) & \hspace{2.2em} c_S^{-} = \varnothing \end{array}\]

Diagnosis · training dynamics

Watch the branch errors during training

We track the per-transition RMS branch errors \(\sqrt{\lVert e_{+}\rVert_{2}^{2}}\) and \(\sqrt{\lVert e_{-}\rVert_{2}^{2}}\) at every optimizer step, for three objectives: naive composed matching, a positive-only ablation \(\ell_{+} = \lVert e_{+}\rVert_{2}^{2}\), and PDM. Hover to inspect the values; click the legend chips to toggle objectives.

Positive-branch error \(\lVert e_{+}\rVert\) (RMS)

Negative-branch error \(\lVert e_{-}\rVert\) (RMS)

Log scale x-axis: training step · hover to read values

Diagnosis · guidance-scale behavior

Intrinsic vs. distillation-induced sensitivity

A performance drop away from γ_train can simply be inherited from the teacher. The two settings separate the cases: in text rendering nothing is added by distillation; under privileged reference conditioning, naive matching adds sensitivity of its own.

Text rendering: sensitivity is inherited, not introduced

OCR reward (↑) on 1,018 held-out prompts, trained at γ_train = 4.5 and swept over inference guidance scales. Both students closely track the teacher across the entire sweep — the drop at γ = 1 comes from the teacher itself, and naive distillation introduces no sensitivity beyond it.

Method · OCR reward ↑	γ = 1	γ = 2	γ = 3	γ = 4	γ = 4.5
Teacher	75.24	92.91	94.16	94.43	94.09
OPD (Naive)	73.87	92.72	94.52	94.43	94.11
OPD (PDM) — Ours	74.48	93.73	93.99	94.02	94.38

Reference conditioning: naive matching adds the sensitivity

Text-only students trained at γ_train = 2 with naive matching or PDM, evaluated on held-out prompts across guidance scales — all outputs share the same prompt and initial noise. The reference-conditioned teacher preserves the reference-supplied appearance across scales, and PDM closely tracks that behavior; naive matching departs from the teacher, with visual distortion and style drift strongest at γ = 1. That excess degradation is exactly the distillation-induced sensitivity NBA predicts.

PDM (Ours) rows are highlighted; the left column shows the reference-style exemplars, seen only by the teacher.

Application

Dense-to-Sparse Video Control

Having isolated when NBA occurs, we apply branch-aware OPD where it matters: a Wan2.1-VACE teacher with dense per-frame control distilled into a student that sees only 4 sparse keyframes (frame 0 serves as the reference anchor), trained at γ_train = 5 on a 600-clip benchmark built from OpenHumanVid — 100 clips from each of six motion-difficulty buckets, 81-frame clips at 480×832, evaluated over full-video rollouts.

\[\begin{array}{ll} c_T^{+} = (y,\ \textcolor{#d1495b}{p_{\mathrm{dense}}}) & \hspace{2.2em} c_S^{+} = (y,\ p_{\mathrm{sparse}}) \\[4pt] c_T^{-} = (\varnothing,\ \textcolor{#d1495b}{p_{\mathrm{dense}}}) & \hspace{2.2em} c_S^{-} = (\varnothing,\ p_{\mathrm{sparse}}) \end{array}\]

The dense control persists on the teacher's negative branch — the privileged-negative regime where naive matching degrades. Columns are frames of the generated clip; the last row is ours (PDM).

Sparse keyframe / control column PDM (Ours)

Quantitative · Main comparison

One student, three controls

Jointly trained on pose + depth + scribble and evaluated per modality (train and inference guidance both γ = 5). Naive OPD trails the branch-aware objectives on every modality — even at the training scale — while PDM gives the most consistent gains and IBM also beats naive on most control-fidelity metrics. Teacher is an upper reference, excluded from ranking.

Method	MPJPE ↓	MPJPE-Key ↓	PCK@0.2 ↑	PCK@0.1 ↑	FID ↓	FVD ↓
Teacher	3.03	2.82	96.02	91.15	14.13	56.01
Student	5.92	4.10	86.78	69.97	15.81	81.41
SFT	5.02	3.31	90.36	76.72	14.11	63.90
Off-policy	5.14	3.44	89.78	75.46	14.49	68.50
OPD (Naive)	4.43	3.14	91.84	80.28	13.49	65.12
OPD (IBM)	4.20	2.88	92.93	82.27	13.26	65.72
OPD (PDM) — Ours	4.13	2.83	92.98	82.48	13.52	62.77

Method	CORR ↑	RMSE ↓	SI-RMSE ↓	FID ↓	FVD ↓
Teacher	96.98	7.50	6.19	11.02	36.89
Student	83.21	16.87	14.36	15.03	71.33
SFT	88.39	13.83	11.96	12.87	58.13
Off-policy	88.25	13.98	12.10	12.66	59.35
OPD (Naive)	90.58	12.39	10.69	12.08	72.20
OPD (IBM)	91.14	12.03	10.37	11.76	60.38
OPD (PDM) — Ours	91.24	11.95	10.31	11.64	60.60

Method	Recall ↑	Precision ↑	F1 ↑	FID ↓	FVD ↓
Teacher	93.26	94.26	93.73	9.56	24.22
Student	64.20	65.50	64.70	12.29	69.13
SFT	67.23	69.52	68.27	10.69	50.56
Off-policy	70.98	72.15	71.49	11.60	52.90
OPD (Naive)	74.78	75.38	75.00	10.53	59.95
OPD (IBM)	75.92	76.47	76.13	10.12	52.67
OPD (PDM) — Ours	76.42	76.13	76.22	10.34	50.53

Green = best among distilled students (Teacher excluded).

The NBA prediction, tested

Robustness across guidance scales

NBA stays hidden at the training scale and surfaces once the guidance composition changes. Trained once at γ_train = 5 (pose-only) and evaluated at γ = 1, 3, 5, that is exactly what happens: naive matching collapses at γ = 1 (FID 14 → 78, FVD 67 → 508) — excess degradation its branch-aware counterparts do not show — while PDM stays flat.

Pose-controlled video across guidance scales

Ground-truth pose control on top; PDM and Naive at γ = 1, 3, 5 over four keyframes. Naive degrades sharply at γ = 1 while PDM stays faithful across scales.

Keyframe (sparse pose control) PDM (Ours) rows highlighted

Method	γ	MPJPE ↓	PCK@0.2 ↑	PCK@0.1 ↑	FID ↓	FVD ↓
Naive	5	4.62	91.35	79.39	14.05	67.44
	3	4.94	90.23	77.92	22.66	134.09
	1	8.98	80.08	69.14	78.20	507.70
IBM	5	4.28	92.67	81.69	12.99	53.83
	3	4.42	92.09	81.41	13.69	56.85
	1	4.91	90.92	80.66	18.00	71.17
PDM — Ours	5	4.27	92.71	82.12	13.83	58.65
	3	4.17	92.81	82.27	13.46	55.85
	1	4.48	92.14	81.72	15.25	60.97

Guidance-scale generalization in the pose-only setting (γ_train = 5). Naive OPD degrades severely as the inference scale departs from training, while branch-aware objectives remain robust.

Ablations

What matters in PDM

Two design choices, on the pose-only setting: the conditional-direction weight λ, and how many student-visited steps K receive teacher supervision.

Conditional-direction weight λ

λ = 0 is positive-only matching. λ = 1 gives the best fidelity/quality balance (lowest keyframe MPJPE, highest PCK@0.1); larger λ trades control accuracy for distributional quality.

PDM variant	MPJPE-Key ↓	PCK@0.1 ↑	FID ↓	FVD ↓
λ = 0 (positive-only)	2.81	81.80	13.97	59.72
λ = 1 (default)	2.75	82.12	13.83	58.65
λ = 3	2.95	81.25	13.33	61.79
λ = 5	3.00	80.89	12.97	56.85
λ = 10	3.07	80.21	12.96	56.91

On-policy supervision horizon K

The student always rolls out the full trajectory, but only the first K states receive teacher supervision. K = 8 gives the strongest control fidelity; the full 50-step trajectory costs ~6× more per step for no gain.

Supervision horizon	MPJPE-Key ↓	PCK@0.1 ↑	FVD ↓	Time (s/step) ↓
K = 1	2.98	81.19	54.41	~23
K = 2	3.11	80.78	60.63	~38
K = 4	2.86	81.71	57.08	~69
K = 8 (default)	2.75	82.12	58.65	~132
K = 50 (full trajectory)	3.14	80.81	61.95	~790

Conclusion

Takeaways & limitations

CFG-composed OPD is under-identified at the branch level — but the ambiguity does not always cause failure. Under shared negative conditioning both branch errors can decrease jointly; when the teacher's negative branch holds privileged information, joint reduction breaks down and naive matching turns antagonistic — the failure mode we call Negative Branch Asymmetry. Positive–Direction Matching supervises the positive prediction and the CFG direction separately; contrasting image-domain studies validate when NBA emerges, and dense-to-sparse video control shows that branch-aware supervision improves knowledge transfer and robustness across inference guidance scales.

Limitations. The relative behavior of branch-aware objectives remains incompletely understood: PDM and IBM share the same branch-level zero-loss solution under non-degenerate weights, so PDM's observed advantage is currently empirical rather than theoretical. Future work may characterize their different optimization behavior and extend the NBA analysis to other guidance mechanisms and teacher–student conditioning asymmetries.

BibTeX

@misc{li2026rethinkingclassifierfreeguidanceonpolicy,
      title={Rethinking Classifier-Free Guidance in On-Policy Diffusion Distillation},
      author={Bingnan Li and Haozhe Wang and Haozhong Xiong and Fangtai Wu and Jinpeng Yu and Yang Shi and Jiaming Liu and Ruihua Huang},
      year={2026},
      eprint={2607.24731},
      archivePrefix={arXiv},
      primaryClass={cs.CV},
      url={https://arxiv.org/abs/2607.24731},
}