InstructAV2AV fine-tuned on our data vs its released checkpoints vs Ours

Test subset: 200 ids sampled with seed 0 from the 1,425 fps-fixed JavisDiT test clips (same ids in every row). InstructAV2AV BASE = the authors' removal / insertion.safetensors, zero-shot. InstructAV2AV FT = the same models fully fine-tuned on our 24,444 training pairs (authors' recipe: lr 1e-5, effective batch 8, 2,500 steps, 4×H100 ≈ 38 h per direction), inference at the same 50 steps / cfg 4.0 video / 3.0 audio / per-clip seed. Addition rows (both BASE and FT) use the first-frame anchor (GT frame 0 + text), the protocol of the paper table. Ours = JavisDiT refwarm ep019 (noref_remove / ref_add). Every clip on this page is the conformed 256×256 / 16 fps / 5.06 s video with the model's own audio, i.e. exactly what the metrics score.

Objective metrics (test200, same 200 ids per row)

Video vs GT: PSNR/SSIM/LPIPS/FVD (eval256). Audio vs GT: FAD / IS / KL / LSD (audioldm_eval, cnn14 IS/KL, LSD offset 0). Removal GT = fps-fixed object-removed clip + SAM-Audio residual; addition GT = the original clip + full mix. FAD and FVD are set-level and depend on n — compare rows on this page to each other, not to the 1,425-clip paper table.

Removal

model	PSNR↑	SSIM↑	LPIPS↓	FVD↓	FAD↓	IS↑	KL↓	LSD↓
Ours mainline (ep018, paper table)	20.48	0.667	0.270	349	2.818	4.51	1.468	12.91
Ours (refwarm ep019)	20.04	0.655	0.282	483	2.912	4.83	1.492	12.84
InstructAV2AV BASE	18.48	0.669	0.299	512	2.249	5.12	1.559	14.73
InstructAV2AV FT step-2500	21.15	0.715	0.237	274	2.541	4.43	1.591	17.31

Addition (first-frame + text)

model	PSNR↑	SSIM↑	LPIPS↓	FVD↓	FAD↓	IS↑	KL↓	LSD↓
Ours mainline (ep010 ref_add, paper table)	16.91	0.532	0.304	1414	5.001	4.75	1.517	12.07
Ours (refwarm ep019, ref_add)	18.79	0.619	0.228	611	4.286	5.14	1.407	11.64
InstructAV2AV BASE (ff)	17.21	0.620	0.278	510	6.743	4.71	2.408	17.27
InstructAV2AV FT step-2500 (ff)	17.44	0.622	0.256	374	4.092	5.61	1.788	16.73

Gemini judge (gemini-3.1-pro-preview, video at 8 fps)

Yes-rate in %. The judge sees one clip (no source, no instruction) and answers three Yes/No questions: VES — is the target object absent (removal) / present and visually consistent (addition)? AES — the same for the target sound; AVS — are audio and video temporally consistent? Paired per id; p = exact McNemar vs the FT row. Removal AES is capped low for every model because the residual GT itself still leaks the sound in most clips.

Removal (n = 199 ids judged so far)

model	VES: target gone (video)	AES: target sound gone (audio)	AVS: A/V timing
Ours (refwarm ep019)	37.7	9.5	88.4
InstructAV2AV BASE	26.1	6.0	94.0
InstructAV2AV FT step-2500	47.7	7.5	85.4

Addition (n = 200 ids judged so far)

model	VES: target present + consistent (video)	AES: target sound present + consistent (audio)	AVS: A/V timing
Ours (refwarm ep019, ref_add)	41.5	80.0	76.0
InstructAV2AV BASE (ff)	34.0	63.0	56.0
InstructAV2AV FT step-2500 (ff)	60.5	73.5	59.5

Paired differences (pp, McNemar p): Δ FT − BASE (pp) / +21.6 (p=1.3e-06) / +1.5 (p=0.65) / -8.5 (p=0.0076) · Δ FT − Ours (pp) / +10.1 (p=0.031) / -2.0 (p=0.48) / -3.0 (p=0.39) · Δ FT − BASE (pp) / +26.5 (p=1.6e-10) / +10.5 (p=0.014) / +3.5 (p=0.53) · Δ FT − Ours (pp) / +19.0 (p=5.1e-05) / -6.5 (p=0.072) / -16.5 (p=0.00026)