JoyAI-Video-Edit

Real-Time Open-Ended Video Editing with Autoregressive Diffusion

🐶 JoyAI-Video-Edit

JoyAI-Video-Edit is a real-time, instruction-guided video editing framework designed for open-ended video streams.

Unlike conventional offline video editing models that require a complete video before inference, JoyAI-Video-Edit processes frames causally as they arrive. It can continuously edit a live camera stream or an uploaded video according to natural-language instructions without accessing future frames or requiring a predefined video length.

The framework combines:

• an MLLM-based condition encoder for understanding editing instructions,
• a causal video VAE for streaming visual representation,
• and a 16B multimodal diffusion transformer for high-quality video editing.

Through autoregressive diffusion modeling, aligned autoregressive distribution matching distillation, long-horizon optimization, bounded KV-state inference, and deployment-oriented scheduling, JoyAI-Video-Edit achieves stable long-range editing while substantially reducing inference latency.

Our deployment system reaches 30 FPS end-to-end at 720 × 1248 resolution, enabling real-time streaming video editing.

---

🔥 News

• 2026.08.06 — 🎉 We released the live demo of JoyAI-Video-Edit.
  Try it here: Streaming V2V Demo

• 2026.08.05 — 🎉 We released the model checkpoints, deployment code, online demo, and technical report.

---

💎 Highlights

⚡ Real-Time Streaming Editing

JoyAI-Video-Edit edits frames continuously as they arrive instead of waiting for the entire video. It supports open-ended streams and does not require a predefined video duration.

🎯 Instruction-Guided Video Editing

The model supports a broad range of editing instructions, including:

• Subject modification
• Subject addition
• Subject removal
• Subject replacement
• Local object editing
• Background replacement
• Style transfer
• Appearance modification
• Motion editing
• Reference-guided video editing

🔄 Autoregressive Diffusion

JoyAI-Video-Edit introduces an autoregressive diffusion framework consisting of:

• MLLM condition encoder
• Causal video VAE
• 16B multimodal diffusion transformer
• Chunk-wise autoregressive generation
• Long-horizon temporal optimization
• Bounded KV-state inference

🚀 Real-Time 720p Deployment

The full deployment pipeline achieves 30 FPS @ 720 × 1248 with end-to-end streaming inference.

---

🎬 Demo

Try our online real-time video editing demo:

👉 https://joyai-labs.jd.com/v2v/

Project repository:

👉 https://github.com/jd-opensource/JoyAI-Video-Edit

Technical report:

👉 https://arxiv.org/abs/2608.03974

---

📚 Citation

If JoyAI-Video-Edit is useful for your research or project, please cite:

@article{xiao2026joyai,
  title={JoyAI-Video-Edit: Real-Time Open-Ended Video Editing with Autoregressive Diffusion},
  author={Xiao, Yicheng and Dai, Wenxun and Qin, Xinran and Song, Lin and Zhang, Maoquan and Xu, Hang and Chen, Yukang and Li, Yitong and Zhang, Guohui and Zhang, Yuan and Zhang, Xuying and Zhang, Tommy and Yuan, Jianlong and Li, Peihao and Lu, Shuai and Fu, Siming and Zhao, Chuyang and Han, Xin and Huang, Jie and Li, Wenbo and Ma, Guoqing and Huang, Wei and Qi, Xiaojuan and Huang, Haoyang and Duan, Nan},
  journal={arXiv preprint arXiv:2608.03974},
  year={2026}
}

---

📄 License

JoyAI-Video-Edit is released under the Apache License 2.0.

Please refer to the project repository for the complete license:

https://github.com/jd-opensource/JoyAI-Video-Edit/blob/main/LICENSE