Xin Gu

Multimodal Video Understanding | Spatio-Temporal Grounding | Video Foundation Models

Email: guxin21@mails.ucas.ac.cn

Biography

I received my Ph.D. degree in Computer Science and Technology from Chinese Academy of Sciences in June 2026. Before that, I received my B.Sc. degree from University of Electronic Science and Technology of China in June 2021.

My research lies at the intersection of multimodal video understanding, spatio-temporal reasoning, and video generation/editing, with particular interests in long-form video reasoning, spatio-temporal video grounding, video captioning, and image/video editing. My recent work has appeared in top-tier conferences and journals, and I have contributed to the Vidi series, a family of large multimodal video foundation models evolving from Vidi to Vidi2 and Vidi2.5 for video understanding and creation.

Education Experience

	Chinese Academy of Sciences `[2021/09 - 2026/06]` \| Ph.D. in Computer Science and Technology GPA: 3.5 / 4.0.
	University of Electronic Science and Technology of China `[2017/09 - 2021/06]` \| B.Sc., School of Information and Software Engineering GPA: 3.9 / 4.0, Rank: 6 / 128.

Internship Experience

	ByteDance `[2021/11 - 2026/07]` \| Computer Vision Algorithm Intern Advisors: Longyin Wen, Guang Chen, Fan Chen, Xiaojie Jin, and Sijie Zhu.
	Institute of Software, Chinese Academy of Sciences `[2020/10 - 2024/12]` \| Cooperative Student Advisors: Libo Zhang, Dawei Du and Heng Fan.

Projects

Vidi Series: Large Multimodal Models for Video Understanding and Creation

Foundation Model | ByteDance Intelligent Creation

Vidi is a series of large multimodal video foundation models, evolving from Vidi to Vidi2 and Vidi2.5. The series supports both video understanding and creation, covering long-form video reasoning, spatio-temporal grounding, temporal retrieval, video question answering, plot reasoning, and video editing planning.

[Latest Report (Vidi2.5)] [Code] [Project Page] [Demo]

Publications

First Author / First Co-Author

	Towards Long-Form Spatio-Temporal Video Grounding Xin Gu, Bing Fan, Jiali Yao, Zhipeng Zhang, Yan Huang, Cheng Han, Heng Fan, Libo Zhang [Paper (ECCV 2026)] [Code]
	Thinking With Videos: Multimodal Tool-Augmented Reinforcement Learning for Long Video Reasoning Haoji Zhang, Xin Gu, Jiawen Li, Chixiang Ma, Sule Bai, Chubin Zhang, Bowen Zhang, Zhichao Zhou, Dongliang He, Yansong Tang [Paper (CVPR 2026)] [Code]
	OmniSTVG: Toward Spatio-Temporal Omni-Object Video Grounding Jiali Yao, Xin Gu, Xinran Deng, Mengrui Dai, Bing Fan, Zhipeng Zhang, Yan Huang, Heng Fan, Libo Zhang [Paper (ICLR 2026)] [Code]
	Knowing Your Target: Target-Aware Transformer Makes Better Spatio-Temporal Video Grounding Xin Gu, Yaojie Shen, Chenxi Luo, Tiejian Luo, Yan Huang, Yuewei Lin, Heng Fan, Libo Zhang [Paper (ICLR 2025)] [Code] (Oral)
	Structured Context Learning for Generic Event Boundary Detection Xin Gu, Congcong Li, Xinyao Wang, Dexiang Hong, Libo Zhang, Tiejian Luo, Longyin Wen, Heng Fan [Paper (WACV 2025)] [Code]
	Multi-Reward as Condition for Instruction-based Image Editing Xin Gu, Ming Li, Libo Zhang, Fan Chen, Longyin Wen, Tiejian Luo, Sijie Zhu [Paper (ICLR 2025)] [Code]
	Edit3k: Universal representation learning for video editing components Xin Gu, Libo Zhang, Fan Chen, Longyin Wen, Yufei Wang, Tiejian Luo, Sijie Zhu [Paper (preprint)] [Code]
	Context-Guided Spatio-Temporal Video Grounding Xin Gu, Heng Fan, Yan Huang, Tiejian Luo, Libo Zhang [Paper (CVPR 2024)] [Code]
	Local Compressed Video Stream Learning for Generic Event Boundary Detection Libo Zhang, Xin Gu,** Congcong Li, Tiejian Luo, Heng Fan [Paper (IJCV)] [Code]
	Accurate and Fast Compressed Video Captioning Yaojie Shen, Xin Gu,** Kai Xu, Heng Fan, Longyin Wen, Libo Zhang [Paper (ICCV 2023)] [Code]
	Text with Knowledge Graph Augmented Transformer for Video Captioning Xin Gu, Guang Chen, Yufei Wang, Libo Zhang, Tiejian Luo, Longyin Wen [Paper (CVPR 2023)] [Code]
	Dual-Stream Transformer for Generic Event Boundary Captioning Xin Gu, Hanhua Ye, Guang Chen, Yufei Wang, Libo Zhang, Longyin Wen 1st on LOVEU@CVPR'22: Generic Event Boundary Captioning [Technical Report (CVPR 2022 Workshops)]

Selected Talent Program Offers

2026 ByteDance - Jindouyun Program;
Tongyi Lab - Ali Star Program;
Taotian Group - Ali Star Program;
Ant Group - Ant Star Program;
Tencent - Qingyun Program;
JD - TGT Program;
Xiaomi - Top Graduate Program;
Baidu - AIDU Program.

Awards

`06/2026`	Outstanding Graduate, University of Chinese Academy of Sciences
`10/2025`	National Scholarship for Doctoral Students
`12/2024`	Chu Lee Yuet Wah Scholarship
`04/2024`	President Award, University of Chinese Academy of Sciences
`06/2021`	Outstanding Graduate, University of Electronic Science and Technology of China
`08/2020`	Third Prize, National WeChat Mini Program Competition
`11/2019`	National Encouragement Scholarship
`10/2019`	Excellent Student Scholarship, University of Electronic Science and Technology of China
`05/2020`	Honorable Mention, Mathematical Contest in Modeling