Dr. Linchao Zhu (朱霖潮) is currently an Assistant Professor with the College of Computer Science at Zhejiang University. Before that, he was a Lecturer at the ReLER lab, University of Technology Sydney. His research focus includes AI simulation, AI for DFT and LLMs. He received his Ph.D. degree in University of Technology Sydney, advised by Prof. Yi Yang. He graduated from Zhejiang University with a bachelor’s degree.

Preprints

• DeltaPhi: Learning Physical Trajectory Residual for PDE Solving
  Xihang Yue, Linchao Zhu, Yi Yang
  arXiv

• Ghost Sentence: A Tool for Everyday Users to Copyright Data from Large Language Models
  Shuai Zhao, Linchao Zhu, Ruijie Quan, Yi Yang
  arXiv [PDF]

Publications

• Holistic Physics Solver: Learning PDEs in a Unified Spectral-Physical Space
  Xihang Yue, Yi Yang, Linchao Zhu
  ICML 2025

• VideoGrain: Modulating Space-Time Attention for Multi-grained Video Editing
  Xiangpeng Yang, Linchao Zhu, Hehe Fan, Yi Yang
  ICLR 2025 [PDF]

• Long-horizon Visual Instruction Generation with Logic and Attribute Self-reflection
  Yucheng Suo, Fan Ma, Kaixin Shen, Linchao Zhu, Yi Yang
  ICLR 2025 [PDF]

• Combating label noise with a general surrogate model for sample selection
  Chao Liang, Linchao Zhu, Humphrey Shi, Yi Yang
  IJCV [PDF]

• Detecting and mitigating hallucination in large vision language models via fine-grained ai feedback
  Wenyi Xiao, Ziwei Huang, Leilei Gan, Wanggui He, Haoyuan Li, Zhelun Yu, Hao Jiang, Fei Wu, Linchao Zhu
  AAAI 2025 [PDF)

• Neural Interaction Energy for Multi-Agent Trajectory Prediction
  Kaixin Shen, Ruijie Quan, Linchao Zhu, Jun Xiao, Yi Yang
  ACM Multimedia 2024 [PDF]

• Slimmable networks for contrastive self-supervised learning
  Shuai Zhao, Xiaohan Wang, Linchao Zhu, Yi Yang
  IJCV [PDF][Code]

• GG-Editor: Locally Editing 3D Avatars with Multimodal Large Language Model Guidance
  Yunqiu Xu, Linchao Zhu, Yi Yang
  ACM Multimedia 2024 [PDF]

• MoS: Mixture of Scale and Shift Experts for Text-Only Video Captioning
  Heng Jia, Yunqiu Xu, Linchao Zhu, Guang Chen, Yufei Wang, Yi Yang
  ACM Multimedia 2024 [PDF]

• FreeLong: Training-Free Long Video Generation with SpectralBlend Temporal Attention
  Yu Lu, Yuanzhi Liang, Linchao Zhu, Yi Yang
  NeurIPS 2024 [PDF][Project Page]

• VillagerAgent: A Graph-Based Multi-Agent Framework for Coordinating Complex Task Dependencies in Minecraft
  Yubo Dong, Xukun Zhu, Zhengzhe Pan, Linchao Zhu, Yi Yang
  ACL Findings 2024 [PDF]

• FragRel: Exploiting Fragment-level Relations in the External Memory of Large Language Models
  Xihang Yue, Linchao Zhu, Yi Yang
  ACL Findings 2024 [PDF]

• CapHuman: Capture Your Moments in Parallel Universes
  Chao Liang, Fan Ma, Linchao Zhu, Yingying Deng, Yi Yang
  CVPR 2024 [PDF]

• Knowledge-Enhanced Dual-stream Zero-shot Composed Image Retrieval
  Yucheng Suo, Fan Ma, Linchao Zhu, Yi Yang
  CVPR 2024 [PDF]

• Test-Time Adaptation with CLIP Reward for Zero-Shot Generalization in Vision-Language Models
  Shuai Zhao, Xiaohan Wang, Linchao Zhu, Yi Yang
  ICLR 2024 [PDF] [Project page]

• Temporal perceiving video-language pre-training
  Fan Ma, Xiaojie Jin, Heng Wang, Jingjia Huang, Linchao Zhu, Jiashi Feng, Yi Yang
  AAAI 2024 [PDF]

• DGL: Dynamic Global-Local Prompt Tuning for Text-Video Retrieval
  Xiangpeng Yang, Linchao Zhu, Xiaohan Wang, Yi Yang
  AAAI 2024 [PDF]

• Text Augmented Spatial-aware Zero-shot Referring Image Segmentation
  Yucheng Suo, Linchao Zhu, Yi Yang
  EMNLP Findings 2023 [PDF]

• MAAL: Multimodality-Aware Autoencoder-Based Affordance Learning for 3D Articulated Objects
  Yuanzhi Liang, Xiaohan Wang, Linchao Zhu, Yi Yang
  ICCV 2023 [PDF]

• WhitenedCSE: Whitening-based Contrastive Learning of Sentence Embeddings
  Wenjie Zhuo, Yifan Sun, Xiaohan Wang, Linchao Zhu, Yi Yang
  ACL 2023 (Oral) [PDF]

• Gloss-Free End-to-End Sign Language Translation
  Kezhou Lin, Xiaohan Wang, Linchao Zhu, Ke Sun, Bang Zhang, Yi Yang
  ACL 2023 (Oral) [PDF]

• Variational cross-graph reasoning and adaptive structured semantics learning for compositional temporal grounding
  Juncheng Li, Siliang Tang, Linchao Zhu, Wenqiao Zhang, Yi Yang, Tat-Seng Chua, Fei Wu, Yueting Zhuang
  TPAMI [PDF]

• PointListNet: Deep Learning on 3D Point Lists
  Hehe Fan, Linchao Zhu, Yi Yang, Mohan Kankanhalli
  CVPR 2023 [PDF]

• Efficient Multimodal Fusion via Interactive Prompting
  Yaowei Li, Ruijie Quan, Linchao Zhu, Yi Yang
  CVPR 2023 [PDF]

• MIST: Multi-modal Iterative Spatial-Temporal Transformer for Long-form Video Question Answering
  Difei Gao, Luowei Zhou, Lei Ji, Linchao Zhu, Yi Yang, Mike Zheng Shou
  CVPR 2023 [PDF]

• DeCap: Decoding CLIP Latents for Zero-Shot Captioning via Text-Only Training
  Wei Li, Linchao Zhu, Longyin Wen, Yi Yang
  ICLR 2023 [PDF]

• Fine-Grained Semantically Aligned Vision-Language Pre-Training
  Juncheng Li, Xin He, Longhui Wei, Long Qian, Linchao Zhu, Lingxi Xie, Yueting Zhuang, Qi Tian, Siliang Tang
  NeurIPS 2022 [PDF]

• CenterCLIP: Token Clustering for Efficient Text-Video Retrieval
  Shuai Zhao, Linchao Zhu, Xiaohan Wang, Yi Yang
  SIGIR 2022 [PDF] [Code]

• Dilated Context Integrated Network with Cross-Modal Consensus for Temporal Emotion Localization in Videos Juncheng Li, Junlin Xie, Linchao Zhu, Long Qian, Siliang Tang, Wenqiao Zhang, Haochen Shi, Shengyu Zhang, Longhui Wei, Qi Tian, Yueting Zhuang
  ACM MM 2022. [PDF]

• A Simple Episodic Linear Probe Improves Visual Recognition in the Wild
  Yuanzhi Liang, Linchao Zhu, Xiaohan Wang, Yi Yang
  CVPR 2022 [PDF] [Code]

• Unified Transformer Tracker for Object Tracking
  Fan Ma, Mike Zheng Shou, Linchao Zhu, Haoqi Fan, Yilei Xu, Yi Yang, Zhicheng Yan
  CVPR 2022 [PDF] [Code]

• SEEG: Semantic Energized Co-speech Gesture Generation
  Yuanzhi Liang, Qianyu Feng, Linchao Zhu, Li Hu, Pan Pan, Yi Yang
  CVPR 2022 [PDF] [Code]

• Complex Video Action Reasoning via Learnable Markov Logic Network
  Yang Jin, Linchao Zhu, Yadong Mu
  CVPR 2022 [PDF]

• Compositional Temporal Grounding with Structured Variational Cross-Graph Correspondence Learning
  Juncheng Li, Junlin Xie, Long Qian, Linchao Zhu, Siliang Tang, Fei Wu, Yi Yang, Yueting Zhuang, Xin Eric Wang
  CVPR 2022 [PDF] [Code]

• Weakly Supervised Moment Localization with Decoupled Consistent Concept Prediction
  Fan Ma, Linchao Zhu, Yi Yang
  IJCV 2022 [PDF]

• Interactive Prototype Learning for Egocentric Action Recognition
  Xiaohan Wang, Linchao Zhu, Heng Wang, Yi Yang
  ICCV 2021 [PDF]

• A Multi-Mode Modulator for Multi-Domain Few-Shot Classification
  Yanbin Liu, Juho Lee, Linchao Zhu, Ling Chen, Humphrey Shi, Yi Yang
  ICCV 2021 [PDF][Supp] [Code]

• Universal-Prototype Enhancing for Few-Shot Object Detection
  Aming Wu, Yahong Han, Linchao Zhu, Yi Yang
  ICCV 2021 [PDF] [Code]

• Adaptive Hierarchical Graph Reasoning with Semantic Coherence for Video-and-Language Inference
  Juncheng Li, Siliang Tang, Linchao Zhu, Haochen Shi, Xuanwen Huang, Fei Wu, Yi Yang, Yueting Zhuang
  ICCV 2021 [PDF]

• Vector-Decomposed Disentanglement for Domain-Invariant Object Detection
  Aming Wu, Rui Liu, Yahong Han, Linchao Zhu, Yi Yang
  ICCV 2021 [PDF] [Code]

• Faster Meta Update Strategy for Noise-Robust Deep Learning
  Youjiang Xu, Linchao Zhu, Lu Jiang, Yi Yang
  CVPR 2021 (Oral) [PDF] [Supp] [Code]

• OpenMix: Reviving Known Knowledge for Discovering Novel Visual Categories in An Open World
  Zhun Zhong, Linchao Zhu, Zhiming Luo, Shaozi Li, Yi Yang, Nicu Sebe
  CVPR 2021 [PDF]

• T2VLAD: Global-Local Sequence Alignment for Text-Video Retrieval
  Xiaohan Wang, Linchao Zhu, Yi Yang
  CVPR 2021 [PDF]

• Instance-Invariant Domain Adaptive Object Detection via Progressive Disentanglement
  Aming Wu, Yahong Han, Linchao Zhu, Yi Yang
  TPAMI, DOI: 10.1109/TPAMI.2021.3060446 [PDF] [Code]

• Symbiotic Attention for Egocentric Action Recognition with Object-centric Alignment
  Xiaohan Wang, Linchao Zhu, Yu Wu, Yi Yang
  TPAMI, DOI: 10.1109/TPAMI.2020.3015894 [PDF][Bibtex]

• Label Independent Memory for Semi-Supervised Few-shot Video Classification
  Linchao Zhu, Yi Yang
  TPAMI, DOI: 10.1109/TPAMI.2020.3007511, 2020 [PDF][Bibtex]

• SF-Net: Single-Frame Supervision for Temporal Action Localization
  Fan Ma, Linchao Zhu, Yi Yang, Shengxin Zha, Gourab Kundu, Matt Feiszli, Zheng Shou
  ECCV 2020 (Spotlight) [PDF][Code]

• Motion-Excited Sampler: Video Adversarial Attack with Sparked Prior
  Hu Zhang, Linchao Zhu, Yi Zhu, Yi Yang
  ECCV 2020 [PDF][Code]

• Learning to Transfer Learn: Reinforcement Learning-Based Selection for Adaptive Transfer Learning
  Linchao Zhu, Sercan O. Arık, Yi Yang, Tomas Pfister
  ECCV 2020 [PDF] [Code]

• ActBERT: Learning Global-Local Video-Text Representations
  Linchao Zhu, Yi Yang
  CVPR 2020 (Oral) [PDF]

• Inflated Episodic Memory with Region Self-Attention for Long-Tailed Visual Recognition
  Linchao Zhu, Yi Yang
  CVPR 2020 [PDF]

• Gated Channel Transformation for Visual Recognition
  Zongxin Yang, Linchao Zhu, Yu Wu, Yi Yang
  CVPR 2020 [Arxiv][Code]

• Semantic Correspondence as an Optimal Transport Problem
  Yanbin Liu, Linchao Zhu, Makoto Yamada, Yi Yang
  CVPR 2020 [PDF][Code]

• Learning Filter Pruning Criteria for Deep Convolutional Neural Networks Acceleration
  Yang He, Yuhang Ding, Ping Liu, Linchao Zhu, Hanwang Zhang, Yi Yang
  CVPR 2020 [PDF]

• Symbiotic Attention with Privileged Information for Egocentric Action Recognition
  Xiaohan Wang, Yu Wu, Linchao Zhu, Yi Yang
  AAAI 2020 (Oral) [PDF]

• FASTER Recurrent Networks for Video Classification
  Linchao Zhu, Laura Sevilla-Lara, Du Tran, Matt Feiszli, Yi Yang, Heng Wang
  AAAI 2020 [PDF]

• Connective Cognition Network for Directional Visual Commonsense Reasoning
  Aming Wu, Linchao Zhu, Yahong Han, Yi Yang
  NeurIPS 2019 [PDF Code]

• Dual Attention Matching for Audio-Visual Event Localization
  Yu Wu, Linchao Zhu, Yan Yan, Yi Yang
  ICCV 2019 (Oral) [PDF]

• Entangled Transformer for Image Captioning
  Guang Li, Linchao Zhu, Ping Liu, Yi Yang
  ICCV 2019 [PDF]

• Auto-ReID: Searching for a Part-Aware ConvNet for Person Re-Identification
  Ruijie Quan, Xuanyi Dong, Yu Wu, Linchao Zhu, Yi Yang
  ICCV 2019 [PDF]

• Sim-Real Joint Reinforcement Transfer for 3D Indoor Navigation
  Fengda Zhu, Linchao Zhu, Yi Yang
  CVPR 2019 [PDF]

• Cubic LSTMs for Video Prediction
  Hehe Fan, Linchao Zhu, Yi Yang
  AAAI 2019 [PDF]

• Compound Memory Networks for Few-shot Video Classification
  Linchao Zhu, Yi Yang
  ECCV 2018 [PDF], [train.list, val.list, test.list]

• Decoupled Novel Object Captioner
  Yu Wu, Linchao Zhu, Lu Jiang, Yi Yang
  ACM MM 2018 [PDF Code]

• Fast Parameter Adaptation for Few-shot Image Captioning and Visual Question Answering
  Xuanyi Dong, Linchao Zhu, De Zhang, Yi Yang, Fei Wu
  ACM MM 2018 [PDF Code]

• Watching a Small Portion could be as Good as Watching All: Towards Efficient Video Classification
  Hehe Fan, Zhongwen Xu, Linchao Zhu, Chenggang Yan, Jianjun Ge, Yi Yang
  IJCAI 2018 [PDF Code]

• Bidirectional Multirate Reconstruction for Temporal Modeling in Videos
  Linchao Zhu, Zhongwen Xu, Yi Yang
  CVPR 2017 (Spotlight) [PDF Code]

• Few-Shot Object Recognition from Machine-Labeled Web Images
  Zhongwen Xu*, Linchao Zhu*, Yi Yang
  CVPR 2017 (Spotlight), * indicates equal contributions [PDF Code]

• Uncovering Temporal Context for Video Question Answering
  Linchao Zhu, Zhongwen Xu, Yi Yang, Alex G. Hauptmann
  IJCV, DOI: 10.1007/s11263-017-1033-7, 2017 [PDF] [Project] [Bibtex]

• Recognizing an action using its name: A knowledge-based approach
  Chuang Gan, Yi Yang, Linchao Zhu, Deli Zhao, Yueting Zhuang
  IJCV, DOI: 10.1007/s11263-016-0893-6, 2016 [PDF]

Competitions

• The first place on behaviour representation learning from video data at MABe 2022, CVPR [Mouse Triplets Track] [Ant-beetle Groups Track].
• The second place in the Evoked Emotion from Videos Challenge, CVPR 2021. See our technical report and code.
• The first place in EPIC-Kitchen Action Recognition 2020. See our report for more details.
• The first place in EPIC-Kitchen Action Recognition 2019. See our report.
• Our report on ActivityNet Trimmed Action Recognition 2017.
• Our report on Google YouTube8M Classification 2017.
• The first place in the video localization competition, TRECVID 2016. See our report.
• The first place in the Action Recognition competition, THUMOS 2015. See our notebook paper.