Sibei Yang

prof.jpeg

Sibei Yang is an Associate Professor and PhD Supervisor at the HCP Lab, School of Computer Science and Engineering, Sun Yat-sen University. Prior to this, she was a Tenure-track Assistant Professor at ShanghaiTech University from 2021 and a Research Assistant Professor at The Hong Kong Polytechnic University since 2020. She received her Ph.D. from the University of Hong Kong in 2020, under the supervision of Prof. Yizhou Yu, with support from the Hong Kong PhD Fellowship. Dr. Yang obtained her B.S. in Computer Science from the Chu Kochen Honors College at Zhejiang University in 2016.

She has published over 60 papers in leading venues such as TPAMI, CVPR, ICCV, ECCV, NeurIPS, ICLR, and SIGGRAPH. As the first or corresponding author, Sibei Yang has contributed to more than 30 papers in top-tier (CCF-A) journals and conferences.

Research Interests

Our current research interests primarily focus on 1) Multimodal Large Language Models (LLM/MLLM), 2) Vision-Language Understanding and Generation, 3) Embodied AI, and 4) Open-world Visual Understanding. Our mission is to facilitate the learning of unified and universal perception, understanding, reasoning, generation, and interaction within the realm of an open world. We envision that harnessing multimodal information—particularly vision and language—in a unified and generalizable manner, and enabling its deployment in real-world physical interactions, is fundamental to advancing our understanding of the world and fostering the emergence of artificial intelligence.

我们正在招2026秋季入学的硕士生和博士生，欢迎感兴趣的同学附上简历联系我sibeiyang9@gmail.com。如果想从学生视角了解我们可以联系组内师兄石骋shicheng2025@connect.hku.hk


News

Oct, 2025 Congratulations to Yulin Zhang and Jiajin Tang for receiving the National Scholarship👍
Sep, 2025 Sibei Yang is listed in the Stanford University/Elsevier “World’s Top 2% Scientists”
Sep, 2025 5 papers are accepted by NeurIPS 2025 🎊🎊🎊
Aug, 2025 Sibei Yang will serve as Area Chair for ICLR 2026
Jun, 2025 7 papers are accepted by ICCV 2025 👏👏👏
May, 2025 2 papers are accepted by ACL 2025 🎉🎉
Feb, 2025 6 papers are accepted by CVPR 2025 🥳🥳🥳
Jan, 2025 3 papers are accepted by ICLR 2025 🎊🎊🎊
Dec, 2024 Sibei Yang will serve as Area Chair for ICCV 2025
Oct, 2024 Congratulations to Cheng Shi for once again receiving the National Scholarship! 🎉🎉🎉
Aug, 2024 One Paper is accepted by TPAMI 2024 👏👏👏
Jul, 2024 Three papers are accepted by ECCV 2024 🎉🎉🎉
Feb, 2024 Two papers are accepted by CVPR 2024 🎊🎊🎊
Jan, 2024 One paper is accepted by ICLR 2024 🐲🐲🐲
Dec, 2023 Congratulations to Cheng Shi for receiving the National Scholarship, and to Jiajin Tang for achieving the Outstanding Student Award.👏👏👏
Sep, 2023 2 papers Free-Bloom (Zero-Shot Text-to-Video Generation) and DDCoT (CoT Prompting for Multimodal Reasoning in LMs) are accepted by NeurIPS 2023 🎉🎉🎉
Jul, 2023 5 papers are accepted by ICCV 2023 🎉
May, 2023 Sibei Yang will serve as Area Chair for WACV2024.
May, 2023 Our website is released! Thanks to Yufan, Hanzhuo and Yuchen :)

Recent Publication

* equal contribution; † corresponding author.

2025

  1. VFL.png
    Vision Function Layer in Multimodal LLMs
    Cheng Shi, Yizhou Yu, and Sibei Yang†
    Accepted by NeurIPS, 2025
    arXiv PDF Code
  2. EyeWO.png
    Eyes Wide Open: Ego Proactive Video-LLM for Streaming Video
    Yulin Zhang, Cheng Shi, Yang Wang, Sibei Yang†
    Accepted by NeurIPS, 2025
    arXiv HTML PDF
  3. 1+1.png
    Discovering Compositional Hallucination in LVLMs
    Sibei Yang†, Ge Zheng, Jiajin Tang, Jiaye Qian, Hanzhuo Huang, Cheng Shi
    Accepted by NeurIPS, 2025
    PDF
  4. allpath.png
    Intervene-All-Paths: Unified Mitigation of LVLM Hallucinations across Alignment Formats
    Jiaye Qian, Ge Zheng, Yuchen Zhu, Sibei Yang†
    Accepted by NeurIPS, 2025
    arXiv PDF Code
  5. autosearch.png
    Auto-Search and Refinement: An Automated Framework for Gender Bias Mitigation in Large Language Models
    Xu Yue, Chengyan Fu, Li Xiong, Sibei Yang, Wenjie Wang
    Accepted by NeurIPS, 2025
    arXiv PDF
  6. Last-vit.jpg
    Vision Transformer Needs More Than Register
    Cheng Shi, Yizhou Yu, and Sibei Yang†
    In submission, 2025
    PDF
  7. Sim-DETR.png
    Sim-DETR: Unlock DETR for Temporal Sentence Grounding
    Jiajin Tang*, Zhengxuan Wei*, Yuchen Zhu, Cheng Shi, Guanbin Li, Liang Lin, Sibei Yang†
    Accepted by ICCV, 2025
    arXiv PDF
  8. HalTrapper.png
    Why LVLMs Are More Prone to Hallucinations in Longer Responses: The Role of Context
    Ge Zheng*, Jiaye Qian*, Jiajin Tang, Sibei Yang†
    Accepted by ICCV, 2025
    arXiv PDF
  9. HOID.png
    No More Sibling Rivalry: Debiasing Human-Object Interaction Detection
    Bin Yang*, Yulin Zhang*, Hong-Yu Zhou, Sibei Yang†
    Accepted by ICCV, 2025
    arXiv PDF
  10. Closed-loop.png
    Closed-Loop Transfer for Weakly-supervised Affordance Grounding
    Jiajin Tang*, Zhengxuan Wei*, Ge Zheng, Sibei Yang†
    Accepted by ICCV, 2025
    arXiv PDF
  11. AMR.png
    Augmenting Moment Retrieval: Zero-Dependency Two-Stage Learning
    Zhengxuan Wei*, Jiajin Tang*, and Sibei Yang†
    Accepted by ICCV, 2025
    arXiv PDF Code
  12. ICCV2025
    Penalizing Boundary Activation for Object Completeness in Diffusion Models
    Haoyang Xu, Tianhao Zhao, Sibei Yang, Yutian Lin
    Accepted by ICCV, 2025
    arXiv PDF Code
  13. ICCV2025
    VLDrive: Vision-Augmented Lightweight MLLMs for Efficient Language-grounded Autonomous Driving
    Ruifei Zhang, Wei Zhang, Xiao Tan, Sibei Yang, Xiang Wan, Xiaonan Luo, Guanbin Li
    Accepted by ICCV, 2025
    arXiv PDF Code
  14. TTA.jpg
    Free on the Fly: Enhancing Flexibility in Test-Time Adaptation with Online EM
    Qiyuan Dai, and Sibei Yang†
    Accepted by CVPR, 2025
    arXiv PDF
  15. GCD.png
    Adaptive Part Learning for Fine-Grained Generalized Category Discovery: A Plug-and-Play Enhancement
    Qiyuan Dai, Hanzhuo Huang, Yu Wu, Sibei Yang†
    Accepted by CVPR, 2025
    arXiv PDF
  16. Rethinking.png
    Rethinking Query-based Transformer for Continual Image Segmentation
    Yuchen Zhu*, Cheng Shi*, Dingyou Wang, Jiajin Tang, Zhengxuan Wei, Yu Wu, Sibei Yang†
    Accepted by CVPR, 2025
    arXiv PDF Code
  17. SegAfford.png
    SeqAfford: Sequential 3D Affordance Reasoning via Multimodal Large Language Model
    Chunlin Yu*, Hanqing Wang*, Ye Shi, Haoyang Luo, Sibei Yang, Jingyi Yu, Jingya Wang
    Accepted by CVPR, 2025
    arXiv HTML Code
  18. vton.png
    VTON 360: High-fidelity virtual try-on from any viewing direction
    Zijian He, Yuwei Ning, Yipeng Qin, Guangrun Wang, Sibei Yang, Liang Lin, Guanbin Li
    Accepted by CVPR, 2025
    arXiv HTML Code
  19. dissecting.png
    Dissecting and Mitigating Diffusion Bias via Mechanistic Interpretability
    Yingdong Shi, Changming Li, Yifan Wang, Yongxiang Zhao, Anqi Pang, Sibei Yang, Jingyi Yu, Kan Ren
    Accepted by CVPR, 2025
    arXiv HTML
  20. delman.jpg
    DELMAN: Dynamic Defense Against Large Language Model Jailbreaking with Model Editing
    Yi Wang, Fenghua Weng, Sibei Yang, Zhan Qin, Minlie Huang, Wenjie Wang
    Accepted by ACL, 2025
    arXiv PDF Code
  21. DSN.png
    Don’t Say No: Jailbreaking LLM by Suppressing Refusal
    Yukai Zhou, Jian Lou, Zhijie Huang, Zhan Qin, Sibei Yang, Wenjie Wang
    Accepted by ACL, 2025
    arXiv PDF Code
  22. mvtokenflow.gif
    MVTokenFlow: High-quality 4D Content Generation using Multiview Token Flow
    Hanzhuo Huang*, Yuan Liu*, Ge Zheng, Jiepeng Wang, Zhiyang Dou, Sibei Yang†
    Accepted by ICLR, 2025
    arXiv HTML Code
  23. cityanchor.png
    CityAnchor: City-scale 3D Visual Grounding with Multi-modality LLMs
    jinpeng Li*, Haiping Wang*, Jiabin Chen, Yuan Liu, Zhiyang Dou, Yuexin Ma, Sibei Yang, Yuan Li, Wang Wenping, Zhen Dong, Bisheng Yang
    Accepted by ICLR, 2025
    PDF Code
  24. ICLR2025
    Discovering Influential Neuron Path in Vision Transformers
    Yifan Wang, Yifei Liu, Yingdong Shi, Changming Li, Anqi Pang, Sibei Yang, Jingyi Yu, Kan Ren
    Accepted by ICLR, 2025
    arXiv Code

2024

  1. TPAMI2024
    A Survey on Graph Neural Networks and Graph Transformers in Computer Vision: A Task-oriented Perspective
    Chaoqi Chen*, Yushuang Wu*, Qiyuan Dai*, Hong-Yu Zhou*, Mutian Xu, Sibei Yang†, Xiaoguang Han†, Yizhou Yu†
    Accepted by TPAMI, 2024
    arXiv
  2. Part2Object.png
    Part2Object: Hierarchical Unsupervised 3D Instance Segmentation
    Cheng Shi*, Yulin Zhang*, Bin Yang, Jiajin Tang, Yuexin Ma, Sibei Yang†
    Accepted by ECCV, 2024
    arXiv Code
  3. Plain-D.png
    Plain-DNet: A Plain Multi-Dataset Object Detector
    Cheng Shi*, Yuchen Zhu*, and Sibei Yang†
    Accepted by ECCV, 2024
    arXiv Code
  4. wildrefer.png
    WildRefer: 3D Object Localization in Large-scale Dynamic Scenes with Multi-modal Visual Data and Natural Language
    Zhenxiang Lin, Xidong Peng, Peishan Cong, Ge Zheng, Yujing Sun, Yuenan Hou, Xinge Zhu, Sibei Yang, Yuexin Ma
    Accepted by ECCV, 2024
    arXiv Code
  5. CVPR2024
    Curriculum Point Prompting for Weakly-Supervised Referring Image Segmentation
    Qiyuan Dai, and Sibei Yang†
    Accepted by CVPR, 2024
    arXiv
  6. devil.png
    The Devil is in the Object Boundary: Towards Annotation-free Instance Segmentation Using Foundation Models
    Cheng Shi, and Sibei Yang†
    Accepted by ICLR, 2024
    arXiv Code
  7. omg.png
    OMG: Towards Open-vocabulary Motion Generation via Mixture of Controllers
    Han Liang, Jiacheng Bao, Ruichi Zhang, Sihan Ren, Yuecheng Xu, Sibei Yang, Xin Chen, Jingyi Yu, Lan Xu
    Accepted by CVPR, 2024
    arXiv Code Video
  8. RealDex.png
    RealDex: Towards Human-like Grasping for Robotic Dexterous Hand
    Yumeng Liu*, Yaxun Yang*, Youzhuo Wang*, Xiaofei Wu, Jiamin Wang, Yichen Yao, Sören Schwertfeger, Sibei Yang, Wenping Wang, Jingyi Yu, Xuming He, Yuexin Ma
    Accepted by IJCAI, 2024
    arXiv

2023

  1. ddcot.jpg
    DDCoT: Duty-Distinct Chain-of-Thought Prompting for Multimodal Reasoning in Language Models
    Ge Zheng*, Bin Yang*, Jiajin Tang*, Hong-Yu Zhou, Sibei Yang†
    Accepted by NeurIPS, 2023
    arXiv Code
  2. flower_bloom.gif
    Free-Bloom: Zero-Shot Text-to-Video Generator with LLM Director and LDM Animator
    Hanzhuo Huang*, Yufan Feng*, Cheng Shi, Lan Xu, Jingyi Yu, Sibei Yang†
    Accepted by NeurIPS, 2023
    arXiv Code
  3. ICCV2023
  4. ICCV2023
    EdaDet: Open-Vocabulary Object Detection Using Early Dense Alignment
    Cheng Shi, and Sibei Yang†
    Accepted by ICCV, 2023
    arXiv HTML PDF
  5. ICCV2023
    CoTDet: Affordance Knowledge Prompting for Task Driven Object Detection
    Jiajin Tang*, Ge Zheng*, Jingyi Yu, Sibei Yang†
    Accepted by ICCV, 2023
    arXiv HTML PDF
  6. ICCV2023
    Temporal Collection and Distribution for Referring Video Object Segmentation
    Jiajin Tang, Ge Zheng, and Sibei Yang†
    Accepted by ICCV, 2023
    HTML PDF
  7. ICCV2023
    Grounded lmage Text Matching with Mismatched Relation Reasoning
    Yu Wu*, Yana Wei*, Haozhe Wang, Yongfei Liu, Sibei Yang, Xuming He†
    Accepted by ICCV, 2023
    arXiv
  8. CVPR2023
    Contrastive Grouping with Transformer for Referring Image Segmentation
    Jiajin Tang, Ge Zheng, Cheng Shi, Sibei Yang†
    Accepted by CVPR, 2023
    PDF Code
  9. AAAI2023
    CCQ: Cross-Class Query Network for Partially Labeled Organ Segmentation
    Xuyang Liu, Bingbing Wen, and Sibei Yang†
    Accepted by AAAI, 2023
    HTML Code
  10. SIGGRAPH2023
    DreamFace: Progressive Generation of Animatable 3D Faces under Text Guidance
    Longwen Zhang*, Qiwei Qiu*, Hongyang Lin*, Qixuan Zhang, Cheng Shi, Wei Yang, Ye Shi, Sibei Yang†, Lan Xu†, Jingyi Yu†
    Accepted by SIGGRAPH, 2023
    arXiv HTML PDF Video
  11. TPAMI2023
    A Unified Visual Information Preservation Framework for Self-supervised Pre-training in Medical Image Analysis
    Hong-Yu Zhou*, Chixiang Lu*, Chaoqi Chen, Sibei Yang, Yizhou Yu†
    Accepted by TPAMI, 2023
    arXiv

2022

  1. ECCV2022
    Spatial and Visual Perspective-Taking via View Rotation and Relation Reasoning for Embodied Reference Understanding
    Cheng Shi, and Sibei Yang†
    Accepted by ECCV, 2022
    PDF Code