Rongyao Fang 方荣耀 Ph.D. Candidate Multimedia Laboratory (MMLab) Department of Electronic Engineering, The Chinese University of Hong Kong (CUHK) Email: rongyaofang@gmail.com [Google Scholar]    [GitHub]    [CV]

Biography

I am a Ph.D. candidate at the Multimedia Laboratory (MMLab) of The Chinese University of Hong Kong (CUHK), fortunate to be supervised by Prof. Hongsheng Li. I also work closely with Prof. Xihui Liu. I anticipate completing my doctorate in 2025.

My research is driven by a passion for Artificial General Intelligence (AGI) with a focus on visual understanding and generation. I am dedicated to developing integrated systems capable of perceiving, understanding, and generating visual content by leveraging advanced techniques with Multimodal Large Language Models.

Previously, I was a visiting scholar at MIT CSAIL, advised by Prof. Dina Katabi. I obtained my B.Eng. degree from Shanghai Jiao Tong University, where I was ranked 1st/157 and advised by Prof. Bingbing Ni.

Education

• [2021 - 2025 (Exp.)] Ph.D. candidate at MMLab, The Chinese University of Hong Kong.
• [2016 - 2020] B.Eng. in Information Engineering, Shanghai Jiao Tong University (Ranking: 1st/157).
• [2019 - 2020] Visiting Scholar at CSAIL, Massachusetts Institute of Technology.

Publications

(* indicates equal contribution)

• GoT-R1: Unleashing Reasoning Capability of MLLM for Visual Generation with Reinforcement Learning
  C Duan*, Rongyao Fang*, Y Wang*, K Wang, L Huang, X Zeng, H Li, X Liu
  arXiv preprint, 2025
• GoT: Unleashing Reasoning Capability of Multimodal Large Language Model for Visual Generation and Editing
  Rongyao Fang, C Duan, K Wang, L Huang, H Li, S Yan, H Tian, X Zeng, R Zhao, J Dai, X Liu, H Li
  arXiv preprint, 2025
• CrossLMM: Decoupling Long Video Sequences from LMMs via Dual Cross-Attention Mechanisms
  S Yan, J Han, J Tsai, H Xue, Rongyao Fang, L Hong, Z Guo, R Zhang
  arXiv preprint, 2025
• SOLVE: Synergy of Language-Vision and End-to-End Networks for Autonomous Driving
  X Chen, L Huang, T Ma, Rongyao Fang, S Shi, H Li
  Conference on Computer Vision and Pattern Recognition (CVPR), 2025
• StreamChat: Chatting with Streaming Video
  J Liu, Z Yu, S Lan, S Wang, Rongyao Fang, J Kautz, H Li, JM Alvare
  arXiv preprint, 2024
• Puma: Empowering Unified MLLM with Multi-Granular Visual Generation
  Rongyao Fang, C Duan, K Wang, H Li, H Tian, X Zeng, R Zhao, J Dai, H Li, X Liu
  International Conference on Computer Vision (ICCV), 2025
• Mimic Before Reconstruct: Enhancing Masked Autoencoders with Feature Mimicking
  P Gao, Z Lin, R Zhang, Rongyao Fang, H Li, H Li, Y Qiao
  International Journal of Computer Vision (IJCV), 2024
• FeatAug-DETR: Enriching One-to-Many Matching for DETRs with Feature Augmentation
  Rongyao Fang, P Gao, A Zhou, Y Cai, S Liu, J Dai, H Li
  IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 2024
• FouriScale: A Frequency Perspective on Training-Free High-Resolution Image Synthesis
  L Huang*, Rongyao Fang*, A Zhang, G Song, S Liu, Y Liu, H Li
  European Conference on Computer Vision (ECCV), 2024
• Clip-Adapter: Better Vision-Language Models with Feature Adapters
  P Gao, S Geng, R Zhang, T Ma, Rongyao Fang, Y Zhang, H Li, Y Qiao
  International Journal of Computer Vision (IJCV), 2024
• InstructSeq: Unifying Vision Tasks with Instruction-Conditioned Multi-Modal Sequence Generation
  Rongyao Fang, S Yan, Z Huang, J Zhou, H Tian, J Dai, H Li
  arXiv preprint, 2023
• Point-M2AE: Multi-Scale Masked Autoencoders for Hierarchical Point Cloud Pre-Training
  R Zhang, Z Guo, P Gao, Rongyao Fang, B Zhao, D Wang, Y Qiao, H Li
  Conference on Neural Information Processing Systems (NeurIPS), 2022
• RBGNet: Ray-Based Grouping for 3D Object Detection
  H Wang, S Shi, Z Yang, Rongyao Fang, Q Qian, H Li, B Schiele, L Wang
  Conference on Computer Vision and Pattern Recognition (CVPR), 2022
• Tip-Adapter: Training-Free CLIP-Adapter for Better Vision-Language Modeling
  R Zhang*, Rongyao Fang*, P Gao*, W Zhang, K Li, J Dai, Y Qiao, H Li
  European Conference on Computer Vision (ECCV), 2022
• Learning Longterm Representations for Person Re-Identification Using Radio Signals
  L Fan*, T Li*, Rongyao Fang*, R Hristov, Y Yuan, D Katabi
  Conference on Computer Vision and Pattern Recognition (CVPR), 2020
• Probabilistic Radiomics: Ambiguous Diagnosis with Controllable Shape Analysis
  J Yang*, Rongyao Fang*, B Ni, Y Li, Y Xu, L Li
  International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI), 2019
• Adversarial Attack and Defense on Point Sets
  J Yang*, Q Zhang*, Rongyao Fang*, B Ni, J Liu, Q Tian
  arXiv preprint, 2019

Experience

• [Feb. 2024 - Present] Research Intern, SenseTime.
• [Jun. 2022 - Apr. 2023] Research Intern, Shanghai AI Laboratory.

Selected Awards

• [2021] Hong Kong PhD Fellowship.
• [2020] Outstanding Graduates of Shanghai (Top 1%).
• [2017 & 2018] National Scholarship (Top 1%).
• [2017 & 2018] Zhiyuan College Honors Scholarship (Top 5%).