Training-Free Multimodal Large Language Model Orchestration
A language-model controller coordinates modality experts, shared cross-modal memory, and full-duplex streaming interaction without additional integration training.
I am a master's student in Artificial Intelligence at Xiamen University, working with Prof. Xiawu Zheng in the MAC Lab. I also conduct research on pre-training and post-training at a leading AI startup.
My research interests lie in omni-modal modeling and full-duplex human–AI interaction. I focus on real-time multimodal understanding, cross-modal context and memory, and the evaluation of interactive models.
Join Our Group
Our group at Xiamen University, led by Prof. Xiawu Zheng, welcomes students interested in multimodal models and agents, automated machine learning, model compression, and efficient deployment. Visit Prof. Zheng's homepage to learn about our research, recent work, and group members.
To apply, please send your CV to zhengxiawu@xmu.edu.cn and cc teery@xmumac.com and maqingchuan@xmumac.com, along with a brief introduction to your research interests and relevant experience.
News
- : Our SCOPE preprint on semantic credit assignment for text-to-image prompt optimization is now on arXiv.
- : RoTA-Bench is now on arXiv, evaluating 3D spatial reasoning through interactive projection alignment.
- : I'm serving as a main conference reviewer for ICLR 2027.
- : Our preprint Duplex-MPE is now on arXiv: a benchmark for multi-party full-duplex dialogue.
- : Updated SocialOmni with a revised evaluation protocol and reproducibility materials.
- : WaveZip, our preprint on efficient video token compression, has been updated.
- : Our preprint on perception–decision misalignment in omni models is now on arXiv.
- : Training-Free Orchestration was accepted to ICML 2026.
- : A2RBench was accepted to ICML 2026.
- : Our Event-Anchored Frame Selection preprint is available, exploring event-aware sampling for long-video understanding.
- : Our CrystaL preprint is now on arXiv, studying visual latent reasoning in multimodal models.
- : WFS-SB was accepted to CVPR 2026.
- : Our MLLMEmbed-ReID preprint is now on arXiv, unifying cross-modal retrieval and knowledge distillation for edge deployment.
- : Joined Xiamen University MAC Lab as a master's student in Artificial Intelligence.
Selected Publications
Full list: publication archive · Google Scholar
SocialOmni: Benchmarking Audio-Visual Social Interactivity in Omni Models
An offline benchmark separating speaker identification, turn-entry decisions, and response generation, with 2,000 perception items and 200 interaction-generation items.
RoTA-Bench: Evaluating 3D Spatial Reasoning through Interactive Projection Alignment
A benchmark of interactive 3D spatial reasoning: models use visual feedback to rotate objects until their projections match target silhouettes, across 674 tasks and a fixed 100-task test set.
Duplex-MPE: Benchmarking Multi-Party Interaction in Full-Duplex Dialogue
Continuous-audio evaluation of when an assistant should answer, stay silent, or stop speaking in multi-party full-duplex dialogue.
WaveZip: Wavelet-Driven Space-Time Decoupling for Video Token Condensation
Training-free video token compression using wavelets to separate temporal redundancy from spatial detail.
Wavelet-based Frame Selection by Detecting Semantic Boundary for Long Video Understanding
Wavelet-based semantic boundaries guide adaptive frame allocation for efficient long-video understanding.
Education
- 2025.09 – present, M.S. student in Artificial Intelligence, Xiamen University (Project 985, Project 211). Advisor: Prof. Xiawu Zheng.
- 2021.09 – 2025.06, B.S. in Data Science and Big Data Technology, Sichuan Agricultural University (Project 211, Double First-Class).
Academic Service
- Main Conference Reviewer, ICLR 2027.
- Program Committee Member, MultiMedia Modeling (MMM) 2027.
- Main Conference Reviewer, NeurIPS 2026.
Research Engineering
I have submitted over 4,000 lines of code to SGLang-Omni and vLLM-Omni. My engineering work covers model integration, evaluation automation, regression testing, and inference optimization.
I work with PyTorch, Megatron-LM, DeepSpeed, and LLaMA-Factory for distributed training, supervised fine-tuning, and reinforcement learning, including multi-GPU training on Slurm clusters.