音频语言模型与多语言评测
在 A*STAR I²R，我主导 MERaLiON AudioLLM 的数据准备与评测，并在六人团队中共同负责模型训练，单次训练使用数百张 H100 GPU。这项工作属于新加坡国家多模态大语言模型计划。
- **AudioBench：**主导覆盖 8 类任务、26 个数据集的音频语言评测基准。第一作者，NAACL 2025。
- **SeaEval：**主导跨语言一致性与文化推理评测。共同第一作者，NAACL 2024。
- **Singlish 语音数据：**主导语料整理与发布。
我也研究对话摘要，并参与多语言指令微调与多模态评测合作。
发表论文
- MERaLiON-AudioLLM: Advancing Speech and Language Understanding for Singapore — ACL 2025, System Demonstrations
- AudioBench: A Universal Benchmark for Audio Large Language Models — NAACL 2025
- Instructive Dialogue Summarization with Query Aggregations — EMNLP 2023
- CRAFT: Extracting and Tuning Cultural Instructions from the Wild — C3NLP 2024
- In2Core: Leveraging Influence Functions for Coreset Selection in Instruction Finetuning of Large Language Models — EMNLP Findings 2024
- Resilience of Large Language Models for Noisy Instructions — EMNLP Findings 2024
- CrossIn: An Efficient Instruction Tuning Approach for Cross-Lingual Knowledge Alignment — SUMEval 2025
- SeaEval for Multilingual Foundation Models: From Cross-Lingual Alignment to Cultural Reasoning — NAACL 2024
- SEACrowd: A Multilingual Multimodal Data Hub and Benchmark Suite for Southeast Asian Languages — EMNLP 2024
- MoWE-Audio: Multitask AudioLLMs with Mixture of Weak Encoders — ICASSP 2025
- CoinMath: Harnessing the Power of Coding Instruction for Math LLM — ACL Findings 2025
- Optimizing Cross-Modality Alignment Module for Audio Large Language Models — Data Intelligence 2025
- MNSC: Advancing Singlish Speech Understanding with Carefully Curated Corpora — ASRU 2025
- Crowdsource, Crawl, or Generate? Creating SEA-VL, a Multicultural Vision-Language Dataset for Southeast Asia — ACL 2025
- NTU Speechlab LLM-Based Multilingual ASR System for Interspeech MLC-SLM Challenge 2025 — MLC-SLM 2025
- Diversity and Complementarity of Speech Encoders across Diverse Tasks in a Multi-modal Large Language Model — ASRU 2025
- Towards Spoken Mathematical Reasoning: Benchmarking Speech-based Models over Multi-faceted Math Problems — arXiv 2025
- IFEval-Audio: Benchmarking Instruction-Following Capability in Audio-based Large Language Models — AACL 2025
- Beyond Classification: Towards Speech Emotion Reasoning with Multitask AudioLLMs — AACL 2025
- Train Multi-Modal LLMs to Understand Diverse Speech Paralinguistics by Distilling from Teachers with Meta-Information — AAAI 2026 Workshop on Audio-Centric AI
指导学生
- Pham The Binh Minh — 本科研究实习生，新加坡南洋理工大学 (2025-01 – 2025-05)。多模态 AudioLLM。
- Yiming Gao — 本科研究实习生，新加坡南洋理工大学 (2025-01 – 2025-05)。多模态大语言模型的指令跟随能力研究。(AACL 2025)
- Tey Xue Cong — A*STAR 学者实习生，新加坡义安理工学院 (2025-02 – 2025-04)。联合指导：Xunlong Zou。多语言语音数据采集与处理。
- Jayden Lum — A*STAR 学者实习生，新加坡义安理工学院 (2025-02 – 2025-04)。联合指导：Xunlong Zou。多语言语音数据采集与处理。
- Yanchao Li — ACIS 博士学者，新加坡南洋理工大学 (2024-01 – 2025-04)。联合指导：Nancy F. Chen。长视频理解。
- Ziyi Xu — 研究实习生，新加坡国立大学 (2024-07 – 2024-12)。联合指导：Sun Shuo。多模态对齐数据的采集与筛选。
- Ayrton San Joaquin — 研究助理，新加坡 DesCarte@CREATE (2023-09 – 2024-08)。基于梯度估计的大语言模型高效训练。(EMNLP 2024 Findings)
- Anh Thuc Nguyen — 研究实习生，美国北卡罗来纳大学教堂山分校 (2024-01 – 2024-05)。MERaLiON 项目的问题生成与评测数据集构建。
学术服务
- 出版主席：EMNLP 2023
- 本地组织委员会：EMNLP 2023
- 领域主席：ACL ARR (2024–2025)
- 编委：APSIPA Transactions on Signal and Information Processing（2023–2025）
- 审稿人：ACL、EMNLP、NAACL、ICASSP、IEEE TASLP
获奖
- 最佳论文奖 — SUMEval Workshop, COLING 2025
- 最佳论文奖 — C3NLP Workshop, ACL 2024
视频
- MERaLiON 项目介绍 — MERaLiON 项目总体介绍。youtube.com/embed/nBA3MqwjN3I
- MERaLiON 演示 — MERaLiON AudioLLM 能力演示。youtube.com/embed/HZSa7vT73Lg
演讲
- 2025.03 — Lorong AI，新加坡。Evaluation on Audio-LLMs and Beyond. 讲稿