a.k.a. Oliver
I'm currently a Research Fellow at National University of Singapore with Prof. Michael Shieh. I obtained my Ph.D. from Nanyang Technological University, Singapore, with Prof. Erik Cambria.
I'm interested in
When I'm not deeply engrossed in research, you can find me indulging in powerlifting, capturing moments through photography, playing ball games, or strategizing over a game of poker.
Google Scholar / X (Twitter) / Blog Posts / Github / LinkedIn / Zhi Hu / Email / CV
I'm open to research collaborations, please feel free to drop me an email if you have any interesting ideas.
| 2024.10 | 🔥🔥🔥 MixEval-X is released! Checkout the Project Website and Paper to learn more about real-world any-to-any evaluation! |
| 2024.09 | 🏄🏻♀️ MixEval is accepted to NeurIPS'24. |
| 2024.06 | 📰 MixEval is available on arXiv. We also provide rigorous insights and meta-evaluations for LLM evaluation, check it out! |
| 2024.06 | 🔥🔥🔥 MixEval is released! Checkout the Project Website and Paper to learn more about this reliable, holistic, and efficient LLM evaluation! |
| 2023.02 | 🏄🏻♀️ Got OpenMoE accepted to ICML'24! Congrats to Fuzhao and the team! |
| 2024.01 | 🔥🔥🔥 The OpenMoE technical report is available on arXiv, check it out! |
| 2023.12 | 🏄🏻♀️ I will be in Singapore for EMNLP'23 🇸🇬! Drop me a message if you are interested in chatting with me! |
| 2023.12 | 🗣️ Gave a talk at DAI'23, with the topic "Learning Lighter Transformer Archtectures". |
| 2023.07 | 🎓 Thrilled to be conferred with the Ph.D. degree! Thank everyone that offered valuable support and help in my academic journey. And cheers, Dr. Ni 🎊! |
| 2023.07 | 🧗🏻♂️ New journey! I'm joining HPC-AI, SoC, NUS as a Research Fellow, working with Prof. Yang You! |
| 2023.05 | 🏄🏻♀️ Got one paper (GHA) on optimizing transformer architecture accepted to ACL'23! Congrats to me and all co-authors! See you in Toronto! |
| 2023.03 | 📰 Our paper reviewing Natural Language Reasoning is available on arXiv, check it out! |
| 2023.02 | 🏄🏻♀️ Got three papers (PAD, APLY, and de'HuBERT) accepted to ICASSP'23! Congrats to Yukun, Dianwen, me, and all co-authors! See you in Greece! |
| 2022.09 | 📰 Our paper on Graph Representation Learning is available on arXiv, check it out! |
| 2022.06 | 📰 The 80-page review on conversational agents is accepted to Artificial Intelligence Review. Congrats to me and all co-authors! |
| 2022.04 | 🧗🏻♀️ Glad to start my internship at DAMO, Alibaba SG! |
| 2022.01 | 🏄🏻♀️ Got one paper on Dialogue Relation Extraction accepted by ICASSP'22. Congrats to Fuzhao! |
| 2021.12 | 🏄🏻♀️ Got two papers (HITKG, FusedChat) accepted to AAAI'22! Congrats to Tom, me, and all co-authors! See you in Vancouver! |
| 2020.08 | 🧗🏻♂️ New journey! I'm joining SCSE, NTU as a Ph.D., working with Prof. Erik Cambria! |
Experiences |
|
Academia |
|
| 2023 |
National University of Singapore
2023 - now
|
| Research Fellow | |
| - Foundation Models. | |
| 2020 |
Nanyang Technological University
2020 - 2023
|
| Ph.D. in Computer Science | |
| - Efficient Language Models and Dialogue Agents. | |
| 2019 |
Harvard University, Institute for Applied Computational Science
|
| Research Assistant (remote) | |
| - VAE-GAN variants. | |
| 2016 |
Northwestern Polytechnical University
2016 - 2020
|
| B.Eng. in Electrical Engineering | |
| - Multimodal Models. | |
Industry |
|
| 2022 |
DAMO Academy, Alibaba Group, Singapore
2022.04 - 2022.10
|
| Research Intern | |
| - In charge of modality alignment for pre-trained models. Worked with Dr. Yukun Ma. | |
| 2020 |
Continental-NTU Corp Lab
2020.09 - 2022.08
|
| Research Intern | |
| - In charge of fusing task-oriented and open-domain dialogue agents. Worked with Dr. Rui Mao. | |
| 2018 |
Chinese Academy of Sciences
|
|
Institute of Automation
2020.02 - 2020.06
|
|
| Research Intern | |
| - In charge of anchor-free position estimation and object detection. Worked with Dr. Sen Xin. | |
|
Institute of Computing Technology
2018.10 - 2018.11
|
|
| Research Intern | |
| - Training abstractive summarization models. Worked with Dr. Shuai Jiao. | |
Featured Research |
|
| 2024 | MixEval-X 🔥 |
| - MixEval-X: Any-to-Any Evaluations from Real-World Data Mixtures [paper][Tweet] | |
| - Jinjie Ni, Yifan Song, Deepanway Ghosal, Bo Li, David Junhao Zhang, Xiang Yue, Fuzhao Xue, Zian Zheng, Kaichen Zhang, Mahir Shah, Kabir Jain, Yang You, Michael Qizhe Shieh | |
| - MixEval-X is the first any-to-any, real-world benchmark featuring diverse input-output modalities, real-world task distributions, consistent high standards across modalities, and dynamism. It achieves up to 0.98 correlation with arena-like multi-modal evaluations while being way more efficient. | |
| MixEval 🔥 | |
| - MixEval: Deriving Wisdom of the Crowd from LLM Benchmark Mixtures [paper][Tweet] | |
| - NeurIPS 2024 main track (poster) | |
| - Jinjie Ni, Fuzhao Xue, Xiang Yue, Yuntian Deng, Mahir Shah, Kabir Jain, Graham Neubig, Yang You | |
| - Building golden-standard LLM evaluation from off-the-shelf benchmark mixtures. The best LLM evaluation at the time of release for its SOTA model ranking accuracy (0.96 correlation with Chatbot Arena) and efficiency (6% the time and cost of running MMLU). Moreover, it’s dynamic. | |
| OpenMoE 🔥 | |
| - OpenMoE: An Early Effort on Open Mixture-of-Experts Language Models [paper][Tweet] | |
| - ICML 2024 (poster) | |
| - Fuzhao Xue, Zian Zheng, Yao Fu, Jinjie Ni, Zangwei Zheng, Wangchunshu Zhou, Yang You | |
| - The first fully open MoE-based Decoder-only LLM trained over chinchilla scaling law. | |
| 2023 | InstructWild |
| - Instruction in the Wild: A User-Based Instruction Dataset [Github] | |
| - Jinjie Ni, Fuzhao Xue, Yuntian Deng, Jason Phang, Kabir Jain, Mahir Hitesh Shah, Zangwei Zheng, Yang You. | |
| - The first large-scale instruction tuning dataset harvested from the web. | |
| GHA | |
| - Finding the Pillars of Strength for Multi-head Attention. [paper] | |
| - ACL 2023 main track (poster) | |
| - Jinjie Ni, Rui Mao, Zonglin Yang, Han Lei, Erik Cambria | |
| - Cutting off redundancy for Transformer layers. SOTA efficiency and performance among efficient transformers. Concurrent work of GQA, cited and discussed in the GQA paper. | |
| PAD | |
| - Adaptive Knowledge Distillation between Text and Speech Pre-trained Models [paper] | |
| - Jinjie Ni, Yukun Ma, Wen Wang, Qian Chen, Dianwen Ng, Han Lei, Trung Hieu Nguyen, Chong Zhang, Bin Ma, Erik Cambria | |
| - Knowledge distillation between text and speech pre-trained models. The SOTA text-speech distillation method at the time of release. | |
| 2022 | HiTKG |
| - HiTKG: Towards Goal-Oriented Conversations Via Multi-Hierarchy Learning [paper] | |
| - AAAI 2022 (oral) | |
| - Jinjie Ni, Vlad Pandelea, Tom Young, Haicang Zhou, Erik Cambria | |
| - The first work that trains agents to actively guide the conversations. It ushers in a new era of intelligence for dialogue agents. The SOTA approach for turn-level dialogue reasoning tasks. | |
| FusedChat | |
| - FusedChat: Towards Fusing Task-Oriented Dialogues and Chitchat in Multi-turn Conversational Agents [paper] | |
| - AAAI 2022 (oral) | |
| - Tom Young, Frank Xing, Vlad Pandelea, Jinjie Ni, Erik Cambria | |
| - The first attempt of fusing task-oriented and open-domain dialogue systems. | |
| 2021 | Recent Advances in Deep Learning Based Dialogue Systems |
| - Recent Advances in Deep Learning Based Dialogue Systems [paper] | |
| - Jinjie Ni, Tom Young, Vlad Pandelea, Fuzhao Xue, Erik Cambria | |
| - An 80-page systematic review for dialogue systems. One of the most cited dialogue system reviews. | |
Activities |
|
Teaching |
|
| 2021 | NTU-SC1003: Introduction to Computational Thinking and Programming |
| Teaching Assistant | |
| NTU-CE2100: Probability and Statistics for Computing | |
| Lecturer | |
| 2020 | NTU-CE1113: Physics for Computing |
| Teaching Assistant | |
| NTU-CZ2007: Introduction To Databases | |
| Teaching Assistant | |
| NTU-CZ2004: Human Computer Interaction | |
| Teaching Assistant | |
Services |
||
| Conference PC Member / Reviewer | ICLR 2025, Neurips 2024, ACL 2024, EMNLP 2024, ACL 2023, EMNLP 2023, AAAI 2023, ICASSP 2023 | |
| Journal Reviewer | Knowledge-Based Systems, Information Fusion, Artificial Intelligence Review, Cognitive Computation | |
| Co-organizer | MLNLP community | |