Vivek Iyer

I am a final-year PhD student in NLP at the University of Edinburgh and an Apple AI/ML PhD Fellow, supervised by Dr. Alexandra Birch.

My research focuses on multilinguality, multimodality and personalization, motivated by a desire to adapt language models for the long tail of users, languages, modalities and cultures. My work has been published at EMNLP, NAACL, EACL and Interspeech. On the industry side, I have done research internships post-training LLMs at Meta and Apple, and earlier at Naver Labs Europe and IBM Research. For more, see my career summary or publications.

I’m currently on the industry job market for Research Scientist roles! Do reach out if you are hiring or think there is a fit.

news

Mar 10 2026	My work on Spectrum, an omnilingual, cross-modal LLM reasoning over 1500 languages in a shared, modality-agnostic latent space, is now published in the Omnilingual SONAR technical report 🌍
Nov 1 2025	Our paper on XL-Suite, an evaluation benchmark and synthetic data generation method for cross-lingual open-ended generation, has been accepted to Findings of EMNLP 2025. 🎉
Sep 1 2025	Started a research internship at Meta (FAIR) in Paris, working on Spectrum, an omnilingual speech-text language model 🇫🇷
Mar 3 2025	Started a research internship at Apple in Pittsburgh, on personalization of reward models 🇺🇸
Dec 5 2024	Selected as an Apple Scholar in AI/ML — one of 21 recipients selected across the globe! 🎓
Sep 20 2024	[EMNLP 2024] 2 long papers on low-resource LLM-MT (Spotlight) and cultural transcreation of menus accepted at WMT 2024! See you all in Miami
Jul 11 2024	Gave an invited talk at IBM Research (slides) as part of the “Papers We Wrote” program which features presentations from their top-performing interns.
Jun 21 2024	[NAACL 2024] Presented a poster on our submission on adapting LLMs for very low-resource MT, ranked #3 at AmericasNLP shared task, at NAACL 2024 in Mexico City.

noteworthy publications

arXiv

Omnilingual SONAR: Cross-Lingual and Cross-Modal Sentence Embeddings Bridging Massively Multilingual Text and Speech

Omnilingual SONAR Team, João Maria Janeiro, Pere-Lluís Huguet Cabot, Ioannis Tsiamas, Yen Meng, Vivek Iyer, Guillem Ramírez, Loic Barrault, Belen Alastruey, Xiang "Tony" Cao, Yu-An Chung, Marta R. Costa-Jussa, David Dale, Kevin Heffernan, Jaehyeong Jo, Artyom Kozhevnikov, Alexandre Mourachko, Christophe Ropers, Holger Schwenk, and Paul-Ambroise Duquenne

2026

TLDR arXiv PDF

Omnilingual SONAR is a massively multilingual speech-text sentence embedding model spanning 1500+ languages. I led post-training efforts for Spectrum, an omnilingual, cross-modal encoder-decoder language model that reasons over OmniSONAR’s language and modality-agnostic space to provide seamless zero-shot cross-lingual and cross-modal performance — while also significantly enhancing long-context efficiency.
EMNLP

XL-Suite: Cross-Lingual Synthetic Training and Evaluation Data for Open-Ended Generation

Vivek Iyer, Pinzhen Chen, Ricardo Rei, and Alexandra Birch

In Findings of the Association for Computational Linguistics: EMNLP 2025 Nov 2025

TLDR Abs PDF

Cross-lingual open-ended generation – responding in a language different from that of the query – is an important yet understudied problem. This work proposes XL-Instruct, a novel technique for generating high-quality synthetic data, and introduces XL-AlpacaEval, a new benchmark for evaluating cross-lingual generation capabilities of large language models (LLMs). Our experiments show that fine-tuning with just 8K instructions generated using XL-Instruct significantly improves model performance, increasing the win rate against GPT-4o-mini from 7.4% to 21.5% and improving on several fine-grained quality metrics. Moreover, base LLMs fine-tuned on XL-Instruct exhibit strong zero-shot improvements to same-language question answering, as shown on our machine-translated m-AlpacaEval. These consistent gains highlight the promising role of XL-Instruct in the post-training of multilingual LLMs. Finally, we publicly release XL-Suite, a collection of training and evaluation data to facilitate research in cross-lingual open-ended generation.

We introduce XL-Instruct, a technique for generating high-quality synthetic data for cross-lingual open-ended generation, and XL-AlpacaEval, a new benchmark for evaluating it. Fine-tuning on just 8K XL-Instruct examples boosts win-rate against GPT-4o-mini from 7.4% to 21.5% and improves same-language QA, with all data and code released as XL-Suite.
WMT (Spotlight)

Quality or Quantity? On Data Scale and Diversity in Adapting Large Language Models for Low-Resource Translation

Vivek Iyer, Bhavitvya Malik, Pavel Stepachev, Pinzhen Chen, Barry Haddow, and Alexandra Birch

In Proceedings of the Ninth Conference on Machine Translation Nov 2024

TLDR Abs PDF

Despite the recent popularity of Large Language Models (LLMs) in Machine Translation (MT), their performance in low-resource languages (LRLs) still lags significantly behind Neural Machine Translation (NMT) models. In this work, we explore what it would take to adapt LLMs for the low-resource setting. Particularly, we re-examine the role of two factors: a) the importance and application of parallel data, and b) diversity in Supervised Fine-Tuning (SFT). Recently, parallel data has seen reduced use in adapting LLMs for MT, while data diversity has been embraced to promote transfer across languages and tasks. However, for low-resource LLM-MT, we show that the opposite is true for both considerations: a) \textitparallel data is critical during both pre-training and SFT; b) diversity tends to cause \textitinterference instead of transfer. Our experiments with three LLMs across two low-resourced language groups—Indigenous American and North-East Indian—reveal consistent trends, underscoring the generalizability of our findings. We believe these insights will be valuable for scaling to massively multilingual LLM-MT models that can effectively serve LRLs.

We re-examine how parallel data and fine-tuning diversity affect LLM adaptation for low-resource machine translation, finding — contrary to recent trends — that parallel data remains critical while diversity tends to cause interference rather than transfer. Experiments across three LLMs and two low-resource language groups (Indigenous American and North-East Indian) confirm these trends generalize.
Interspeech

mHuBERT-147: A Compact Multilingual HuBERT Model

Marcely Zanon Boito, Vivek Iyer, Nikolaos Lagos, Laurent Besacier, and Ioan Calapodescu

In Proceedings of INTERSPEECH 2024 Sep 2024

TLDR Abs PDF

We present mHuBERT-147, the first general-purpose massively multilingual HuBERT speech representation model trained on 90K hours of clean, open-license data. To scale up the multi-iteration HuBERT approach, we use faiss-based clustering, achieving 5.2x faster label assignment than the original method. We also apply a new multilingual batching up-sampling strategy, leveraging both language and dataset diversity. After 3 training iterations, our compact 95M parameter mHuBERT-147 outperforms larger models trained on substantially more data. We rank second and first on the ML-SUPERB 10min and 1h leaderboards, with SOTA scores for 3 tasks. Across ASR/LID tasks, our model consistently surpasses XLS-R (300M params; 436K hours) and demonstrates strong competitiveness against the much larger MMS (1B params; 491K hours). Our findings indicate that mHuBERT-147 is a promising model for multilingual speech tasks, offering an unprecedented balance between high performance and parameter efficiency.

mHuBERT-147 is a compact 95M parameter multilingual HuBERT model trained on 90K hours of clean, open-license data. It outperforms larger models trained on substantially more data, ranking second and first on the ML-SUPERB 10min and 1h leaderboards, with SOTA scores for 3 tasks. Across ASR/LID tasks, our model consistently surpasses XLS-R (300M params; 436K hours) and demonstrates strong competitiveness against the much larger MMS (1B params; 491K hours).