Jaesung Bae

jb82 [at] illinois [dot] edu

About

I am a PhD candidate in the Computer Science (CS) Department at the University of Illinois Urbana-Champaign, advised by Prof. Minje Kim, and working closely with Prof. Mark Hasegawa-Johnson. Before starting my PhD, I researched mainly text-to-speech (TTS) systems in industry for 5 years, including at Samsung Research, especially focusing on building more expressive and natural TTS systems. My current research interests include speech and audio representation learning, as well as data augmentation methods for low-resource and underrepresented domains in speech and audio, including dysarthric speech.

   Below shows my education and work experience, projects, publications, invited talks, and academic services. Please refer to my CV for further details.

Education & Work Experience

Education

University of Illinois Urbana-Champaign

PhD in Computer Science

Advised by Prof. Minje Kim.

Korea Advanced Institute of Science and Technology (KAIST)

MS in Electrical Engineering

Advised by Prof. Dae-Shik Kim; thesis on speech command recognition using capsule networks.

University of Applied Sciences Upper Austria

Exchange Student

Upper Austria, Austria.

Yonsei University

BS in Electrical and Electronic Engineering

Honors, Fall 2016.

Work Experience

Meta

Research Scientist Intern · Paris, France

Comms & Language Team, FAIR, Meta Superintelligence Labs.
Research Topics: Egocentric and baby speech modeling, data augmentation, meta-learning, and domain adaptation.

Meta

Research Scientist Intern · Menlo Park, CA

Voice Modeling Team, Meta Superintelligence Labs.
Research Topics: Improving controllable generation and understanding in full-duplex speech-language models.

Samsung Research

Full-Time, Speech AI Researcher · Seoul, South Korea

Contributed to Galaxy S24's TTS systems.
Research Topics: Zero-shot, personalized, on-device, and expressive TTS systems.

NCSOFT

Full-Time, Speech AI Researcher · Seongnam, South Korea

Research Topics: Expressive TTS, fine-grained prosody control, and multi-speaker TTS.

News

  1. (Jun 2026) Our paper, "Something from Nothing: Data Augmentation for Robust Severity Level Estimation of Dysarthric Speech," has been accepted to the INTERSPEECH 2026 Long Paper Track!
  2. (May 2026) I started working as a Research Scientist Intern at Meta FAIR Team in Paris, supervised by Jiayi Shen.
  3. (May 2026) I was selected as a Gold Reviewer for ICML 2026, which included a complimentary registration offer.
  4. (Mar 2026) I am participating as a volunteer and student organizer for MSLD (Midwest Speech and Language Days) 2026 at UIUC.
  5. (Dec 2025) I passed my PhD qualifying exam!
  6. (May 2025) I started working as a Research Scientist Intern at MSL (Meta Superintelligence Labs), supervised by Yun Wang.
  7. (Dec 2024) I am organizing the "ICASSP 2025 Generative Data Augmentation Challenge: Zero-Shot Speech Synthesis for Personalized Speech Enhancement." Looking forward to your participation! [link]
  8. (Aug 2024) Starting my PhD at the University of Illinois Urbana-Champaign (UIUC).
  9. (Dec 2023) Two papers have been accepted to ICASSP 2024! (one first author, one second author)

Publications

*: Equal Contribution

2026

Something from Nothing: Data Augmentation for Robust Severity Level Estimation of Dysarthric Speech
Jaesung Bae*, Xiuwen Zheng*, Minje Kim, Chang D Yoo, Mark Hasegawa-Johnson
In Proc. INTERSPEECH (Long Paper Track), 2026.
[paper] [github] [huggingface]

Semantics-Aware Generative Latent Data Augmentation for Learning in Low-Resource Domains
Jaesung Bae, Minje Kim
arXiv preprint arXiv:2602.02841, 2026.
[paper]

2025

Generative Data Augmentation Challenge: Zero-Shot Speech Synthesis for Personalized Speech Enhancement
Jae-Sung Bae, Anastasia Kuznetsova, Dinesh Manocha, John Hershey, Trausti Kristjansson, Minje Kim
In Proc. of the IEEE Int. Conf. on Acoustics, Speech, and Signal Processing Workshops (ICASSPW): Generative Data Augmentation for Real-World Signal Processing Applications (GenDA 2025), 2025.
[paper] [publisher] [code] [website]

2024

Latent Filling: Latent Space Data Augmentation for Zero-shot Speech Synthesis
Jae-Sung Bae, Joun Yeop Lee, Ji-Hyun Lee, Seongkyu Mun, Taehwa Kang, Hoon-Young Cho, Chanwoo Kim
In Proc. IEEE Int. Conf. on Acoustics, Speech and Signal Processing (ICASSP), 2024.
[paper] [publisher] [demo]

MELS-TTS : Multi-Emotion Multi-Lingual Multi-Speaker Text-to-Speech System via Disentangled Style Tokens
Heejin Choi, Jae-Sung Bae, Joun Yeop Lee, Seongkyu Mun, Jihwan Lee, Hoon-Young Cho, Chanwoo Kim
In Proc. IEEE Int. Conf. on Acoustics, Speech and Signal Processing (ICASSP), 2024.
[publisher]

2023

Hierarchical Timbre-Cadence Speaker Encoder for Zero-shot Speech Synthesis
Joun Yeop Lee, Jae-Sung Bae, Seongkyu Mun, Jihwan Lee, Ji-Hyun Lee, Hoon-Young Cho, Chanwoo Kim
In Proc. INTERSPEECH, 2023.
[paper] [demo]

Avocodo: Generative Adversarial Network for Artifact-free Vocoder
Taejun Bak, Junmo Lee, Hanbin Bae, Jinhyeok Yang, Jae-Sung Bae, Young-Sun Joo
In Proc. AAAI, 37(11):12562–12570, 2023. © 2023 AAAI.
[paper] [publisher] [demo] [code]

2022

Hierarchical and Multi-Scale Variational Autoencoder for Diverse and Natural Non-Autoregressive Text-to-Speech
Jae-Sung Bae, Jinhyeok Yang, Tae-Jun Bak, Young-Sun Joo
In Proc. INTERSPEECH, 2022.
[paper] [demo] [video]

Into-TTS : Intonation Template Based Prosody Control System
Jihwan Lee, Joun Yeop Lee, Heejin Choi, Seongkyu Mun, Sangjun Park, Jae-Sung Bae, Chanwoo Kim
arXiv preprint arXiv:2204.01271, 2022.
[paper] [demo]

2021

Hierarchical Context-Aware Transformers for Non-Autoregressive Text to Speech
Jae-Sung Bae, Tae-Jun Bak, Young-Sun Joo, and Hoon-Young Cho
In Proc. INTERSPEECH, 2021.
[paper] [demo]

GANSpeech: Adversarial Training for High-Fidelity Multi-Speaker Speech Synthesis
Jinhyeok Yang*, Jae-Sung Bae*, Taejun Bak, Youngik Kim, and Hoon-Young Cho
In Proc. INTERSPEECH, 2021.
[paper] [demo]

FastPitchFormant: Source-filter based Decomposed Modeling for Speech Synthesis
Taejun Bak, Jae-Sung Bae, Hanbin Bae, Young-Ik Kim, and Hoon-Young Cho
In Proc. INTERSPEECH, 2021.
[paper] [demo]

A Neural Text-to-Speech Model Utilizing Broadcast Data Mixed with Background Music
Hanbin Bae, Jae-Sung Bae, Young-Sun Joo, Young-Ik Kim, and Hoon-Young Cho
In Proc. IEEE Int. Conf. on Acoustics, Speech and Signal Processing (ICASSP), 2021.
[paper] [publisher] [demo]

2020

Speaking Speed Control of End-to-End Speech Synthesis using Sentence-Level Conditioning
Jae-Sung Bae, Hanbin Bae, Young-Sun Joo, Junmo Lee, Gyeong-Hoon Lee, Hoon-Young Cho
In Proc. INTERSPEECH, 2020.
[paper] [demo] [video]

2018

End-to-End Speech Command Recognition with Capsule Network
Jae-Sung Bae, Dae-Shik Kim
In Proc. INTERSPEECH, 2018.
[paper]

Go to top

Projects

Click each project to see demos and further information.

Live Translation

On-device TTS System in various languages for Galaxy S24's Live Translation
Mar 2023 - Jan 2024 (@Samsung Research)

I contributed to the research and development of an on-device TTS system in eight different languages, which is included as a Live Translation feature and introduced as a main AI feature in the Galaxy S24. My contribution involved enhancing the model architecture and achieving a high-quality TTS system that supports various languages with a reduced model size.

Bixby Custom Voice

On-device Personalized TTS System for Bixby Custom Voice Creation
May 2022 - Jan 2024 (@Samsung Research)

I contributed to the research and development of an on-device personalized TTS system, which was integrated into Samsung Galaxy Bixby's Custom Voice Creation and utilized within Bixby Text-call functionality. This system can create a personalized TTS system by fine-tuning the TTS directly on the user's device with just 10 utterances.

Prosody Control

Fine-grained Prosody Control of TTS System (prototype web service)
Mar 2021 - Apr 2022 (@NCSOFT)

I conducted research and developed a TTS system that is capable of controlling the prosody of speech in a fine-grained level. With this system, users were able to modify the speech to have desired prosody. This system is released as an in-company web service and was widely used to make guide videos of NCSOFT's game.

UNIVERSE

TTS System for K-pop Fandom Platform, "UNIVERSE" (live service)
Mar 2019 - Apr 2022 (@NCSOFT)

I contributed to the research and development of a multi-speaker TTS system replicating the voices of numerous K-pop artists, approximately 100 in total, within a single TTS system. This TTS system was used in "UNIVERSE" service, which is a K-pop fan community platform.

TTS System in Baseball Broadcast Scenario
Mar 2019 - Mar 2021 (@NCSOFT)

I researched and developed an expressive TTS system that can generate speech with dynamic expressions suitable for diverse baseball situations. I published several demos on NCSOFT's official blog and news articles. Kindly recommend clicking this project to see the demo videos.

Go to top

Invited Talks

End-to-End Speech Command Recognition with Capsule Network
NAVER Corp., Seongnam, Republic of Korea
Sep 2018

Go to top

Academic Services

  1. Student Organizer for MSLD (Midwest Speech and Language Days) 2026
  2. Challenge Organizer for the ICASSP 2025 Generative Data Augmentation for Real-World Signal Processing Applications (GenDA 2025) Workshop: Zero-Shot Speech Synthesis for Personalized Speech Enhancement [link]
  3. Reviewer: AAAI, NeurIPS, ICASSP 2025, ICML (Gold Reviewer) 2026