🚀 A curated list of papers, code and projects on Audio Generation. Please join us for more comprehensive summary. If you have any additions to the list, please raise them in the issue section. 欢迎补充👏
If you find this repo useful for your research, please 🌟 and cite:
@software{awesomeaudio2025,
author = {Zixiang Wan},
title = {{Awesome Audio Generation}},
year = {2025},
publisher = {GitHub},
url = {https://github.com/ggiggit/awesome-audio-generation}
}| Date | Paper Title | Links |
|---|---|---|
| 2025-09 | RFM-Editing: Rectified Flow Matching for Text-guided Audio Editing |  |
| 2025-09 | Continuous Audio Language Models | |
| 2025-09 | DreamAudio: Customized Text-to-Audio Generation with Diffusion Models |  |
| 2025-09 | PicoAudio2: Temporal Controllable Text-to-Audio Generation with Natural Language Description |  |
| 2025-08 | AudioStory: Generating Long-Form Narrative Audio with Large Language Models |  |
| 2025-07 | Audio Flamingo 3: Advancing Audio Intelligence with Fully Open Large Audio Language Models |  |
| 2025-07 | DeSTA2.5-Audio: Toward General-Purpose Large Audio Language Model with Self-Generated Cross-Modal Alignment |  |
| 2025-06 | Step-Audio-AQAA: a Fully End-to-End Expressive Large Audio Language Model | |
| 2025-05 | AudioTurbo: Fast Text-to-Audio Generation with Rectified Diffusion | |
| 2025-05 | From Alignment to Advancement: Bootstrapping Audio-Language Alignment with Synthetic Data | |
| 2025-05 | T2A-Feedback: Improving Basic Capabilities of Text-to-Audio Generation via Fine-grained AI Feedback | |
| 2025-05 | Fast Text-to-Audio Generation with Adversarial Post-Training |  |
| 2025-02 | AudioGenX: Explainability on Text-to-Audio Generative Models | |
| 2025-01 | Fugatto 1: Foundational Generative Audio Transformer Opus 1 |  |
| 2024-12 | TangoFlux: Super Fast and Faithful Text to Audio Generation with Flow Matching and Clap-Ranked Preference Optimization |  |
| 2024-12 | Sketch2Sound: Controllable Audio Generation via Time-Varying Signals and Sonic Imitations | |
| 2024-11 | Audiobox TTA-RAG: Improving Zero-Shot and Few-Shot Text-To-Audio with Retrieval-Augmented Generation | |
| 2024-10 | FlashAudio: Rectified Flows for Fast and High-Fidelity Text-to-Audio Generation | |
| 2024-09 | Text2FX: Harnessing CLAP Embeddings for Text-Guided Audio Effects | |
| 2024-09 | PTQ4ADM: Post-Training Quantization for Efficient Text Conditional Audio Diffusion Models | |
| 2024-09 | AudioComposer: Towards Fine-grained Audio Generation with Natural Language Descriptions |  |
| 2024-09 | EzAudio: Enhancing Text-to-Audio Generation with Efficient Diffusion Transformer |  |
| 2024-08 | MorphFader: Enabling Fine-grained Controllable Morphing with Text-to-Audio Models | |
| 2024-07 | Stable Audio Open | |
| 2024-07 | PicoAudio: Enabling Precise Timestamp and Frequency Controllability of Audio Events in Text-to-audio Generation |  |
| 2024-06 | Taming Data and Transformers for Audio Generation |  |
| 2024-06 | Improving Text-To-Audio Models with Synthetic Captions | |
| 2024-06 | UniAudio 1.5: Large Language Model-driven Audio Codec is A Few-shot Audio Task Learner |  |
| 2024-06 | LAFMA: A Latent Flow Matching Model for Text-to-Audio Generation |  |
| 2024-06 | AudioLCM: Text-to-Audio Generation with Latent Consistency Models |  |
| 2024-05 | SoundCTM: Unifying Score-based and Consistency Models for Full-band Text-to-Sound Generation | |
| 2024-04 | Tango 2: Aligning Diffusion-based Text-to-Audio Generations through Direct Preference Optimization |  |
| 2024-02 | Fast Timing-Conditioned Latent Audio Diffusion | |
| 2024-02 | Audio Flamingo: A Novel Audio Language Model with Few-Shot Learning and Dialogue Abilities | |
| 2024-01 | Auffusion: Leveraging the Power of Diffusion and Large Language Models for Text-to-Audio Generation |  |
| 2023-12 | Audiobox: Unified Audio Generation with Natural Language Prompts | |
| 2023-10 | UniAudio: An Audio Foundation Model Toward Universal Audio Generation |  |
| 2023-09 | Retrieval-Augmented Text-to-Audio Generation | |
| 2023-09 | NExT-GPT: Any-to-Any Multimodal LLM |  |
| 2023-08 | Audio Generation with Multiple Conditional Diffusion Model | |
| 2023-08 | AudioLDM 2: Learning Holistic Audio Generation With Self-Supervised Pretraining |  |
| 2023-05 | Make-An-Audio 2: Temporal-Enhanced Text-to-Audio Generation |  |
| 2023-05 | Any-to-Any Generation via Composable Diffusion |  |
| 2023-04 | Text-to-Audio Generation using Instruction-Tuned LLM and Latent Diffusion Model | |
| 2023-04 | AudioGPT: Understanding and Generating Speech, Music, Sound, and Talking Head |  |
| 2023-01 | Make-An-Audio: Text-To-Audio Generation with Prompt-Enhanced Diffusion Models |  |
| 2023-01 | AudioLDM: Text-to-Audio Generation with Latent Diffusion Models |  |
| 2022-10 | Full-band General Audio Synthesis with Score-based Diffusion | |
| 2022-09 | AudioGen: Textually Guided Audio Generation |  |
| 2022-09 | AudioLM: a Language Modeling Approach to Audio Generation |  |
| 2022-07 | Diffsound: Discrete Diffusion Model for Text-to-sound Generation |  |
| 2022-02 | General-purpose, long-context autoregressive modeling with Perceiver AR |  |
| 2021-07 | Conditional Sound Generation Using Neural Discrete Time-Frequency Representation Learning |  |
| 2021-02 | On Generative Spoken Language Modeling from Raw Audio |  |
| 2020-09 | DiffWave: A Versatile Diffusion Model for Audio Synthesis |  |
| 2020-09 | WaveGrad: Estimating Gradients for Waveform Generation |  |
| 2019-10 | MelGAN: Generative Adversarial Networks for Conditional Waveform Synthesis | |
| 2019-05 | Acoustic Scene Generation with Conditional Samplernn |  |
| 2018-02 | Efficient Neural Audio Synthesis | |
| 2016-09 | WaveNet: A Generative Model for Raw Audio |  |
| Date | Paper Title | Links |
|---|---|---|
| 2024-07 | AudioTime: A Temporally-aligned Audio-text Benchmark Dataset |  |
| 2023-09 | Auto-ACD: A Large-scale Dataset for Audio-Language Representation Learning |  |
| 2023-03 | WavCaps: A ChatGPT-Assisted Weakly-Labelled Audio Captioning Dataset for Audio-Language Multimodal Research |  |
| 2022-11 | (LAION-Audio-630K)Large-scale Contrastive Language-Audio Pretraining with Feature Fusion and Keyword-to-Caption Augmentation |  |
| 2022-09 | (WavText5K)Audio Retrieval with WavText5K and CLAP Training |  |
| 2021-12 | (SoundDescs)Audio Retrieval with Natural Language Queries: A Benchmark Study |  |
| 2021-07 | MACS - Multi-Annotator Captioned Soundscapes | |
| 2020-04 | VGGSound: A Large-scale Audio-Visual Dataset |  |
| 2019-10 | AudioCaps: Generating Captions for Audios in The Wild |   |
| 2019-10 | Clotho: An Audio Captioning Dataset |  |
| 2019-05 | (Medley-solos-DB)Joint Time–Frequency Scattering | |
| 2017-03 | Audio Set: An ontology and human-labeled dataset for audio events | |
| 2016-08 | (UrbanSound8K)Deep Convolutional Neural Networks and Data Augmentation for Environmental Sound Classification |  |
| 2015-10 | ESC: Dataset for Environmental Sound Classification |  |
| 2013-10 | Freesound technical demo |  |
| Free To Use Sounds | |
|
| BBC Sound Effect Library | |
|
| BigSoundBank | |
|
| SoundBible | |
|
| Sonniss Game Effects | |
|
| Paramount Motion | |
|
| Audiostock | |
|
| Epidemic Sound | |
| Date | Paper Title | Links |
|---|---|---|
| 2025-09 | SupCLAP: Controlling Optimization Trajectory Drift in Audio-Text Contrastive Learning with Support Vector Regularization | |
| 2022-07 | Masked Autoencoders that Listen |  |
| 2022-06 | CLAP: Learning Audio Concepts From Natural Language Supervision |  |
| 2021-10 | SSAST: Self-Supervised Audio Spectrogram Transformer |  |
| 2020-10 | Contrastive Learning of General-Purpose Audio Representations |  |
| Date | Paper Title | Links |
|---|---|---|
| 2022-02 | HTS-AT: A Hierarchical Token-Semantic Audio Transformer for Sound Classification and Detection |  |
| Date | Paper Title | Links |
|---|---|---|
| 2024-05 | SemantiCodec: An Ultra Low Bitrate Semantic Audio Codec for General Sound |  |
| 2021-07 | HARP-Net: Hyper-Autoencoded Reconstruction Propagation for Scalable Neural Audio Coding | |
| 2021-07 | SoundStream: An End-to-End Neural Audio Codec |  |
| 2019-06 | Cascaded Cross-Module Residual Learning towards Lightweight End-to-End Speech Coding | |
| 2019-06 | Generating Diverse High-Fidelity Images with VQ-VAE-2 |  |
| 2017-11 | Neural Discrete Representation Learning | |
| 2017-04 | Soft-to-Hard Vector Quantization for End-to-End Learning Compressible Representations | |
| Date | Paper Title | Links |
|---|---|---|
| 2024-03 | sDPO: Don't Use Your Data All at Once | |
| 2024-02 | BATON: Aligning Text-to-Audio Model with Human Preference Feedback | |
| 2024-01 | Self-Rewarding Language Models | |
| 2024-01 | Self-Play Fine-Tuning Converts Weak Language Models to Strong Language Models |  |
| 2023-11 | Diffusion Model Alignment Using Direct Preference Optimization | |
| 2023-08 | Reinforced Self-Training (ReST) for Language Modeling | |
| 2023-05 | Direct Preference Optimization: Your Language Model is Secretly a Reward Model | |
| 2022-03 | Training language models to follow instructions with human feedback | |
| Date | Paper Title | Links |
|---|---|---|
| 2022-07 | Classifier-Free Diffusion Guidance | |
| 2022-02 | Progressive Distillation for Fast Sampling of Diffusion Models | |
| 2021-12 | High-Resolution Image Synthesis with Latent Diffusion Models |  |
| 2021-08 | SDEdit: Guided Image Synthesis and Editing with Stochastic Differential Equations | |
| 2021-07 | Structured Denoising Diffusion Models in Discrete State-Spaces | |
| 2021-02 | Argmax Flows and Multinomial Diffusion: Learning Categorical Distributions | |
| 2020-10 | Denoising Diffusion Implicit Models | |
| 2020-06 | Denoising Diffusion Probabilistic Models | |
| 2019-07 | Generative Modeling by Estimating Gradients of the Data Distribution | |
| Date | Paper Title | Links |
|---|---|---|
| 2025-02 | Variational Rectified Flow Matching | |
| 2022-10 | Flow Matching for Generative Modeling | |
| 2022-09 | Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flow | |
| 2020-06 | OT-Flow: Fast and Accurate Continuous Normalizing Flows via Optimal Transport |  |
本项目部分代码参考了以下仓库：