Neural Audio Lab
How neural networks hear, separate and make sound. For readers who want the research explained, with open models run on the same inputs.
- MFCC, CQT and STFT: Audio feature Extraction Compared
MFCCs, constant-Q transform, and STFT are the main advanced techniques for extracting audio features, each with distinct strengths for speech or music.
- Speaker diarization needs an extra step when two people talk at once
Overlap-aware resegmentation cut the diarization error rate on the AMI meeting corpus by 20 percent relative to a baseline system, but its authors say it is not a complete solution to overlapping speech [2].
- What to check when an audio AI model fails on your recording
Most bad results from audio AI models come from mismatched sample rates, channels, missing decoders, clipping, or data mismatches.
- Automatic music transcription detects notes from onsets
Automatic music transcription starts by detecting note onsets in audio, then estimates each note’s pitch to create a score from the recording.
- PESQ, MOS and SNR: how to use each to evaluate audio models
Use Mean Opinion Score for human-perceived quality, PESQ for objective speech assessment, and SNR for signal clarity—each fits a different need.
- Deep learning revolutionized audio AI in 2012
In 2012, deep learning let audio models learn features directly from raw data, replacing manual feature engineering with end-to-end neural networks.
- A 5-second clip becomes a 201 by 498 spectrogram at 16 kHz
A spectrogram turns audio into a visual map of sound over time and frequency, letting models spot patterns the ear cannot isolate.
- Neural audio codec: how SoundStream turns sound into tokens
Neural audio codecs like SoundStream use convolutional encoders and vector quantizers to compress sound into tokens, enabling low-bitrate, high-quality streaming.
- Neural networks use spectrogram masks to isolate vocals
Neural networks separate a voice from a song by learning to mask regions of a spectrogram that correspond to vocals, leaving the rest as accompaniment.