Spectral-based multi-band recurrent neural networks for black-box modeling of dynamic range compressors

Deep learning approaches have been increasingly adopted for virtual analog modeling, which aims to replicate the sonic characteristics of analog audio devices. In the context of analog dynamic range compressor modeling, many existing methods operate directly on raw audio waveforms which are high-dimensional and contain fine-grained temporal features at high sampling rates. These representations are computationally demanding and limit model efficiency. We propose a feature extraction pipeline that leverages the magnitude component of the Short-Time Fourier Transform in combination with a spectral amplification mechanism which acts similarly to a spectral mask but can both attenuate and amplify selected frequency components. We employ multi-band Long Short-Term Memory (LSTM) and Gated Recurrent Unit (GRU) architectures that split the magnitude spectrum into several frequency bands for independent processing, substantially reducing computational complexity while preserving high modeling accuracy. To evaluate our approach, we created two datasets consisting of recordings of the consumer-grade analog compressor Alesis 3630 and its digital counterpart, discoDSP NightShine. We conducted extensive experiments comparing our method against raw waveform baselines using four objective metrics, theoretical and empirical measurements of computational performance, and a subjective listening test. Results indicate that single-band models based on the proposed feature extraction pipeline outperform raw-audio baselines across all evaluation metrics. Multi-band configurations further improve the efficiency balance. In particular, four-band LSTM and GRU architectures achieve higher perceptual fidelity at substantially lower computational cost. Moreover, we conducted a subjective listening test that yielded results aligned with the objective metrics. All source code and pretrained models are provided for reproducibility.

1. Wilmering T., Moffat D., Milo A., Sandler M. A history of audio effects. Applied Sciences, 2020, vol. 10, no. 3, pp. 791. https://doi.org/10.3390/app10030791

2. Montenegro J. Design of an audio compressor with digital control. TECCIENCIA, 2021, vol. 16, no. 30, pp. 51–64. https://doi. org/10.18180/tecciencia.2021.30.4

3. Välimäki V., Reiss J. All about audio equalization: solutions and frontiers. Applied Sciences, 2016, vol. 6, no. 5, pp. 129. https://doi.org/10.3390/app6050129

4. Réveillac J.-M. Musical Sound Effects: Analog and Digital Sound Processing. Wiley-ISTE, 2017, 558 p.

5. Chowdhury J. A comparison of virtual analog modelling techniques for desktop and embedded implementations. arXiv, 2020. arXiv:2009.02833. https://doi.org/10.48550/arXiv.2009.02833

6. Purwins H., Li B., Virtanen T., Schlüter J., Chang S.-Y., Sainath T. Deep learning for audio signal processing. IEEE Journal of Selected Topics in Signal Processing, 2019, vol. 13, no. 2, pp. 206–219. https://doi.org/10.1109/jstsp.2019.2908700

7. Liu X., Sahidullah M., Kinnunen T. A comparative re-assessment of feature extractors for deep speaker embeddings. Proc. of the Annual Conference of the International Speech Communication Association Interspeech, 2020, pp. 3221–3225. https://doi.org/10.21437/interspeech.2020-1765

8. Sun Y., Yang L., Zhu H., Hao J. Funnel deep complex U-Net for phase-aware speech enhancement. Proc. of the Annual Conference of the International Speech Communication Association Interspeech, 2021, pp. 161–165. https://doi.org/10.21437/Interspeech.2021-10

9. Kong J., Kim J., Bae J. HiFi-GAN: Generative adversarial networks for efficient and high fidelity speech synthesis. Proc. of the 34th International Conference on Neural Information Processing Systems, 2020, pp. 17022–17033.

10. Zölzer U. DAFX: Digital Audio Effects. Wiley, 2011, 624 p

11. Kates J.M. Principles of digital dynamic-range compression. Trends in Amplification, 2005, vol. 9, no. 2, pp. 45–76. https://doi.org/10.1177/108471380500900202

12. Giannoulis D., Massberg M., Reiss J.D. Digital dynamic range compressor design—A tutorial and analysis. Journal of the Audio Engineering Society, 2012, vol. 60, no. 6, pp. 399–408.

13. D’Angelo S. Lightweight virtual analog modeling. Proc. of the 22nd Colloquio di Informatica Musicale (CIM), 2018.

14. Eichas F., Zölzer U. Virtual analog modeling of guitar amplifiers with Wiener–Hammerstein models. Proc.of the 44th Annual Convention on Acoustics (DAGA), 2018.

15. Cheng C.M., Peng Z.K., Zhang W.M., Meng G. Volterra-series-based nonlinear system modeling and its engineering applications: A stateof- the-art review. Mechanical Systems and Signal Processing, 2017, vol. 87, part A. pp. 340–364. https://doi.org/10.1016/j.ymssp.2016.10.029

16. van den Oord A., Dieleman S., Zen H., Simonyan K., Vinyals O., Graves A., et al. WaveNet: A generative model for raw audio. arXiv, 2016. arXiv:1609.03499. https://doi.org/10.48550/arXiv.1609.03499

17. Wright A., Damskägg E.-P., Välimäki V. Real-time black-box modelling with recurrent neural networks. Proc. of the 22nd International Conference on Digital Audio Effects (DAFx-19), 2019, pp. 1–9.

18. Ramirez M.A.M., Benetos E., Reiss J.D. Deep learning for black-box modeling of audio effects. Applied Sciences, 2020, vol. 10, no. 2, pp. 638. https://doi.org/10.3390/app10020638

19. Damskägg E.-P., Juvela L., Välimäki V. Real-time modeling of audio distortion circuits with deep learning. Proc. of the 16th Sound and Music Computing Conference, 2019, pp. 332–339.

20. Hawley S.H., Colburn B., Mimilakis S.I. SignalTrain: profiling audio compressors with deep neural networks. arXiv, 2019. arXiv:1905.11928. https://doi.org/10.48550/arXiv.1905.11928

21. Steinmetz C.J., Reiss J.D. Efficient neural networks for real-time analog audio effect modeling. arXiv, 2021. arXiv:2102.06200. https://doi.org/10.48550/arXiv.2102.06200

22. Simionato R., Fasciani S. Fully conditioned and low-latency blackbox modeling of analog compression. Proc. of the International Conference on Digital Audio Effects Dafx, 2023.

23. Yin H., Cheng G., Steinmetz C.J., Yuan R., Stern R.M., Dannenberg R.B. Modeling analog dynamic range compressors using deep learning and state-space models. arXiv, 2024. arXiv:2403.16331. https://doi.org/10.48550/arXiv.2403.16331

24. Simionato R., Fasciani S. Modeling time-variant responses of optical compressors with selective state space models. AES Journal of the Audio Engineering Society, 2025, vol. 73, no. 3. pp. 144–165. https://doi.org/10.17743/jaes.2022.0194

25. Fonseca E., Favory X., Pons J., Font F., Serra X. FSD50K: An open dataset of human-labeled sound events. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 2022, vol. 30, pp. 829–852. https://doi.org/10.1109/taslp.2021.3133208

26. Yamagishi J., Veaux C., MacDonald K. CSTR VCTK corpus: English multi-speaker corpus for CSTR Voice Cloning Toolkit (version 0.92). University of Edinburgh, Centre for Speech Technology Research (CSTR), 2019, https://doi.org/10.7488/ds/2645