Interpreting CNN Models for Musical Instrument Recognition Using Multi-Spectrogram Heatmap Analysis: A Preliminary Study
Date
Authors
Supervisor
Item type
Journal Article
Degree name
Journal Title
Journal ISSN
Volume Title
Publisher
Frontiers Media SA
Abstract
Introduction: Musical instrument recognition is a critical component of music information retrieval (MIR), aimed at identifying and classifying instruments from audio recordings. This task poses significant challenges due to the complexity and variability of musical signals. Methods: In this study, we employed convolutional neural networks (CNNs) to analyze the contributions of various spectrogram representations—STFT, Log-Mel, MFCC, Chroma, Spectral Contrast, and Tonnetz—to the classification of ten different musical instruments. The NSynth database was used for training and evaluation. Visual heatmap analysis and statistical metrics, including Difference Mean, KL Divergence, JS Divergence, and Earth Mover’s Distance, were utilized to assess feature importance and model interpretability. Results: Our findings highlight the strengths and limitations of each spectrogram type in capturing distinctive features of different instruments. MFCC and Log-Mel spectrograms demonstrated superior performance across most instruments, while others provided insights into specific characteristics. Discussion: This analysis provides some insights into optimizing spectrogram-based approaches for musical instrument recognition, offering guidance for future model development and improving interpretability through statistical and visual analyses.Description
Keywords
convolutional neural networks, feature extraction, feature maps, heatmaps, music information retrieval, musical instrument recognition, pattern recognition, spectrogram analysis, 46 Information and Computing Sciences, 4602 Artificial Intelligence, 4611 Machine Learning, Bioengineering, 4007 Control engineering, mechatronics and robotics, 4602 Artificial intelligence, 4611 Machine learning
Source
Frontiers in Artificial Intelligence, ISSN: 2624-8212 (Print); 2624-8212 (Online), Frontiers Media SA, 7, 1499913-. doi: 10.3389/frai.2024.1499913
Rights statement
© 2024 Chen, Ghobakhlou and Narayanan. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.