Improved analysis of deep bioacoustic embeddings through dimensionality reduction and interactive visualisation

Deep neural networks (DNN) are a popular tool to process environmental sounds and identify sound-producing animals, but it can be difficult to understand the decision-making logic, particularly when it does not produce the expected results. Here we describe a new and enhanced visual interactive analysis of embeddings and explore its application in bioacoustics. Embeddings are the output of the penultimate layer of a DNN, an N-dimensional vector that, only one step removed from the final output, represent the inner-workings of a DNN model. Using existing dimensionality reduction techniques we converted the N-dimensional embeddings into 2 or 3-dimensional arrays displayed in scatterplots. By incorporating sound samples into the scatterplots we developed a visual and aural interactive interface and demonstrate its utility in assessing the performance of trained bioacoustic models, facilitating post-processing of results, error detection, input selection and the detection of rare events, which the reader can experience in online examples with publicly available code.