MetaPerch: Learning from metadata for bioacoustics foundation models

ML SSLRO TLDA AI Apps / Applied AI Other Interdisciplinary AI
2026年07月15日
生物声学基础模型依赖于Xeno-Canto等大规模公民科学平台,以获取在地理分布和生态类型上均具有高度多样性的数据。近期研究表明,仅依靠监督学习,在此类大规模数据上训练即可获得当前最优(SOTA)的物种检测模型;然而,这些由社区驱动的数据中心所天然附带的录音元数据,其潜力尚未被充分挖掘。本文探索将元数据(如录音地点与时间)作为辅助监督信号加以利用,使模型能在其学习到的表征中显式建模并利用物种与元数据之间的关联性。元数据相关的辅助损失函数可提供超越声学信号本身的信息,从而引导模型学习更丰富、更鲁棒的表征,显著提升其对物种空间分布变化及声学域偏移的泛化能力——而这恰恰是真实世界被动声学监测(PAM)场景部署所面临的关键挑战。为此,我们提出了新型基础模型MetaPerch,该模型在多个极具挑战性的任务领域中均展现出优异的物种识别性能;同时,我们还开展了详尽的实证研究,系统评估了9类不同来源的元数据在17个生物声学数据集上的影响效果。
Bioacoustic foundation models rely on large-scale citizen science platforms like Xeno-Canto for geographically and ecologically diverse data. Recent work has shown that supervision alone can produce SotA species detection models when trained on this large-scale data -- however, there remains unutilized potential in the form of recording metadata readily available within these community-driven data hubs. In this work, we explore the use of metadata -- such as location and time -- as auxiliary supervision signals, allowing the model to leverage species-metadata correlations in its learned representation. Auxiliary metadata losses provide additional information beyond vocalizations alone that can encourage a richer, more robust representation that generalizes better to species distribution and acoustic domain shifts -- important challenges for deployment in real-world passive acoustic monitoring (PAM) settings. We introduce MetaPerch, a new foundation model that achieves strong species identification performance across multiple challenging domains and present an extensive empirical study of the effects of 9 diverse metadata sources on 17 bioacoustic datasets.
许愿