Health acoustics, are a range of sounds including obvious ones such as coughs and breathing, but also includes sounds from human digestive tracts and bowels, that can hold valuable health information. This is the reason doctors listen to one’s heart and lungs and ask patients to cough when examining them. It helps identify certain issues and points the doctor towards a diagnosis.
This capability can now be used in medical machine learning processes.
Existing deep learning models for these acoustics are often quite task-specific, limiting their general useability. Researchers from Google Research and the Center of Infectious Disease Research in Zambia have developed HeAR, a scalable deep-learning system based on SSL.
HeAR utilizes masked auto-encoders trained on an extensive dataset of over 300 million two-second audio clips. The model has established itself as the state-of-the-art standard for health audio embedding, and can be used across 33 health acoustic tasks from 6 different datasets.
According to a Google Blog, HeAR is now available to researchers to help accelerate development of custom bioacoustic models with less data, setup and computation requirements. Their stated goal is to enable further research into models for specific conditions and populations, even if data is sparse or if cost or where compute barriers exist.
HeAR comprises of three distinct components: data curation (including a health acoustic event detector), general-purpose training used for developing an audio encoder, and task-specific evaluation using the trained embedding. The system encodes two-second audio clips to generate embeddings for downstream tasks used in patient diagnostics.
The health acoustic event detector, is able to identify six non-speech health events like coughing and breathing. HeAR is trained on a large dataset and benchmarked across various health acoustic tasks, demonstrating superior performance compared to state-of-the-art audio encoders like TRILL, FRILL, and CLAP.
This new model is part of a strategic effort in leveraging the power of Ai in the healthcare sector and could have a big impact in the rapid or early identification of illness or conditions, enabling early diagnosis and treatment possibilities
