REFINED: improving hearing aids using AI

There are currently nearly 466 million people worldwide suffering from some sort of hearing loss. 34 million of them live in the European Union, with 6 million of those people living in France. These are alarming statistics, and the situation is expected to get worse in the coming years as the world’s population gets older. The WHO expects the number of people with hearing loss to reach 900 million by 2050.

Portable hearing aids have been used to assist with hearing impairments for the best part of a century. In recent years a great deal of research has gone into miniaturising these devices and improving their performance.

However, 50% of people with hearing difficulties and who need hearing devices do not use them, mainly because of how ineffective they are in complex or noisy environments.

“Most traditional hearing aids work through sound enhancement and spatial filtering. They separate what a person is saying in order to make it as intelligible and as pleasant to listen to as possible, based on criteria set by people with normal hearing. This works for people suffering from a simple loss of auditory sensitivity, which can be caused by overexposure to noise or ageing. But it’s less effective for people suffering from other types of hearing loss, such as auditory neuropathies”, explains Romain Serizel.

Unlike other more common types of hearing loss, which affect the hearing threshold, auditory neuropathies do not necessarily have an impact on auditory sensitivity, but instead affect temporal information processing. This is vital when it comes to distinguishing between different noise sources in noisy environments, with multiple speakers distributed spatially and competing with one another.

What is most important for patients suffering from auditory neuropathy, for whom current hearing aids have little or no effect, is not restoring audibility, but rather improving their speech perception, particularly in noisy environments. This involves compensating for a deterioration in acoustic cues, which rely on temporal precision.

It was with this in mind that the Hearing Institute (at the Pasteur Institute), LORIA (through the Multispeech project team) and the CEA List decided to work together to improve the capacities of hearing aids for people suffering from this type of neuropathy.

This ANR project, which was given the name REFINED, was launched back in April and has four objectives.

• To identify perception deficiencies in people diagnosed with auditory neuropathy and what their needs are in terms of auditory and extra-auditory cues, the goal being to improve their understanding of speech.

“This may be types of phonemes which are difficult to pick up on, or types of noises which, when coupled with speech, create difficulties. We will need to test multiple scenarios in order to see which ones have an impact on speech understanding. This will allow us to identify aspects which we as IT specialists will be able to process using our algorithms”, explains Romain Serizel.

• To develop new AI algorithms for source separation and noise reduction which target these cues, going beyond simply restoring speech.

“We’re moving away from models based on people with normal hearing, adapting instead to people with hearing difficulties. Our aim is to identify critical aspects and to convert these into mathematical criteria for adjusting our speech enhancement algorithms.”

• To find an effective way of incorporating these algorithms despite the limited resources of portable devices.

“Hearing aids are tiny devices with very little battery or processing power. Unlike the speech enhancement used on phones for hands-free calls, hearing aids have to work continuously for as long as they are being worn. As a result, there are significant constraints to deal with in terms of energy consumption and processing complexity. Our aim is to develop algorithms which function in a relatively simple way and which don't use up too much energy, without this having a negative impact on performance with regard to auditory neuropathy.”

• To validate the effectiveness of these algorithms in factoring in the handicaps specific to each user.

“We eventually want to test one of our algorithms on prototype hearing aids worn by volunteers in real-life situations.”

Although limited to the selected group, the results of the project will prove that it is possible to go beyond current limits, while factoring in embedding constraints and focusing on practicality and adaptability in real-life situations.

REFINED is the first project aimed at developing an adaptive, AI-based, end-to-end solution for patients suffering from auditory neuropathy, the end goal being the embedding of algorithms. A huge step forward for a hearing impairment that is still relatively unknown in France.