Smart­Na­vi

Prac­tical help for peop­le with im­pai­red hea­ring

SmartNavi project: Smart orientation aid to ensure and increase social integration skills

How can you intelligently master everyday life with declining hearing? This question arises, especially against the backdrop of an ageing society in Germany. As a preliminary scientific project in the focus area “Human-technology interaction for demographic change”, the research team led by Dr. Markus Borschbach dealt with a “smart orientation aid to maintain and increase social integration skills”. A human-machine interface should maintain natural autonomy in everyday life. The aim of the project was to develop a system for analyzing acoustic environments with the help of a smartphone and filtering it for transmission to a hearing aid. Hearing loss is accompanied by inner ear hearing loss and a reduction in processing strategies in the higher centers of the auditory pathway (synapse loss). This leads to a variety of dangers and problems in everyday life, associated with orientation and communication difficulties, which significantly restrict participation in the social environment. An omnipresent and intuitive human-machine interface can adaptively recreate the scenarios of typical everyday communication in a natural form.

The acoustic environment is analyzed with the help of a smartphone using signal source separation. It allows the separation of everyday acoustic situations into individual (speech) signals as a synthetic reproduction for selective hearing. The first milestone was an interactive app for simulating a smartphone-based hearing aid for recording acoustic scenarios with subsequent signal source separation into the individual (speech) sources. The separated signal sources are displayed via the user interface as playable audio files for concentration on the user-preferred signal source.

In the course of the project, extensive empirical investigations were carried out into the software and hardware requirements of various internal and external smartphone microphones for good separation performance. The basic results and findings from the experiments regarding technical details, positioning, distance, orientation and software requirements of these internal and external microphones were presented and discussed at international conferences. In addition, investigations were carried out with regard to the separation quality and runtime of various challenging acoustic scenarios with reverberation and room echo.

The best practices and techniques were then used to investigate the real-time capability of the separation. The real-time capability was carried out with regard to mobile platforms and PC platforms. The results of the runtime analysis on different platforms were presented to an international audience of experts. The findings of these investigations were decisive for the decision regarding the implementation of an online or offline process.