Human-in-the-loop approach for AI-based speech enhancement assessments

In the age of remote collaboration and videoconferencing, AI plays an important role in speech processing to remove background noise, enhance speech, and gain insights into speech and audio streams. But how do you assess the efficacy of such technology in the context of decades of legacy hardware and software solutions? At BabbleLabs, in addition to objective metrics, we use a listener perception-based framework to evaluate the efficacy of our AI-based speech enhancement models and products.

Subjective assessments with listener perceptions on sound quality

The gold standard for assessing speech and sound quality utilizes the subjective opinions of a large, diverse panel of human listeners. Traditionally, this process is laborious and expensive to conduct. There are established objective measures using predictive models. However, the predictions may be valid only for very specific types of distortions, and useful, at the earlier stages of audio algorithm development.

At BabbleLabs, we use objective measures (such as: PESQ, ESTOI, SNR) in the early stages of the algorithm development, since they are fast and inexpensive to apply. In the intermediate and, especially at the final stages, we place greater reliance on subjective, human listener opinions in order to:

Leverage the gold standard in sound quality assessment Get feedback from real listeners in real-life listening environments Gather reliable opinions for many types of audio distortions

To this end, we have developed a simple, comparative, crowdsourced, subjective testing framework that is performed at scale with a large number of individuals in “our ...

Continue Reading

Creating the Connection and Regaining Productivity in a Virtual Work World

If you’re a business professional, you’ve spent the last few months working from home due to shelter-in-place orders. Distributed workforce models aren’t new, but measures implemented during the pandemic have shown organizations that they can succeed, at-scale, swiftly. Many will make a permanent shift that will allow employees to work from home a lot more often, if not exclusively. You, personally, may have managed OK through a temporary period, but what does life look like when your company transitions to a new model and the majority of your communications are through a computer or mobile device?

Well before COVID-19, I managed dispersed teams – first, while working from home with a field sales team during a 12-month period while we looked for new office space and another time during a long tenure with a Fortune 1000 corporation that moved through various degrees of remote work. In these roles I managed home-based local talent, national teams and international teams who worked remotely and in corporate offices. Here are a few things I learned that allowed us to have more meaningful and effective remote communications and maximize productivity.

Create the Connection

Even with the right collaboration tools, choosing which to use can make or break your effectiveness, and sometimes sanity. With the availability of high-quality, easy-to-use video conferencing software, most teams today default to video as their primary collaboration tool for live conversations. The belief is that you can interpret and show emotions and understanding through body ...

Continue Reading