The National Advertising Division (NAD) addressed whether advertisers can use artificial intelligence to substantiate product performance claims, ruling that Starkey could not rely on unvalidated AI testing to prove its Omega AI device outperformed competitors in noisy conditions.
The Bottom Line
- Starkey claimed a 6.5 dB advantage in speech intelligibility using AI test measurements, but the NAD found the method lacked real-world validation.
- The advertising watchdog established that AI testing is acceptable only when properly validated to match actual consumer experiences.
- Competitive claims must reflect current market conditions, as four of five rival models tested by Starkey had already been updated.
Challenging AI Testing Methodologies in Hearing Aid Advertising
The regulatory review began when hearing aid maker Sonova challenged competitive performance claims made by rival Starkey regarding its Omega AI device. Starkey marketed the product as better than competing brands in noisy environments, citing a 70% improvement and a 6.5 dB advantage in speech intelligibility. To back these metrics, the company deployed an automated testing framework rather than human test subjects. Starkey mounted various hearing aids on a dummy head, recorded output audio in noisy settings, and utilized Whisper, an AI speech-to-text tool, to measure word recognition counts.
Defending its methodology during the proceedings, Starkey argued that artificial intelligence testing is consistent, objective, and removes the variation that comes with human test subjects. The manufacturer maintained that automated evaluations allow for a larger number of measurements than human subjects would tolerate, generating better predictions of real-world performance. However, the regulatory body scrutinized the connection between digital transcription scores and actual user perception. The official decision noted that while automated testing represents an emerging area with valid operational benefits, any data used for claims asserting real-world consumer benefits must undergo rigorous validation.
The Regulatory Threshold for Predictive Claims
The core dispute centered on whether automated speech recognition tools accurately mirror human auditory experiences. The evaluation revealed that nothing in the evidentiary record demonstrated that Whisper’s transcription outputs correlated directly with what actual hearing aid wearers encounter. Starkey’s own internal documentation acknowledged that further research remains necessary to establish the validity of AI-based acoustic measurements. Although the company attempted to mitigate the issue by incorporating the word “predictive” into its marketing copy, the adjustment proved insufficient to bridge the empirical gap.
Beyond the validation hurdle, the proceeding highlighted strict standards regarding the temporal relevance of comparative data. By the time of the challenge, four out of the five competing device models evaluated in Starkey’s testing protocols had already been updated. Consequently, the watchdog recommended the discontinuation of the contested claims while clarifying that manufacturers may still leverage artificial intelligence to substantiate carefully tailored claims.
| Testing Parameter | Starkey Approach | NAD Finding |
|---|---|---|
| Testing Subject | Dummy head with automated recording | Lacks real-world consumer validation |
| Measurement Tool | Whisper AI speech-to-text tool | Unproven correlation to human hearing |
| Competitive Benchmarks | Prior generation rival devices | Outdated; requires current market data |
The guidance establishes a clear operational boundary for firms integrating automated evaluation into their product development cycles. Advertisers must now ensure that digital benchmarking models reflect how real consumers will experience the product before deploying them in promotional campaigns. Furthermore, comparative product testing must reflect the current market.
Disclaimer: The information provided in this article is for educational and informational purposes only and does not constitute financial advice.
