UW study finds AI accent bias in speech evaluations

By Tom Whitfield

A University of Washington study of five accents found that leading AI models generally scored the same English script highest for American and British speakers and lowest for Chinese and Nigerian speakers. The research covered job interviews, conference presentations and English proficiency tests, exposing a risk for employers using AI to assess recorded interviews or oral exams.

The finding does not prove that any named hiring product or English test discriminates, because the study examined general-purpose models. It does give HR teams a concrete validation issue: whether accent changes a model's assessment when the words remain constant.

What did the study test?

Researchers tested models from Google, OpenAI and Alibaba using speech from five accent groups: American, British, Chinese, Indian and Nigerian. Across the three settings, American and British generally ranked highest, Indian sat in the middle, and Chinese and Nigerian ranked lowest.

The penalty concentrated on delivery, the study found. Content scores for Chinese and Nigerian speakers sometimes fell as well, even though every person read the same script and the models had been instructed to judge only substance.

Orevaoghene Ahia is a UW doctoral student in the Paul G. Allen School of Computer Science & Engineering and co-lead author. Ahia said: “Even though they repeatedly claimed Chinese and Nigerian speakers ‘mumbled’ or were ‘difficult to understand,’ the models were actually very good at transcribing the accented speech.”

The controlled wording shows that the models did not score answers on substance alone.

Why does this matter to employers?

Companies increasingly use AI to screen recorded job interviews, while instructors are turning to oral exams as written work becomes easier to fake with AI. Those uses put accent-sensitive scoring near hiring and education decisions, although the study identified no legal violation or compliance deadline.

Pearson’s PTE Academic, an English test used for visas and immigration, uses AI to score some spoken answers. The paper cites figures showing that the test was given about 1.2 million times in 2023.

For employers, the immediate question is where model output sits in the process: as one input to human review, a filter, or a decisive score. Each product needs accent-specific validation because general-model results cannot establish how a deployed system behaves.

Automated assessment cannot be treated as neutral merely because it transcribes the words accurately.

What did the research not establish?

The researchers did not test the proprietary products used by hiring platforms or test makers. Such systems usually do not disclose how they work, and the paper notes that the authors knew of no public audits of AI-scored English tests.

They chose state-of-the-art general-purpose models because these are systems companies could deploy or build on. That tests a plausible building block, not a complete hiring or testing system.

The study was funded in part by gifts from Google and OpenAI, whose models were among those tested. That disclosure is material context for readers weighing the evidence.

A model-level finding is a warning signal, not a verdict on any vendor's deployed product.

Why did models favor some accents?

To probe the pattern, researchers measured how acoustically different each accent was from American English. Chinese and Nigerian accents were farthest from that baseline, and model scores fell as the acoustic distance grew. Indian and British accents sat close to American speech.

The authors offer three possible training explanations. None of the three was isolated as a proven cause.

• Training data that describes accented speakers as unclear.
• Datasets that underrepresent some accents.
• Training that reinforces ideas about how a professional should sound.

English is an official language in Nigeria and the language of instruction in Indian schools, yet both accents scored below American and British ones. The result challenges any assumption that formal use of English guarantees favorable treatment.

The association is measured; the proposed causes remain hypotheses.

What should employers ask before deployment?

These questions concern deployment, not vendor claims.

• Which model does the tool use, and does the vendor publish evaluation results by accent, dialect and subpopulation rather than only an overall accuracy rate?
• Does the employer's own validation use the same script across accent groups, allowing a direct check on whether delivery changes scores?
• Would the tool merely inform recruiter judgment, or would its score determine who advances?

Yulia Tsvetkov, a UW associate professor and the study's senior author, set out the reporting standard. “For all of these, we hope that developers build and publish evaluations of their models that don’t focus only on average performance, but also break it down into subpopulations, dialects and so on,” she said.

“We don’t want the average performance to be 97% accurate for the general population, but for a smaller group only 20%.” An aggregate score cannot answer those subgroup questions.

How does this fit UW's wider work?

This study extends several earlier UW projects on model bias.

• Resume-screening models favored résumés with white- and male-associated names.
• A related project found that politically slanted chatbots could shift people's views, while education could offer protection.
• AI-written children's stories nearly erased female animal characters.

Those projects do not establish the same mechanism as the accent study. They do show why subgroup testing is a recurring issue.

What happens next?

The team presented the research Oct. 25 at the Conference on Empirical Methods in Natural Language Processing in Budapest. Researchers next plan to examine more accents, including regional variations within countries. They also plan to study how models handle audio in medical settings such as automated emergency response.

Those tests will matter because an average result says little about a model's weakest subgroup. Until product-level audits become available, employers and test providers have evidence of a risk, but no measured error rate for their own systems.

The next research phase will test whether accent bias persists across regional varieties and high-stakes medical audio. Deployment decisions will remain difficult until developers publish the subgroup results researchers are asking for.