CuCap: Comparative Analysis of Customized Captioning between North American and South Korean d/Deaf and Hard-of-Hearing Users

Abstract

Affective and prosodic captions convey not only what a speaker says, but also how they say it—louder words may appear thicker, quieter ones thinner; angry in red, calm in blue. These captions can improve access, satisfaction, and engagement for d/Deaf and Hard-of-Hearing (DHH) users. While prior work has explored their design space, it has focused largely on DHH participants in North America, limiting generalizability beyond English and Latin-based scripts. To uncover the role of culture and language, we ran an exploratory study with 49 DHH participants from North America and South Korea using CuCap, a tool that allowed them to personalize which speech features were displayed, and how. While emotion visualization was a universally favored choice, confirming prior findings, prosody preferences varied across cultures, reflecting linguistic and hearing factors. These findings point to the need for flexible captioning systems that account for cultural, linguistic, and individual differences.