Enhancing Music Accessibility through AI Systems for and with d/Deaf Individuals

Abstract

Researchers have investigated visual and vibrotactile approaches to making music more accessible to d/Deaf individuals, focusing on music appreciation. However, these approaches often fail to help d/Deaf users fully understand and engage with the various musical elements of a song. My research addresses this gap through a series of design and evaluation studies with d/Deaf and non-d/Deaf participants. It begins with a formative study that identifies key attributes of song signing valued by the Deaf community. Building on this, a controlled study explores the use of disclosure statements to mitigate cultural misrepresentation. Finally, a systems study leverages Large Language Models (LLMs) to support the translation of lyrics to sign language. My next steps involve developing collaborative tools for song signers and facilitating culturally sensitive music experiences. These projects collectively bridge the gap between d/Deaf and non-d/Deaf communities, promoting intercultural understanding and expanding musical inclusivity.