Bridging Linguistics and Artificial Intelligence: A Phoneme-Centric Method for Assessing Synthetic Speech.

This study bridges the gap between traditional linguistic studies, particularly phonology, and modern Machine Learning (ML) developments. It explores the mutually beneficial interactions between the two domains: by utilizing Speech-To-Text (STT) models to aid in phonology and by integrating phonological insights into the evaluation of Text-To-Speech (TTS) generation. Phonology, the study of phonemes — fundamental sound units in human speech — is pivotal for understanding speech variations caused by accents, speech disorders, and individual pronunciation differences. Leveraging the International Phonetic Alphabet (IPA), this work analyzes speech patterns to enhance explainability and accuracy in TTS models, particularly those employing transformer architectures. This work aims to refine ML tools, expand the phonological approach, and further integrate linguistic knowledge into TTS evaluation and improvement. This approach for phoneme analysis is tested on the Speech Accent Archive and synthetic speech samples generated by the OpenAI and Azure text-to-speech systems. By employing the wav2vec model for phoneme detection and DeepPhonemizer for grapheme-to-phoneme conversion, phoneme distances are calculated using an adapted Levenshtein distance. This analysis reveals significant differences in phoneme accuracy between native English speakers, nonnative speakers, and synthetic speech, highlighting areas for improvement in synthetic speech generation. Results indicate that while synthetic speech often rivals or slightly exceeds real speech in overall phoneme distance, it also exhibits higher error variability, thus pinpointing specific areas for targeted improvement. This approach not only aids in refining TTS systems but also offers insights into developing more unique and personalized synthetic voices. [ABSTRACT FROM AUTHOR]