Assessing the Diagnostic Capacity of Artificial Intelligence Chatbots for Dysphonia Types: Model Development and Validation Publisher



Saeedi S¹ ; Aghajanzadeh M² Show Affiliations
Source: European Annals of Otorhinolaryngology, Head and Neck Diseases Published:2025

Abstract

Purpose: User-friendly artificial intelligence (AI) chatbots are increasingly being explored to assist healthcare teams in their decision-making processes. As accurate diagnosis in all medical fields is vital in treatment planning, this research seeks to explore the function of two specific AI chatbots, ChatGPT and Perplexity AI, in distinguishing the various types of dysphonia (organic, functional, and neurological). Material and methods: In experiment 1, a script combining voice self-assessments plus the acoustic analysis, and in experiment 2, only the acoustic analysis of 37 dysphonic patients was fed into the ChatGPT and Perplexity AI chatbots specifying the type and asked to develop a complex AI-based model to determine dysphonia type. Then, the same process was redone with data from a sample of 27 other patients as a test. Results: Although ChatGPT could not analyze the data and only provided guidance, the Cohen's Kappa agreement between experts’ diagnoses and Perplexity AI diagnoses in experiment 1 (P = 0.773) and experiment 2 (P = 0.067) lacked statistically significance. Conclusion: Regarding the preliminary poor performance of AI chatbots in differential diagnosis of dysphonia type, it is not currently recommended to use them in clinical settings. However, modifications in AI chatbots in the future might provide more promising results in determining the dysphonia type. Further research is needed to shed light on AI chatbots ability in voice clinics. © 2025 Elsevier Masson SAS