ChatGPT's Role in Addressing the Real-World Concerns of Adolescents With Hearing Loss: Insights From Otolaryngologists With Lived Experience.

Résumé

PURPOSE: This study aimed to evaluate ChatGPT-generated responses to psychosocial concerns of hearing-impaired adolescents, assessed by otolaryngologists with lived experiences, exploring artificial intelligence's (AI's) potential as complementary support when professional or peer assistance is unavailable.METHODS: Twenty scenario-based inquiries spanning 5 domains (interpersonal communication, education, emotional well-being, future aspirations, and hearing aid use) were submitted to ChatGPT-4o on August 25, 2024. Responses were evaluated by 6 board-certified otolaryngologists with lived experiences of adolescent hearing loss using 7 criteria: overall quality, empathy, medical consensus alignment, appropriateness, inquiry comprehension, harm, and age-appropriate comprehensibility. Ratings were assigned on a five-point Likert scale; inter-rater reliability was assessed using Fleiss' kappa.RESULTS: The Fleiss' kappa was 0.442. The mean ratings for all criteria exceeded 4.0, except for overall quality. Logistic regression analysis identified high levels of empathy (odds ratio, 7.189; p < .0001) and inquiry comprehension (odds ratio, 4.163; p = .0185) as significant predictors of high-quality responses. The Kruskal-Wallis test revealed significant differences in ratings for the "medical consensus" criterion across domains (p = .0094). Post hoc analysis indicated that responses in the "emotional well-being" domain (median, 4.0) scored significantly lower than those in the "hearing aid use" domain (median, 5.0) for medical consensus (p = .0058).DISCUSSION: Incorporating evaluators with lived experiences provided a reliable framework for assessing AI's potential as a complementary support tool for hearing-impaired adolescents. The findings highlight AI's potential for integrating empathy and improved inquiry comprehension into text-based health-care consultations, particularly in contexts lacking professional or peer support.