Artificial intelligence chatbots and idiopathic pulmonary fibrosis: are they ready to inform patients?

Abstract

OBJECTIVE: The aim of this study was to determine the quality, reliability, and readability of the responses provided by artificial intelligence chatbots about idiopathic pulmonary fibrosis. METHODS: The clinically relevant questions about idiopathic pulmonary fibrosis diagnosis, treatment, prognosis, and lifestyle management were submitted to four widely used artificial intelligence chatbots, including ChatGPT, Perplexity, Gemini, and Copilot. Responses were assessed by five clinicians based on readability, understandability, quality, and content reliability using standardized tools. RESULTS: The overall readability of chatbot-generated responses was low, corresponding to high educational requirements. Understandability ranged from moderate to good, whereas actionability remained moderate. Gemini produced the most readable and understandable outputs. Journal of the American Medical Association and Likert scores indicated limited source transparency but good guideline concordance. DISCERN analysis for the treatment question showed significant variation (p=0.003), with Perplexity achieving the highest total score. CONCLUSION: Although artificial intelligence chatbots offer rapid and accessible information, their readability and source reliability remain limited. These findings highlight the necessity of expert supervision and further model improvement before artificial intelligence chatbots can be safely integrated into patient education.