CHATGPT demonstrates higher accuracy and completeness than Google Web Search when providing information about biologic therapy for systemic lupus erythematosus (SLE), according to a new comparative study.
SLE is a complex autoimmune disease that requires careful patient education to support acceptance of emerging biologic therapies. With the increasing reliance on digital sources for health information, researchers evaluated whether artificial intelligence tools such as ChatGPT could enhance access to accurate, comprehensive content compared with traditional web searches.
The study analyzed 20 common questions about biologic therapy for SLE, with responses generated by ChatGPT-4o and retrieved from Google Web Search. Questions were categorized as fact, policy, or value, and responses were assessed for accuracy, completeness, and readability.
ChatGPT produced responses with a mean accuracy score of 5.83, significantly higher than Google’s mean score of 4.75. Completeness also favored ChatGPT, with a mean score of 2.88 compared with 1.68 for Google. The sources used by ChatGPT were predominantly academic (29 of 48), while Google’s 20 sources were distributed more evenly across academic, commercial, governmental, medical practice, and social media platforms.
Readability analysis revealed both ChatGPT and Google responses were written at a graduate-level difficulty, potentially limiting accessibility for patients without advanced education. However, when prompted, ChatGPT was able to generate answers at a sixth-grade level, highlighting its adaptability for patient-facing communication.
The authors concluded that ChatGPT’s responses were accurate, rigorous, and well-supported by professional sources, while also demonstrating a degree of humanistic care. Despite these advantages, they noted that broader research is needed to evaluate the tool’s performance across other diseases, question types, and languages.
This study underscores the potential role of AI-driven platforms in supporting healthcare professionals and patients with reliable and comprehensive information on complex treatments such as biologic therapy for SLE.
Reference:
Li K et al. Evaluating ChatGPT’s Utility in Biologic Therapy for Systemic Lupus Erythematosus: Comparative Study of ChatGPT and Google Web Search. JMIR Form Res. 2025;9:e76458.
