Assessing ChatGPT vs. evidence-based online responses for polycystic ovary syndrome self-management and education: an international cross-sectional blinded survey of healthcare professionals.

Artificial intelligence (AI)-powered large language models, such as ChatGPT, are increasingly used by the public for health information. The reliability of such novel AI-tools in providing credible polycystic ovary syndrome (PCOS) information/advice requires investigation. Healthcare professionals involved in PCOS care (n = 43 from 14 countries) used a 5-point Likert scale to evaluate ChatGPT-generated responses to frequently asked questions about PCOS against the corresponding patient-orientated, evidence-based recommendations/responses available online. ChatGPT responses were rated significantly higher than the evidence-based responses for 11 of the 12 study questions, with moderate to large effect sizes (rrb = −0.46 to −1.00; all p-values <0.05), with ChatGPT answers being rated on average 0.824 units higher. Scoring agreement varied (poor to fair), with seven questions showing statistically fair agreement (κ = 0.24–0.37, p < 0.05). Readability analyses found no statistically significant differences between ChatGPT and evidence-based responses. However, using ChatGPT for simplifying the responses resulted in significant improvement. ChatGPT holds potential as a complementary patient self-education tool in PCOS, capable of interactive engagement and simplifying medical language. Further research is needed to identity optimal integration of AI tools and validate their clinical applicability for PCOS self-education/management.

Citation

Graca S, Dallaway A, Alloh F, Randeva HS, Kite C and Kyrou I (2026) Assessing ChatGPT vs. evidence-based online responses for polycystic ovary syndrome self-management and education: an international cross-sectional blinded survey of healthcare professionals. Front. Digit. Health 7:1700018. doi: 10.3389/fdgth.2025.1700018

PubMed ID

41988362 (pubmed)

Additional Links

https://www.frontiersin.org/journals/digital-health/articles/10.3389/fdgth.2025.1700018/full

Type

Journal article

Language

en

Description

© 2026 The Authors. Published by Frontiers Media SA. This is an open access article available under a Creative Commons licence. The published version can be accessed at the following link on the publisher’s website: https://doi.org/10.3389/fdgth.2025.1700018

ISSN

2673-253X

EISSN

2673-253X

Collections

Faculty of Education, Health and Wellbeing