Israel Medical Association Journal

Artificial Intelligence (AI), particularly large language models (LLMs) like OpenAI's ChatGPT, has shown potential in various medical fields, including pediatrics. We evaluated the utility and integration of LLMs in pediatric medicine. We conducted a search in PubMed using specific keywords related to LLMs and pediatric care. Studies were included if they assessed LLMs in pediatric settings, were published in English, peer-reviewed, and reported measurable outcomes. Sixteen studies spanning pediatric sub-specialties such as ophthalmology, cardiology, otology, and emergency medicine were analyzed. The findings indicate that LLMs provide valuable diagnostic support and information management. However, their performance varied, with limitations in complex clinical scenarios and decision-making. Despite excelling in tasks requiring data summarization and basic information delivery, the effectiveness of the models in nuanced clinical decision-making was restricted. LLMs, including ChatGPT, show promise in enhancing pediatric medical care but exhibit inconsistent performance in complex clinical situations. This finding underscores the importance of continuous human oversight. Future integration of LLMs into clinical practice should be approached with caution to ensure they supplement, rather than supplant, expert medical judgment.

Background: Advancements in artificial intelligence (AI) and natural language processing (NLP) have led to the development of language models such as ChatGPT. These models have the potential to transform healthcare and medical research. However, understanding their applications and limitations is essential.

Objectives: To present a view of ChatGPT research and to critically assess ChatGPT's role in medical writing and clinical environments.

Methods: We performed a literature review via the PubMed search engine from 20 November 2022, to 23 April 2023. The search terms included ChatGPT, OpenAI, and large language models. We included studies that focused on ChatGPT, explored its use or implications in medicine, and were original research articles. The selected studies were analyzed considering study design, NLP tasks, main findings, and limitations.

Results: Our study included 27 articles that examined ChatGPT's performance in various tasks and medical fields. These studies covered knowledge assessment, writing, and analysis tasks. While ChatGPT was found to be useful in tasks such as generating research ideas, aiding clinical reasoning, and streamlining workflows, limitations were also identified. These limitations included inaccuracies, inconsistencies, fictitious information, and limited knowledge, highlighting the need for further improvements.

Conclusions: The review underscores ChatGPT's potential in various medical applications. Yet, it also points to limitations that require careful human oversight and responsible use to improve patient care, education, and decision-making.

Background: Completing internal medicine specialty training in Israel involves passing the Israel National Internal Medicine Exam (Shlav Aleph), a challenging multiple-choice test. multiple-choice test. Chat generative pre-trained transformer (ChatGPT) 3.5, a language model, is increasingly used for exam preparation.

Objectives: To assess the ability of ChatGPT 3.5 to pass the Israel National Internal Medicine Exam in Hebrew.

Methods: Using the 2023 Shlav Aleph exam questions, ChatGPT received prompts in Hebrew. Textual questions were analyzed after the appeal, comparing its answers to the official key.

Results: ChatGPT 3.5 correctly answered 36.6% of the 133 analyzed questions, with consistent performance across topics, except for challenges in nephrology and biostatistics.

Conclusions: While ChatGPT 3.5 has excelled in English medical exams, its performance in the Hebrew Shlav Aleph was suboptimal. Factors include limited training data in Hebrew, translation complexities, and unique language structures. Further investigation is essential for its effective adaptation to Hebrew medical exam preparation.