DOI 2024 CHAT
Jovana Petrović, Iva Binić, Maša Vacev and Stevo Lukić
Abstract / PovzetekChatGPT is one of the most advanced and rapidly evolvinglarge language model-based chatbots. It excels in everythingfrom handling simple questions to performing complex medicalexaminations. While current technology cannot replace theexpertise and judgment of skilled psychiatrists, it can assist inearly detection of mental problems, patient evaluations,differential diagnoses, psychotherapy and in planning andconducting medical research. Ensuring privacy and adhering toprofessional, ethical, and legal standards is crucial whenprocessing training data. This is especially important in mentalhealth settings, where disclosing sensitive personal informationincreases the risk of data misuse and the potential for harmfuladvice. Current uses of ChatGPT in mental health care areconstrained by its design as a general chatbot, rather than aspecialized psychiatric tool. Despite this, the model provesuseful for handling routine psychiatric and administrative tasks.As GPT technology evolves, it holds significant promise forpsychiatry, including integration into diagnostics,psychotherapy, and early detection of mental health issues. Todeploy these advancements responsibly and effectively, it iscrucial to develop and refine professional ethical standards andpractice guidelines.
IS2024_-_CHATGPT_in_MEDICINE_paper_6-1Download [...]
Gordana Petrovska Dojchinovska, Monika Simjanoska Misheva and Kostadin Mishev
ABSTRACTIntegrating advanced open-source large language models, such asLlama or GatorTron, into the healthcare domain presents a novelapproach to enhancing communication between physicians andpatients. This review paper examines the potential of Llama andGatorTron to improve patient-provider interactions, focusing on themodels’ ability to process and generate human-like language in real-time clinical settings. Any large language model (LLM) used in aclinical setting, especially one that is used to improve patient-doctorcommunication, needs to undergo cultural sensitivity training, asthe datasets these models are trained on seem to be diverse and theyinclude specificities, both linguistic and medical, that may be validto one population but completely exclude another. Furthermore,the model needs to be capable of providing accurate responses thatare context-sensitive and that also align with clinical guidelinesand patient needs. Other things to take into consideration whenfine-tuning LLMs to local medical customs are to adapt them to dif-ferent geographies and the underlying linguistic demands, to helpthem employ ethical and responsible AI practices so that no biasesor stereotypes are perpetuated and that, at the same time, help toprotect patient privacy and data security. Finally, feedback providedby both the physicians and the patients needs to be incorporated tofurther refine any model that is to be used in a medical setting. Overtime, this will help the model to become more nuanced. Commoncommunication barriers, such as medical jargon, cultural differ-ences, and patient literacy, which often hinder effective dialogueare also considered in this review paper, as well as how fine-tunedLLMs address those issues. By synthesizing current research andpractical applications, this paper aims to provide a comprehensiveunderstanding of the potential of fine-tuned large language mod-els to transform healthcare communication, ultimately improvingpatient outcomes and satisfaction.
IS2024_-_CHATGPT_in_MEDICINE_paper_11-1Download [...]
Alexander Perko, Iulia Nica and Franz Wotawa
AbstractLarge Language Models (LLMs) like GPT-4o are of growing interest. Interfaces such as ChatGPT invite an ever-growing number of people to ask questions, including health advice, which bringsin additional risks for harm. It is well known that tools based on LLMs tend to hallucinate or deliver different answers for the same or similar questions. In both cases, the outcome might bewrong or incomplete, possibly leading to safety issues. In this paper, we investigate the outcome of ChatGPT when we ask similar questions in the medical domain. In particular, we suggest usingcombinatorial testing to generate variants of questions aimed at identifying wrong or misleading answers. In detail, we discuss the general framework and its parts and present a proof-of-conceptutilizing a medical query and ChatGPT.
IS2024_-_CHATGPT_in_MEDICINE_paper_10-2Download [...]
Matic Zadobovšek, Primož Kocuvan and Matjaž Gams
AbstractThe escalating workload demands on healthcare professionalsare leading to systemic overload, resulting in a decline in theefficiency of public health services. This situation necessitatesthe development of solutions that can alleviate the burden onphysicians while ensuring comprehensive patient care. Recentadvancements in generative artificial intelligence, particularlyin the field of medicine, have demonstrated that large languagemodels (LLMs) can outperform doctors in specific tasks, high-lighting their potential as valuable tools for reducing the strainon healthcare providers. This study focuses on the developmentof the HomeDOCtor application, which integrates additional val-idated medical knowledge into the GPT-4o LLM. The objective ofthis application and the enhanced LLM is to offer users reliableaccess to a medical chatbot capable of providing accurate andtimely responses to health-related inquiries. The chatbot’s be-havior has been meticulously tested and refined in collaborationwith a team of physicians. The findings of this research offerinsights into the development of such systems and explore theirpotential application within the Slovenian healthcare system.
IS2024_-_CHATGPT_in_MEDICINE_paper_8-1Download [...]
Alexander Perko and Franz Wotawa
AbstractLarge Language Models and chat interfaces like ChatGPT havebecome increasingly important recently, receiving a lot of attentioneven from the general public. People use these tools not onlyto summarize or translate text but also to answer questions, includingmedical ones. For the latter, giving reliable feedback is ofutmost importance, which is hard to assess. Therefore, we focuson validating the feedback of ChatGPT and propose a testing procedureutilizing other medical sources to determine the qualityof feedback for more straightforward medical diagnostic tasks.This paper outlines the problem, discusses available sources, andintroduces the validation method. Moreover, we present the firstresults obtained when applying the testing framework to Chat-GPT.
IS2024_-_CHATGPT_in_MEDICINE_paper_7-2Download [...]
Zlate Dodevski, Tanja Pavleska and Vladimir Trajkovikj
Abstract
Federated learning (FL) represents a pivotaladvancement in applying Machine Learning (ML) inhealthcare. It addresses the challenges of data privacy andsecurity by facilitating model transferability across institutions.This paper explores the effective employment of FL to enhancethe deployment of large language models (LLMs) in healthcaresettings while maintaining stringent privacy standards.Along a detailed examination of the challenges in applyingLLMs to the healthcare domain, including privacy, security,regulatory constraints, and training data quality, we present afederated learning architecture tailored for LLMs in healthcare.This architecture outlines the roles and responsibilities ofparticipating entities, providing a framework for securecollaboration. We further analyze privacy-preservingtechniques such as differential privacy and secure aggregationin the context of federated LLMs for healthcare, offeringinsights into their practical implementation.Our findings suggest that federated learning is a viablechoice for enhancing he capabilities of LLMs in healthcare whilepreserving patient privacy. In addition, we also identifypersistent challenges in areas such as computational andcommunicational efficiency, lack of benchmarks and tailoredFL aggregation algorithms applied to LLMs, modelperformance, and ethical concerns in participant selection. Bycritically evaluating the proposed approach and highlighting itspotential benefits and limitations in real-world healthcaresettings, this work provides a foundation for future research insecure and privacy-preserving ML deployment in healthcare.
IS2024_-_CHATGPT_in_MEDICINE_paper_5-1Download [...]
Kennedy Addo
AbstractThis study assesses the impact of AI, specifically deeplearning algorithms, on diagnostic accuracy. It identifiesvarious diseases where AI is applied, such as cancer,cardiovascular diseases, and neurological disorders. Byanalyzing existing AI implementations, the researchhighlights improvements in early detection and diagnosticprecision. Additionally, the study proposes a novel deeplearning model to enhance clinicians’ decision-making atthe point of care. This framework aims to integrateseamlessly with clinical workflows, providing real-timeinsights and recommendations. The proposed modelfocuses on improving patient outcomes through accurate,timely, and data-driven clinical decisions.
IS2024_-_CHATGPT_in_MEDICINE_paper_4-1Download [...]
Matjaz Gams and Primož Kocuvan
ABSTRACTThe emergence of artificial intelligence (AI) has significantlyimpacted various fields, including the peer review process inacademic and scientific research. ChatGPT, a large languagemodel developed by OpenAI, has shown potential in automatingand enhancing the review of conference healthcare papers. OurHealthReview AI tool can process and analyze large volumes oftext rapidly, providing feedback and insights that streamline thepeer review process, reduce human workload, and increaseefficiency. This paper presents a web application developedusing the Flask framework that enables users to upload PDF filescontaining research papers and utilizes ChatGPT to generatereviews for each paper. The methodology, results and potentialimplications of this application are discussed, highlighting boththe advantages and the challenges of integrating AI into theacademic review process.
IS2024_-_CHATGPT_in_MEDICINE_paper_3-1Download [...]