{"id":16545,"date":"2024-09-20T11:59:12","date_gmt":"2024-09-20T09:59:12","guid":{"rendered":"https:\/\/is.ijs.si\/?p=16545"},"modified":"2026-02-17T13:43:41","modified_gmt":"2026-02-17T12:43:41","slug":"use-and-limitations-of-chatgpt-in-mental-health-disorderstesting-chatgpts-performance-on-medical-diagnostic-tasksuse-and-limitations-of-chatgpt-in-mental-health-disorders","status":"publish","type":"post","link":"https:\/\/is.ijs.si\/?p=16545","title":{"rendered":"Testing ChatGPT\u2019s Performance on Medical Diagnostic Tasks"},"content":{"rendered":"\n<p>Alexander Perko and Franz Wotawa<\/p>\n<p><strong>Abstract<\/strong><br \/>Large Language Models and chat interfaces like ChatGPT have<br \/>become increasingly important recently, receiving a lot of attention<br \/>even from the general public. People use these tools not only<br \/>to summarize or translate text but also to answer questions, including<br \/>medical ones. For the latter, giving reliable feedback is of<br \/>utmost importance, which is hard to assess. Therefore, we focus<br \/>on validating the feedback of ChatGPT and propose a testing procedure<br \/>utilizing other medical sources to determine the quality<br \/>of feedback for more straightforward medical diagnostic tasks.<br \/>This paper outlines the problem, discusses available sources, and<br \/>introduces the validation method. Moreover, we present the first<br \/>results obtained when applying the testing framework to Chat-<br \/>GPT.<\/p>\n\n\n\n<div data-wp-interactive=\"core\/file\" class=\"wp-block-file\"><object data-wp-bind--hidden=\"!state.hasPdfPreview\" hidden class=\"wp-block-file__embed\" data=\"https:\/\/is.ijs.si\/wp-content\/uploads\/2024\/09\/IS2024_-_CHATGPT_in_MEDICINE_paper_7-2.pdf\" type=\"application\/pdf\" style=\"width:100%;height:600px\" aria-label=\"Embed of IS2024_-_CHATGPT_in_MEDICINE_paper_7.\"><\/object><a id=\"wp-block-file--media-37f6e025-0e25-4e7b-bd26-365bdfc1d264\" href=\"https:\/\/is.ijs.si\/wp-content\/uploads\/2024\/09\/IS2024_-_CHATGPT_in_MEDICINE_paper_7-2.pdf\">IS2024_-_CHATGPT_in_MEDICINE_paper_7<\/a><a href=\"https:\/\/is.ijs.si\/wp-content\/uploads\/2024\/09\/IS2024_-_CHATGPT_in_MEDICINE_paper_7-2.pdf\" class=\"wp-block-file__button wp-element-button\" download aria-describedby=\"wp-block-file--media-37f6e025-0e25-4e7b-bd26-365bdfc1d264\">Download<\/a><\/div>\n","protected":false},"excerpt":{"rendered":"<p>Alexander Perko and Franz Wotawa AbstractLarge Language Models and chat interfaces like ChatGPT havebecome increasingly important recently, receiving a lot of attentioneven from the general public. People use these tools not onlyto summarize or translate text but also to answer questions, includingmedical ones. For the latter, giving reliable feedback is ofutmost importance, which is hard to assess. Therefore, we focuson validating the feedback of ChatGPT and propose a testing procedureutilizing other medical sources to determine the qualityof feedback for more straightforward medical diagnostic tasks.This paper outlines the problem, discusses available sources, andintroduces the validation method. Moreover, we present the firstresults obtained when applying the testing framework to Chat-GPT.<\/p>\n","protected":false},"author":29,"featured_media":24966,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[117,102],"tags":[],"class_list":["post-16545","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-doi-chat-2024","category-papers"],"_links":{"self":[{"href":"https:\/\/is.ijs.si\/index.php?rest_route=\/wp\/v2\/posts\/16545","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/is.ijs.si\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/is.ijs.si\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/is.ijs.si\/index.php?rest_route=\/wp\/v2\/users\/29"}],"replies":[{"embeddable":true,"href":"https:\/\/is.ijs.si\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=16545"}],"version-history":[{"count":5,"href":"https:\/\/is.ijs.si\/index.php?rest_route=\/wp\/v2\/posts\/16545\/revisions"}],"predecessor-version":[{"id":26318,"href":"https:\/\/is.ijs.si\/index.php?rest_route=\/wp\/v2\/posts\/16545\/revisions\/26318"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/is.ijs.si\/index.php?rest_route=\/wp\/v2\/media\/24966"}],"wp:attachment":[{"href":"https:\/\/is.ijs.si\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=16545"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/is.ijs.si\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=16545"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/is.ijs.si\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=16545"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}