{"id":16575,"date":"2024-09-20T12:22:14","date_gmt":"2024-09-20T10:22:14","guid":{"rendered":"https:\/\/is.ijs.si\/?p=16575"},"modified":"2025-03-26T13:16:46","modified_gmt":"2025-03-26T12:16:46","slug":"multilingual-hate-speech-modeling-by-leveraging-inter-annotator-disagreement","status":"publish","type":"post","link":"https:\/\/is.ijs.si\/?p=16575","title":{"rendered":"Multilingual Hate Speech Modeling by Leveraging Inter-Annotator Disagreement"},"content":{"rendered":"\n<p>Patricia-Carla Grigor, Petra Kralj Novak and Bojan Evkoski<\/p>\n<p><strong>Abstract<\/strong><br \/>As social media usage increases, so does the volume of toxic<br \/>content on these platforms, motivating the Machine Learning<br \/>(ML) community to focus on automating hate speech detec-<br \/>tion. While modern ML algorithms are known to provide nearly<br \/>human-like results for a variety of downstream Natural Lan-<br \/>guage Processing (NLP) tasks, the classification of hate speech<br \/>is still an open challenge, partially due to its subjective anno-<br \/>tation, which often leads to disagreement between annotators.<br \/>This paper adopts a perspectivist approach that embraces sub-<br \/>jectivity, leveraging conflicting annotations to enhance model<br \/>performance in real-world scenarios. A state-of-the-art multi-<br \/>lingual language model for hate speech detection is introduced,<br \/>trained, and evaluated using diamond standard data with metrics<br \/>that consider disagreement. Various strategies for incorporat-<br \/>ing disagreement are compared in the process. Results demon-<br \/>strate that the model performs equally or better on all evalu-<br \/>ated languages compared to respective monolingual models and<br \/>drastically outperforms on multilingual data. This highlights<br \/>the effectiveness of multilingual and perspectivist methods in<br \/>addressing the complexities of hate speech detection. The pre-<br \/>sented multilingual hate speech detection model is available at:<br \/>https:\/\/huggingface.co\/IMSyPP\/hate_speech_multilingual.<\/p>\n<p>\u00a0<\/p>\n\n\n\n<div data-wp-interactive=\"core\/file\" class=\"wp-block-file\"><object data-wp-bind--hidden=\"!state.hasPdfPreview\" hidden class=\"wp-block-file__embed\" data=\"https:\/\/is.ijs.si\/wp-content\/uploads\/2024\/10\/IS2024_-_SIKDD_2024_paper_7-2.pdf\" type=\"application\/pdf\" style=\"width:100%;height:600px\" aria-label=\"Embed of IS2024_-_SIKDD_2024_paper_7-2.\"><\/object><a id=\"wp-block-file--media-4ee78f72-8591-4700-ad71-aa459695c0c2\" href=\"https:\/\/is.ijs.si\/wp-content\/uploads\/2024\/10\/IS2024_-_SIKDD_2024_paper_7-2.pdf\">IS2024_-_SIKDD_2024_paper_7-2<\/a><a href=\"https:\/\/is.ijs.si\/wp-content\/uploads\/2024\/10\/IS2024_-_SIKDD_2024_paper_7-2.pdf\" class=\"wp-block-file__button wp-element-button\" download aria-describedby=\"wp-block-file--media-4ee78f72-8591-4700-ad71-aa459695c0c2\">Download<\/a><\/div>\n","protected":false},"excerpt":{"rendered":"","protected":false},"author":29,"featured_media":24966,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[109,102],"tags":[],"class_list":["post-16575","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-doi-sikdd-2024","category-papers"],"_links":{"self":[{"href":"https:\/\/is.ijs.si\/index.php?rest_route=\/wp\/v2\/posts\/16575","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/is.ijs.si\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/is.ijs.si\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/is.ijs.si\/index.php?rest_route=\/wp\/v2\/users\/29"}],"replies":[{"embeddable":true,"href":"https:\/\/is.ijs.si\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=16575"}],"version-history":[{"count":4,"href":"https:\/\/is.ijs.si\/index.php?rest_route=\/wp\/v2\/posts\/16575\/revisions"}],"predecessor-version":[{"id":24999,"href":"https:\/\/is.ijs.si\/index.php?rest_route=\/wp\/v2\/posts\/16575\/revisions\/24999"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/is.ijs.si\/index.php?rest_route=\/wp\/v2\/media\/24966"}],"wp:attachment":[{"href":"https:\/\/is.ijs.si\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=16575"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/is.ijs.si\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=16575"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/is.ijs.si\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=16575"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}