LLM-BotGuard: A Novel Framework for Detecting LLM-Driven Bots With Mixture of Experts and Graph Neural Networks
Date
Supervisor
Item type
Journal Article
Degree name
Journal Title
Journal ISSN
Volume Title
Publisher
Institute of Electrical and Electronics Engineers (IEEE)
Abstract
Detecting social media bots has become increasingly critical due to their detrimental impact on online environments. With the emergence of sophisticated large language models (LLM) such as ChatGPT, bot detection faces new challenges. These bots based on LLMs exhibit human-like behaviors, and it is difficult for traditional detection approaches to identify them effectively. Such conventional methods struggle with the advanced features associated with LLM-driven bots, which possess contextual understanding and mimic human interaction patterns. The significance of detecting LLM-driven bots lies in their increased difficulty of detection and their potential to inflict more covert harm compared with traditional bots. To address these challenges, we propose LLM-BotGuard, a novel detection model that is capable of capturing the unique features of LLM-driven bots alongside other bot characteristics through three key modules, i.e., pattern-informed feature extraction module, mixture of experts module, and graph module with graph sample and aggregation networks. Extensive experiments have been conducted to evaluate the performance of the proposed LLM-BotGuard. The results demonstrate that LLM-BotGuard significantly outperforms baseline methods in detecting LLM-driven bots. The proposed LLM-BotGuard offers a robust solution for identifying sophisticated LLM-driven bots in online social networks.Description
Keywords
Source
IEEE Transactions on Computational Social Systems, ISSN: 2373-7476 (Print); 2329-924X (Online), Institute of Electrical and Electronics Engineers (IEEE), 1-13. doi: 10.1109/tcss.2025.3545822
Publisher's version
Rights statement
© 2024 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.