BadMoE: Backdooring Mixture-of-Experts LLMs via Routing-Aware Triggers and Task-Underutilized Experts
arXiv:2504.18598v3 Announce Type: replace-cross Abstract: Mixture-of-Experts (MoE) architectures are rapidly becoming the standard for building scalable and efficient large language models (LLMs), thanks to their routing mechanism that activates only a subset of experts for each input. However, the modular structure and sparse activation of experts also introduce new security risks, as manipulating a small portion of experts may influence the behavior. In this work, we study a white-box, task-specific model-supply-chain setting and propose BadMoE, a routing-aware backdoor attack against MoE LLMs. We motivate BadMoE through a routing-aware decomposition of the MoE-layer output, which separates the contributions of the selected target experts from those of the remaining experts. This analysis highlights two measurable factors underlying the attack: the routing mass assigned to the target experts and their relative contribution to the layer output. Guided by this insight, BadMoE selects task-underutilized experts and utilizes routing-aware triggers to activate them, enabling reliable activation of task-specific, predefined backdoor behaviors. Specifically, BadMoE involves three steps: 1) identifying experts with low usage on a target-task calibration set, 2) optimizing a routing-aware trigger toward these experts, and 3) associating the selected-expert activations with attacker-specified behavior. We evaluate BadMoE on Mixtral, OLMoE, and Deepseek across sentiment, topic, and emotion classification, sentiment steering, and targeted refusal.
Get stories like this every Monday
The week's most important AI news in one email. Free, unsubscribe anytime.
Score: 3.15
Engagement proxy: 0
Canonical link: https://arxiv.org/abs/2504.18598