NeurIPS 2026 · Poster

Backdoor Containment via Expert Quarantine and Shutdown in LLMs

Jianwei Li  ·  Min-Seon Kim  ·  Jung-Eun Kim

QES steers backdoor behavior into a designated LoRA expert during fine-tuning, then disables that expert at deployment.

Read the paper arXiv Code

01 / Overview

Can fine-tuning make a backdoor removable?

A model fine-tuned on poisoned data can learn an association between a hidden trigger and an attacker’s target response alongside useful behavior. Quarantined Expert Shutdown (QES) trains the model to concentrate trigger-conditioned behavior in a designated expert that can be switched off.

The paper calls this strategy learn, but channel. Training shapes where behavior is learned, and deployment disables the quarantined expert for every input.

02 / Method

Train with a quarantine expert, then switch it off

  1. STEP 01

    Add routed LoRA experts

    Freeze the base model, add expert-specific LoRA branches with lightweight routers, and designate one expert for quarantine.

  2. STEP 02

    Shape routing during training

    Use attention-derived token scores and routing objectives to steer trigger-conditioned computation toward the quarantine expert while training on the full dataset.

  3. STEP 03

    Disable the quarantine expert

    Exclude that expert from routing at deployment. The remaining experts handle inputs without a separate trigger detector.

03 / Selected result

One expert shutdown sharply reduced attack success

Qwen2-7B-Instruct with BadNets on Sentiment Steering.

Attack success rate100% → 1.5%Expert enabled → disabled
Downstream utility (GSM8K)−1.14 ppAccuracy change after shutdown

Both measurements compare the same checkpoint before and after disabling the quarantine expert. See Figure 3 for the full expert shutdown comparison.

04 / Scope

When containment works

QES assumes the defender controls the model’s fine-tuning and can add routed LoRA experts. The method uses the potentially poisoned dataset without requiring exact trigger-token annotations or a clean reference model.

Shutdown depends on how well training concentrates backdoor behavior in the designated expert. The paper reports limitations for digit-heavy Sleeper-style triggers and VPI-style attacks on the tested grouped-query attention models.

05 / Citation

Cite this work

BibTeX
@misc{li2026qes,
  title = {Backdoor Containment via Expert Quarantine and Shutdown in {LLMs}},
  author = {Li, Jianwei and Kim, Min-Seon and Kim, Jung-Eun},
  year = {2026},
  eprint = {2610.00663},
  archivePrefix = {arXiv},
  primaryClass = {cs.AI},
  url = {https://arxiv.org/abs/2610.00663}
}