SEAL Anchors MoE Safety in Shared Experts Ahead of CCS 2026

A Vrije Universiteit Amsterdam paper accepted to ACM CCS 2026 trains only shared experts so Mixture-of-Experts safety does not depend on the router. On four hybrid MoE models, SEAL cut several attack-success rates while changing general-capability scores by at most about 1.4 percent.

Mixture-of-Experts (MoE) models grow parameter counts without a matching rise in compute per inference because a router sends each token to a small subset of experts. The parameters that actually run are therefore not fixed. A paper from Vrije Universiteit Amsterdam, SEAL: Reinforcing Global Safety in Mixture-of-Experts through Shared Expert ALignment, treats that as a structural safety risk: safety behavior can depend on the routing path. The paper has been accepted to ACM CCS 2026. A preprint is posted at https://arxiv.org/pdf/2609.02293.

Safety that follows sparse routes

In a dense Transformer, each layer typically runs attention and a feed-forward network (FFN) for every token. MoE replaces one FFN with many expert FFNs. A model may have 64 experts and, for a given token, run only two of them. The next token may use a different pair.

If refusal and harm recognition sit mainly in a few routed experts, a shift in routing from a prompt, from fine-tuning, or from weight edits can leave those experts unused. The safety parameters remain in the file, but they do not run. Many existing MoE defenses try to keep the router sending risky inputs to the right experts, or to repair those routed experts. Those defenses still require a reliable router. SEAL instead looks for a location every token must pass, independent of the router.

Shared experts as a required hop

Many current models use a hybrid MoE: routed experts run only when selected, while shared experts run for every token and do not compete in the router. Shared experts were built to hold common knowledge so it is not copied across many experts. SEAL treats that always-on path as a safety anchor. When the router works, routed safety experts and the shared expert can both run. When routing is disturbed, the shared expert still runs.

The authors first asked whether shared experts already hold safety-related features. They studied four hybrid MoE models: Qwen1.5-MoE, DeepSeekMoE, GLM-4.7-Flash, and Qwen3.5. They compared neuron activations on harmful and ordinary inputs to find safety-critical neurons.

Shared-expert versus routed-expert safety-neuron density ratios were about 0.63, 0.88, 0.89, and 0.86 on the four models. Density in shared experts was usually a bit lower, but of the same order. Coverage differs. A routed expert's safety neurons fire only if the router picks that expert. Shared-expert safety neurons run on every token.

SEAL training

SEAL does not retrain the full model and does not change the router. The authors add LoRA only on each layer's shared-expert FFN and align it with Direct Preference Optimization (DPO). The set has about 41,000 human-labeled safety preference pairs. Updates stay inside the shared expert.

Trainable parameters were 0.06 percent to 0.25 percent of each model. On 35 billion-parameter Qwen3.5, about 19.7 million parameters were trained.

SEAL++ adds a further constraint. Before DPO, the authors identify the original model's safety-critical neurons and apply an orthogonality penalty so new updates do not overwrite those directions. The stated aim is to keep existing safety features more stable under later neuron-pruning attacks.

Measured attack success and capability cost

Tests used the four architectures under prompt-level harmful requests and jailbreaks, malicious fine-tuning that can change weights, pruning of safety-critical neurons, and combined prompt-plus-pruning cases.

On direct harmful prompts, DeepSeekMoE attack success fell from 61.3 percent to 8.5 percent. GLM fell from 20.7 percent to as low as 1.8 percent. Qwen1.5 fell from 27.5 percent to 13.1 percent. Qwen3.5 started at 5.8 percent and fell to as low as about 1.1 percent. Mean scores on five general-capability benchmarks moved by at most about 1.4 percent.

A targeted case left the shared expert unpruned and pruned routed-expert safety neurons, then added harmful prompts. On DeepSeekMoE, attack success was 74.2 percent on the base model, 13.8 percent with SEAL, and 14.0 percent with SEAL++. SEAL had not trained any routed-expert parameters. The authors call the remaining shared-expert signal Cross-Scope Transfer.

On Qwen1.5, training only the top-10 routed experts by safety score cut harmful-prompt success from 27.5 percent to 18.8 percent. Training only the shared expert reached 13.1 percent, using about one-fourth to one-fifth as many trained parameters.

Limits

SEAL did not keep models safe after malicious fine-tuning. DeepSeekMoE still showed 86.5 percent attack success after that training even with SEAL. Qwen1.5 stayed above 70 percent. The authors describe the method as adding resilience, because fine-tuning can rewrite many other parameters.

Results also depend on shared-expert capacity and where safety features already live. Qwen3.5's shared expert has intermediate dimension 512, about one-tenth of Qwen1.5, and more safety capacity sits in routed experts. In some harmful-prompt plus routed-expert pruning cases, SEAL raised attack success from 5.9 percent to 13.2 percent, and SEAL++ to 15.3 percent. The authors attribute that to a shared expert too small to replace lost routed signals, plus training that changed activations and routing.

A shared expert is a convenient place to host safety capacity only when that capacity is large enough and is not concentrated almost entirely in routed experts. SEAL does not apply to MoE models without shared experts, such as Mixtral. The study covers four architectures and a largest model of about 35 billion parameters.