Skip to content

[Core] Add Qwen-MoE dense/source-layout and routing-side hooks #48

Description

@XbzOnGit

Part of #47.

Goal

Add Qwen-MoE support for the safe hook subset that can reuse the current transport and DB path.

This issue is scoped to:

  • Dense/source-layout hooks.
  • Token-major MoE routing-side hooks:
    • router_logits
    • topk_ids
    • topk_weights
  • vLLM Qwen2-MoE integration and validation.
  • HF hook surface wiring where practical.

Full dynamic expert-local EP artifacts are intentionally out of scope for this issue and should be tracked separately.

Completed

  • Added Qwen2-MoE vLLM hook variants.
  • Added MoE routing-side hooks:
    • router_logits
    • topk_ids
    • topk_weights
  • Kept supported hooks in token-major/source-layout form so they reuse the existing transport and DB path.
  • Added vLLM EP test controls:
    • E2E_ENABLE_EP
    • E2E_ALL2ALL_BACKEND
  • Validated vLLM paths on the current machine:
    • TP=2, EP disabled, compare passed.
    • TP=2, EP enabled with allgather_reducescatter, eager passed.
    • TP=2, EP enabled with allgather_reducescatter, CUDA graph passed.

Tasks

  • Add Qwen2-MoE vLLM hook variants.
  • Add MoE routing-side hook specs.
  • Add vLLM EP test controls.
  • Validate vLLM TP=2 EP-disabled path.
  • Validate vLLM TP=2 EP-enabled allgather_reducescatter eager path.
  • Validate vLLM TP=2 EP-enabled allgather_reducescatter CUDA graph path.
  • Document remaining full EP artifact work as follow-up scope.

Activity

  1. self-assigned this
    on May 14, 2026
  2. changed the title [-][Core] Support EP MLP_POST hooks (dynamic per-rank token counts)[/-] [+][Core] Add Qwen-MoE dense/source-layout and routing-side hooks[/+] on May 31, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions