Repository navigation

[PyTorch] DeepSeekV3Layer: full MoE transformer layer (MLA + DeepSeek MoE) - #3471

Open

pggPL wants to merge 61 commits into

NVIDIA:mainfrom

pggPL:deepseek_v3_layer

Open

[PyTorch] DeepSeekV3Layer: full MoE transformer layer (MLA + DeepSeek MoE)#3471
pggPL wants to merge 61 commits into
NVIDIA:mainfrom
pggPL:deepseek_v3_layer

Commits on Aug 18, 2026
- [PyTorch] Add DeepSeekV3Layer skeleton (MLA + MoE)
  
  pggPL
  and
  claude
  committed
  45ed767
- Move DeepSeekV3 skeleton to models/deepseek_v3 subpackage
  
  pggPL
  and
  claude
  committed
  c306c6f
- Add DeepSeekV3 layer entries to PyTorch API docs
  
  pggPL
  and
  claude
  committed
  f73d04e
- [PyTorch] Implement DeepSeekV3Layer: MLA + MoE from TE building blocks
  
  pggPL
  and
  claude
  committed
  09f28a9
- Add distributed EP test for DeepSeekV3 MoE/layer
  
  pggPL
  and
  claude
  committed
  e23100b
- Fix EP wgrad test collective + zero EP recv/grad buffers
  
  pggPL
  and
  claude
  committed
  4c6e1e8
- Use fused MLA RoPE kernels in MultiLatentAttention
  
  pggPL
  and
  claude
  committed
  aa17c37
Commits on Aug 21, 2026
- Add HF transformers numeric reference test for DeepSeekV3Layer
  
  pggPL
  and
  claude
  committed
  c713af7
- Docstring cleanups for lint and docs build
  
  pggPL
  and
  claude
  committed
  88028a9
- Move model-specific layers to a dedicated docs page
  
  pggPL
  and
  claude
  committed
  5f68c9b
- Drop HF-transformers comparison test from the repo
  
  pggPL
  and
  claude
  committed
  9db495a
- Docs: reduce models page to a plain API listing
  
  pggPL
  and
  claude
  committed
  4883b17
Commits on Sep 3, 2026
Commits on Sep 8, 2026
- Merge remote-tracking branch 'origin/main' into deepseek_v3_layer
  pggPL
  committed
  5c41a3b
- Merge remote-tracking branch 'upstream/main' into deepseek_v3_layer
  pggPL
  committed
  b58a7ae
Commits on Sep 9, 2026
- [PyTorch] DeepSeekV3MoE EP: drop full-buffer zeroing of recv/grad buffers
  
  pggPL
  and
  claude
  committed
  93a40df
- [PyTorch] Add DeepSeekV3Layer expert-parallel example
  
  pggPL
  and
  claude
  committed
  5e36ef1
- [PyTorch] DeepSeekV3Layer example: naive PyTorch MoE and dense baselines
  
  pggPL
  and
  claude
  committed
  3612879
- [PyTorch] DeepSeekV3Layer example: two-level plain PyTorch MoE baseline
  
  pggPL
  and
  claude
  committed
  eb36d7d
- [PyTorch] DeepSeekV3: MLA TP gradient reduction, RoPE guards, per-forward EP buffer
  
  pggPL
  and
  claude
  committed
  7a1693a
- [PyTorch] DeepSeekV3Layer example: argument validation, gradient checks, differentiable naive probs
  
  pggPL
  and
  claude
  committed
  49ecaff
- [PyTorch] Drop example-dependent tests from test_models.py
  
  pggPL
  and
  claude
  committed
  a131eb3
- [Docs] Organize DeepSeekV3 example README around results
  pggPL
  committed
  b5605bc
- [Docs] Keep naive MoE comparison in headline results
  pggPL
  committed
  e9fac4f
- [PyTorch] DeepSeekV3Layer example README: remeasure all variants on the current code
  
  pggPL
  and
  claude
  committed
  00bcbd9
- [PyTorch] DeepSeekV3Layer example README: split NCCL EP permute from rank-skew wait in the breakdown
  
  pggPL
  and
  claude
  committed
  4ffa639
- [PyTorch] DeepSeekV3MoE: optional caller-owned EpBuffer
  
  pggPL
  and
  claude
  committed
  0367781
- [PyTorch] DeepSeekV3Layer example README: medians of three runs, breakdown from an unskewed node
  
  pggPL
  and
  claude
  committed
  da81d1d
Commits on Sep 10, 2026
- [PyTorch] Validate MoE grouping and check synchronized expert bias updates
  pggPL
  committed
  2f7859a
Commits on Sep 17, 2026
- Merge upstream/main into deepseek_v3_layer
  pggPL
  committed
  5371619
- [PyTorch] Move MLA RoPE helpers into attention
  pggPL
  committed
  db03bd8
- [PyTorch] Document MLA RoPE helpers
  pggPL
  committed
  e90cbd7
- [Docs] Nest MLA RoPE helpers under Other
  pggPL
  committed
  7030c91
Commits on Sep 28, 2026
- Merge upstream/main into deepseek_v3_layer
  pggPL
  committed
  793b46b
Commits on Sep 30, 2026
- Fix DeepSeekV3 layer review findings
  pggPL
  committed
  939c9db
- Use bounded inputs and RHT in quantized MoE regression test
  pggPL
  committed
  00dbae6
- Merge remote-tracking branch 'upstream/main' into deepseek_v3_layer
  pggPL
  committed
  271a1be
- Refresh DeepSeekV3 benchmarks on GB200 and GB300
  pggPL
  committed
  16eb75d
Commits on Oct 1, 2026
- Remove benchmark commit provenance from DeepSeek README
  pggPL
  committed
  b32a4f8
- Merge upstream main for MoE Sequential ops
  pggPL
  committed
  85612d4
- Compose DeepSeek expert parallel path with MoE ops
  pggPL
  committed
  0343fab
Commits on Oct 5, 2026
- Let MoE EP ops allocate their data buffers
  pggPL
  committed
  72abef7
Commits on Oct 6, 2026
- Fix DeepSeek sanity coverage and grouped FP8 requirements
  pggPL
  committed
  b14c09f
Commits on Oct 7, 2026
- Remove kernel error handling changes from the DeepSeek PR
  pggPL
  committed
  21781c3

Commits

Commits on Aug 18, 2026

Commits on Aug 21, 2026

Commits on Sep 3, 2026

Commits on Sep 8, 2026

Commits on Sep 9, 2026

Commits on Sep 10, 2026

Commits on Sep 17, 2026

Commits on Sep 28, 2026

Commits on Sep 30, 2026

Commits on Oct 1, 2026

Commits on Oct 5, 2026

Commits on Oct 6, 2026

Commits on Oct 7, 2026