Skip to content

glm47.md

Latest commit

 

History

History
28 lines (18 loc) · 1.88 KB

glm47.md

File metadata and controls

28 lines (18 loc) · 1.88 KB
Raw

GLM-4.7 and GLM-4.7-Flash

GLM-4.7 and GLM-4.7-Flash are GLM MoE language models from Zhipu AI. Megatron Bridge supports both checkpoints, with the full GLM-4.7 model handled by the GLM-4.5 bridge registration and the Flash variant handled by a dedicated GLM-4.7-Flash bridge.

Supported Variants

Variant Hugging Face ID Bridge Notes
GLM-4.7 zai-org/GLM-4.7 GLM45Bridge Full MoE model, about 358B total parameters and 32B active
GLM-4.7-Flash zai-org/GLM-4.7-Flash GLM47FlashBridge MLA + MoE model, about 30B total parameters and 3B active

Architecture Notes

  • GLM-4.7 uses the glm4_moe architecture already covered by GLM45Bridge.
  • GLM-4.7-Flash uses glm4_moe_lite, Multi-Latent Attention, GLM-style MoE routing, and expert-bias routing.
  • GLM-4.7-Flash stores per-expert gate/up/down tensors in Hugging Face checkpoint format; the bridge maps them into Megatron MoE tensors.
  • transformers >= 5.0.0rc0 is required for the GLM-4.7 Hugging Face model classes used by these bridges.

Examples

For checkpoint conversion, inference, hardware notes, and validated parallelism settings, see the GLM-4.7 examples README.

Related Implementation