DeepEP

lastmod 2025-02-26created 2025-02-261 edits

an efficient expert-parallel communication library

Features

• High-performance communication library for Mixture-of-Experts (MoE) and Expert Parallelism (EP)
• Provides GPU-based All-to-All kernels to process MoE dispatch and combine operations at high speed
• Supports low-precision operations such as FP8
• Applies the group-limited gating algorithm proposed in the DeepSeek-V3 paper to optimize asymmetric-domain bandwidth forwarding
  • e.g., optimizing NVLink → RDMA data transfer
  • Provides high throughput suited to training and inference prefilling tasks
• Includes RDMA-only low-latency kernels for latency-sensitive inference decoding
• Provides a communication-computation overlap technique (does not occupy SM resources)

See also

• DeepSeek

Favorite site

• DeepSeek open-sources DeepEP (2 of 5) | GeekNews