독서리서치레이더투자 프레임워크
로그인 / 회원가입
← 읽기로 돌아가기
뉴스주제 · NCCL对称内存与GIN간단한 요점1 분 소요DisconfirmAI 편집부

NVIDIA Engineers Explain NCCL Symmetric Memory and GPU-Initiated Networking

NVIDIA engineers Jeff Hammond, Hanyue He, and Ke Wen published a tutorial on the PyTorch official channel introducing new NCCL features: symmetric memory and GPU-initiated networking (GIN), and their application in MoE implementations.

2026-09-25 04:35 GMT+8

Three NVIDIA engineers, Jeff Hammond, Hanyue He, and Ke Wen, published a tutorial on advanced NCCL features on the official PyTorch channel. The video, released on September 24, 2026, primarily explains two newly introduced functions in NCCL: symmetric memory and GPU-initiated networking (GIN).

According to the publisher's description, these new features are used in state-of-the-art Mixture-of-Experts (MoE) implementations such as DeepEP v2. The tutorial assumes viewers already know how to initialize NCCL and perform common host-initiated operations. It covers how to scale communication within an NVLink domain using symmetric memory and how to scale across interconnected GPU nodes using GIN. All software materials are open source, and content is available in both English and Chinese.

출처

  • youtube.com↗
주제

NCCL对称内存与GIN

【背景】在分布式AI训练中,传统通信由CPU协调,导致延迟高且难以实现计算与通信融合。为解决此问题,NVIDIA在NCCL库中引入了对称内存(Symmetric Memory)和GPU发起的网络(GIN, GPU-Initiated Net…

5 단계2 건업데이트됨 2026-11-04 20:09

로그인 / 회원가입
독서리서치레이더투자 프레임워크

현재까지의 과정

  1. 2025-11-24NVIDIA发布arXiv论文《GPU-Initiated Networking for NCCL》,正式介绍GIN架构及其在MoE通信中的应用。arxiv.org ↗
  2. 2026-04-01在PyTorch Conference Europe上,Ke Wen和Sylvain Jeaugey展示PyTorch对称内存与NCCL设备API的结合方案。hosted-files.sched.co ↗
  3. 2026-07-01GitHub用户d4l3k提交PR #3932,为PyTorch官方教程添加关于NCCL GIN和对称内存的使用指南。github.com ↗

아직 해결되지 않은 문제

  • PyTorch 2.15中NCCL后端成为默认选项后,对现有依赖旧版NCCL行为的项目有多大兼容性影响?
  • GIN的GDAKI后端在非NVIDIA专有NIC或特定Linux内核版本下的实际部署门槛有多高?
  • 除了MoE场景,GIN在常规All-Reduce操作中相比传统Host-initiated模式的具体性能增益数据是多少?

이 스레드의 다른 내용

  • PyTorch Official Channel Announces Live Q&A for Version 2.15: Meta Team to Explain NCCL Defaultization and NVFP4/FP8 Optimizations2026-10-29
전체 주제 읽기 →
리서치 계속하기 →