LesenForschungRadarAnlageframework
Anmelden / Registrieren
← Zurück zur Lektüre
NewsThema · NCCL对称内存与GINWesentlich2 Min. LesezeitDisconfirmAI Redaktion

PyTorch Official Channel Announces Live Q&A for Version 2.15: Meta Team to Explain NCCL Defaultization and NVFP4/FP8 Optimizations

The official PyTorch channel published a live Q&A video preview for version 2.15 on October 28, 2026. This version includes 3,117 commits since 2.14, with key updates such as the rewritten NCCL backend becoming the default, NVFP4 low-precision decoding optimizations on NVIDIA Blackwell GPUs, and FP8 support on Apple Silicon. The Q&A session scheduled for November 5, 2026, will be led by Tristan Rice, Natalia Gimelshein, Huy Do, and Jane Xu from Meta, moderated by Chris Gottbrath. For engineers focused on AI infrastructure, this resource offers direct insight into new features and rollback paths. As no verified transcript was obtained, readers are recommended to watch the video directly.

2026-10-28 23:12 GMT+8

The official PyTorch channel released a live Q&A video preview for PyTorch version 2.15 on October 28, 2026. According to the publisher's information, PyTorch 2.15 comprises 3,117 commits from 583 contributors since version 2.14. Core changes in this update include making the rewritten NCCL backend the default option, optimizing 4-bit (NVFP4) LLM decoding performance on NVIDIA Blackwell GPUs via NVGEMM, and adding FP8 support and faster decode kernels on Apple Silicon.

In the upcoming Q&A session scheduled for 11 AM PT on November 5, 2026, four core contributors from Meta—Tristan Rice, Natalia Gimelshein, Huy Do, and Jane Xu—will provide an overview of the 2.15 release and answer community questions about PyTorch and its new capabilities. Chris Gottbrath from Gottbrath Tech will moderate the session. This arrangement indicates that the resource is not just a routine introduction but a channel for first-line maintainers to address technical queries directly.

For readers involved in AI infrastructure development, this update involves changes to several critical underlying mechanisms. For instance, the defaultization of the NCCL backend introduces a flight recorder that dumps data on hangs, with TORCH_DIST_USE_NCCL2=0 or nccl-legacy provided as rollback paths; torch.distributed adds support for one-sided transports, including a NIXL backend and CUDA-graph-capturable transfers; and FSDP2 and pipelining implement finer memory and communication control. Additionally, FlyDSL kernels on AMD gfx950, simplified SYCL kernel authoring on Intel GPUs, and torch.compile support on Python 3.15 expand platform compatibility.

Anmelden / Registrieren
LesenForschungRadarAnlageframework

Given that only the episode description and topic list have been obtained, without a complete dialogue transcript, this recommendation aims to guide readers to this authoritative technical Q&A resource. Engineers can watch the video to hear the Meta team explain specific changes such as NCCL defaultization, NVFP4 decoding optimization, and Apple Silicon FP8 support, as well as how they handle potential rollback and compatibility issues.

Quellen

  • youtube.com↗
Thema

NCCL对称内存与GIN

【背景】在分布式AI训练中,传统通信由CPU协调,导致延迟高且难以实现计算与通信融合。为解决此问题,NVIDIA在NCCL库中引入了对称内存(Symmetric Memory)和GPU发起的网络(GIN, GPU-Initiated Net…

5 Schritte2 BeiträgeAktualisiert 2026-11-04 20:09

Wie es dazu kam

  1. 2025-11-24NVIDIA发布arXiv论文《GPU-Initiated Networking for NCCL》,正式介绍GIN架构及其在MoE通信中的应用。arxiv.org ↗
  2. 2026-04-01在PyTorch Conference Europe上,Ke Wen和Sylvain Jeaugey展示PyTorch对称内存与NCCL设备API的结合方案。hosted-files.sched.co ↗
  3. 2026-07-01GitHub用户d4l3k提交PR #3932,为PyTorch官方教程添加关于NCCL GIN和对称内存的使用指南。github.com ↗

Was noch ungeklärt ist

  • PyTorch 2.15中NCCL后端成为默认选项后,对现有依赖旧版NCCL行为的项目有多大兼容性影响?
  • GIN的GDAKI后端在非NVIDIA专有NIC或特定Linux内核版本下的实际部署门槛有多高?
  • 除了MoE场景,GIN在常规All-Reduce操作中相比传统Host-initiated模式的具体性能增益数据是多少?

Auch in diesem Thread

  • NVIDIA Engineers Explain NCCL Symmetric Memory and GPU-Initiated Networking2026-10-14
Vollständiges Thema lesen →
Recherche fortsetzen →