NVIDIA Engineers Detail Acceleration of Sequential Recommender Models in PyTorch: SASRec and HSTU GPU Optimization Practices

NVIDIA engineers Zan Huang and Jiayu Sun released a technical presentation titled "Accelerating Sequential Recommender Models in the PyTorch Ecosystem" on the official PyTorch YouTube channel on September 24, 2026. The video focuses on the natural connection between sequential recommendation models and Large Language Models (LLMs) regarding self-attention usage, as well as the unique challenges introduced by production recommendation systems.

According to the publisher's summary, the talk begins with SASRec, one of the earliest Transformer decoder-based sequential recommenders, walking through its rewrite from TensorFlow to PyTorch. This section highlights how PyTorch reduced modeling complexity and surfaced gaps between academic prototypes and production systems. The presentation then examines HSTU, an attention operator for sequential recommendation at scale, analyzing what makes its attention pattern structurally different from standard Transformer attention. Key GPU insights shared include kernel fusion, variable-length sequences, and PyTorch integration via FBGEMM. Finally, the speakers reflect on sequential recommendation and LLMs from both modeling and systems perspectives, sharing observations on emerging trends and optimization strategies across recommendation workloads.