読書リサーチレーダー投資フレームワーク
ログイン / 新規登録
← 読解に戻る
ニュース重大2 分で読めるDisconfirmAI 編集部

AWS Launches SageMaker HyperPod Global Inference Gateway for Cross-Region AI Routing

AWS introduces a managed gateway to route AI inference across clusters and regions, optimizing GPU utilization and automating failover.

2026-11-21 06:35 GMT+8

Amazon Web Services (AWS) has introduced the Amazon SageMaker HyperPod Global Inference Gateway, a new managed service designed to route artificial intelligence inference requests across multiple compute clusters and AWS Regions through a single endpoint. This addresses a critical infrastructure challenge where GPU capacity is fragmented across different locations, leading to idle resources in one region while requests queue in another.

The gateway operates as a second tier in the HyperPod architecture. While the existing Inference Gateway routes requests to specific pods within a single cluster, the Global Inference Gateway selects the best-suited cluster from all registered clusters. It uses four weighted signals to make routing decisions: KV cache headroom (available memory), queue depth (current load), region proximity (latency), and error rate (health). By default, the system prioritizes proximity but can be configured to favor resilience by increasing the weight of error rates.

For enterprises, this update eliminates the need to build custom cross-region networking, load balancers, and DNS management. Traffic is carried over the AWS backbone using AWS PrivateLink, ensuring privacy and reducing latency compared to public internet paths. Automatic failover moves traffic away from failed clusters without requiring client-side configuration changes. Authentication is handled via AWS Signature Version 4 (SigV4) and Identity and Access Management (IAM).

Jinhyung Park, VP of AI Platform at Bagelcode, noted that the beta version allowed their small platform team to achieve cross-region failover without maintaining their own routing layer. The feature is available now for users with multiple SageMaker HyperPod clusters.

ログイン / 新規登録
読書リサーチレーダー投資フレームワーク

ソース

  • aws.amazon.comAmazon Web Services↗
リサーチを続ける →