SageMaker Inference now supports Multi-Instance GPU to maximize GPU utilization for model endpoints

AWS Machine LearningNovember 6, 2026en

Amazon SageMaker Inference now supports GPU partitioning with NVIDIA Multi-Instance GPU (MIG), which divides one GPU into hardware-isolated partitions so you can run a different model in each or take a right-sized fraction of a newer accelerator. This post shows how two optional API fields place one model per partition, and what we measured against a whole older GPU.

This is a short summary published by AI Global Wire. The full article is owned and hosted by AWS Machine Learning — open it there to read it in full.