AWS ML BlogWednesday · June 3, 2026FREEAccelerate LLM model loading and increase context windows with GPUDirect on Amazon FSx for Lustre and TurboQuant
AWS announced GPUDirect support for Amazon FSx for Lustre, a high-performance file system, in combination with TurboQuant, a quantization technique. This integration allows GPU instances to directly access data from FSx for Lustre without intermediate CPU buffering, reducing latency and memory overhead. The feature is available now for supported GPU instance types (e.g., p4d, p5) and FSx for Lustre file systems. TurboQuant, which uses 4-bit quantization, further reduces model size and memory footprint. Together, they cut LLM model loading time by up to 60% and enable larger context windows by freeing GPU memory previously used for data staging. No additional cost for GPUDirect support beyond standard FSx for Lustre and GPU instance pricing. This is particularly beneficial for large models like Llama 3.1 405B, where loading time can drop from minutes to under a minute.
Faster model loading and larger context windows reduce inference latency and enable more complex AI workloads.