AMD unveils new AI accelerators & network solutions

AMD has announced its latest range of accelerator and networking solutions intended to support future AI infrastructure: the AMD Instinct MI325X accelerators, the AMD Pensando Pollara 400 NIC, and the AMD Pensando Salina DPU.

The AMD Instinct MI325X accelerators, built on the AMD CDNA 3 architecture, are engineered to offer exceptional performance and efficiency for various demanding AI tasks, including foundation model training, fine-tuning, and inferencing. These accelerators are anticipated to enable customers and partners to create highly efficient and optimally performing AI solutions across system, rack, and data centre levels.

Forrest Norrod, Executive Vice President and General Manager, Data Center Solutions Business Group at AMD, remarked, "AMD continues to deliver on our roadmap, offering customers the performance they need and the choice they want, to bring AI infrastructure, at scale, to market faster. With the new AMD Instinct accelerators, EPYC processors and AMD Pensando networking engines, the continued growth of our open software ecosystem, and the ability to tie this all together into optimized AI infrastructure, AMD underscores the critical expertise to build and deploy world class AI solutions."

The AMD Instinct MI325X features 256GB of HBM3E memory with a bandwidth of 6.0TB/s, providing 1.8 times the capacity and 1.3 times the bandwidth compared to previous models. This substantial memory and compute power can enhance inference performance on models like Mistral 7B, Llama 3.1 70B, and Mixtral 8x7B.

The company plans to begin production shipments of the AMD Instinct MI325X accelerators in the fourth quarter of 2024, with a broader system availability expected in the first quarter of 2025 from major platform providers such as Dell Technologies, Hewlett Packard Enterprise, and others.

A preview of the upcoming AMD Instinct MI350 series accelerators was also shared, promising a 35-times improvement in inference performance compared to the current CDNA 3-based models. The MI350 series aims to maintain memory capacity leadership with up to 288GB of HBM3E memory per accelerator and is expected to be available in the second half of 2025.

The AMD Pensando Salina DPU and the Pensando Pollara 400 NIC target AI networking needs. These solutions address the challenges in front-end and back-end network management for AI clusters, aiming to optimise performance and scalability throughout AI systems. The Salina DPU, the third generation of its kind, offers up to double the performance of its predecessor, supporting 400G throughput.

The Pensando Pollara 400 NIC, renowned as the industry's first UEC-ready AI NIC, supports emerging RDMA software and is designed to enhance accelerator-to-accelerator communication efficiency in back-end networks. Both the Salina DPU and the Pollara 400 NIC are expected to be available in the first half of 2025.

AMD continues to enhance its open software capabilities through its ROCm open software stack to support generative AI applications and models. By supporting popular AI frameworks such as PyTorch and collaborating with platforms like Hugging Face, AMD seeks to deliver improved performance and support for its Instinct accelerators. ROCm 6.2 has introduced new features aimed at advancing performance on inference and training for large language models.