AI engine
Computational core for AI inference and signal processing.
An AI engine is the core computational component of an AI system, primarily handling AI inference. The term also describes a specific architecture from AMD, which originated with Xilinx before that company was acquired by AMD in 2022. This architecture uses software layers to run AI workloads, manage memory, and handle scheduling, which makes application development simpler. It is often used to speed up linear algebra operations, AI algorithms, digital signal processing, and general high-performance computing.
The architecture was first released by Xilinx, a company focused on field-programmable gate arrays (FPGAs). Its initial aim was to accelerate signal processing and other tasks where data parallelism could provide major gains. The first products to include AI engines were the Versal adaptive compute acceleration platforms, which combine scalar, adaptable, and intelligent engines connected through a Network on Chip (NoC). The initial Versal systems, the VCK190 and VCK5000, each contained 400 AI engines in their AI engine layer, linked via a VC1902 device.
In 2022, after AMD acquired Xilinx, the AI engine project shifted. AMD integrated the engines with other computing systems to reach more applications, especially AI workloads. Although the Versal architecture was powerful, it was complex and unfamiliar to many in academia and industry. To address this, AMD and third-party developers released improved toolsets and software stacks to simplify programming and boost productivity.
In 2023, AMD announced the second generation of the architecture, called AI engine ML (AIE-ML). This version added support for AI-specific data types like bfloat16, which is common in deep learning. It kept the same vector processing capabilities but had larger memory for intermediate computations. Starting with this generation, AMD integrated AI engines with other processing units like CPUs and GPUs, including in modern Ryzen AI processors. In these systems, AI engines are often called Compute Tiles—self-contained blocks that efficiently run AI and signal processing workloads. These blocks are combined with Memory tiles and Shim tiles. The overall structure of these three interconnected tile types is named XDNA. The first generation, XDNA 1, debuted on Ryzen AI Phoenix PCs. Alongside this release, AMD continued its focus on programmability by releasing the open-source tool Rial
- origin
- Xilinx, Inc. (later AMD)
- first release
- Versal adaptive compute acceleration platforms
- field
- Artificial intelligence hardware, digital signal processing, high-performance computing
- known for
- AI inference acceleration, SIMD vector processing, integration with FPGAs, CPUs, and GPUs
- generations
- First generation (AI engine), Second generation (AI engine ML / AIE-ML)
Lore & Background
The architecture was originally released by Xilinx, Inc., a company specializing in field-programmable gate arrays (FPGAs). Their initial goal was to accelerate signal processing and applications where data parallelism could offer significant improvements. Initially, AI engines were released combined with an FPGA layer in the Versal platforms. The initial systems, the VCK190 and VCK5000, contained 400 AI engines in their AI engine layer, connected through a VC1902. For connectivity, this architecture class relied on a Network on Chip (NoC) for high-performance on-chip connectivity. In 2022, the AI engine project changed when Xilinx was officially acquired by AMD. The AI engines were integrated with other computing systems to target a wider range of applications, finding benefits when considering AI workloads.
Reader's Guide
The AI engine architecture represents a significant development in specialized hardware for AI inference and signal processing. Originally created by Xilinx to accelerate data-parallel applications, the architecture evolved after AMD's acquisition to target a wider range of AI workloads. The first generation combined 400 AI engines in Versal platforms, each a 7-way VLIW processor with vector and scalar capabilities. The second generation, AIE-ML, improved performance for machine learning by adding native support for INT8 and bfloat16 data types, delivering up to three times more TOPS per watt than the first generation. The XDNA architecture, combining Compute, Memory, and Shim tiles, has been integrated into consumer Ryzen AI processors, marking a shift from specialized FPGA-based systems to broader computing platforms. Researchers and industry professionals increasingly categorize XDNA architectures as Neural processing units (NPUs), though the term includes architectures from other companies like Huawei and Tesla. AMD has released open-source tools like Riallto and the Ryzen AI SW toolchain to improve programmability, addressing the complexity that initially limited adoption among academic and industrial communities.
Did You Know?
- According to AMD, the 'AI' in 'AI engine' is not an acronym for artificial intelligence or any other term.
- A single AI engine is a 7-way VLIW processor with a 128-bit wide vector unit and a maximum vector register size of 1024-bit.
- The first generation of Versal systems combined 400 AI engines together, each with 32 KB of memory extendable to 128 KB using neighboring engines.
- The XDNA architecture's first generation, XDNA 1, was released on Ryzen AI Phoenix PCs.
More in American inventions 1-24
Spotted an error? Know more?
This is a living reference — every entry is fact-audited, and reader corrections feed straight into our audit queue. Suggest an edit · See this site's audit record