On-Chip Accelerators | Future Architecture and System Technology for Scalable Computing

On-Chip Accelerators

Intel has integrated diverse on-chip accelerators into its recent flagship datacenter CPU products. Our group has conducted in-depth investigations four of these cutting-edge accelerators, Data Streaming Accelerator (DSA), Dynamic Load Balancer (DLB), In-memory Analytics Accelerator (IAA), and QuickAssist Technology (QAT). We are the first research group to offer comprehensive guidelines, practical use cases, and distinctive enhancements to maximize the effectiveness and performance of these accelerators.

The XCC version of the SPR SoC chiplet with a Data Accelerator Complex (DAC) tile.

Modern workloads are rapidly outpacing the gains delivered by simply adding more CPU cores or cranking up clock speeds. Modern workloads also place increased demands on compute, network and storage resources. As a result, there is a trend to deploy and integrate energy-efficient accelerators—like AI chips, security processors, HPC units, and network or storage offload engines—for offloading specialized tasks. That way, system can achieve significant energy savings, while preserving general-purpose cores for more versatile workloads. Intel has integrated the broadest set of built-in accelerators in 4th Gen Intel® Xeon® Scalable processors to boost performance, reduce latency and increase power efficiency. Furthermore, to support the easy and efficient use of these accelerators for successful deployment in production hyperscale services, Intel has developed a hardware/software ecosystem.

Data Streaming Accelerator (DSA)

We present the newest features of Data Streaming Accelerator (DSA), explore its versatility, and quantify its throughput gains through microbenchmarks and real-world use cases. From these results and DSA’s rich software ecosystem, we summarize several insights and guidelines for the programmer to make the most out of DSA, and use an in-depth case study of DPDK Vhost to demonstrate how these guidelines benefits a real application (Kuper et al., 2024 ; Ji et al., 2025 ).

Dynamic Load Balancer (DLB)

Uneven CPU core utilization create excessive queuing delays and latency variance in modern networks. Intel Dynamic Load Balancer (DLB) uses hardware queues, arbiters, and priority-based QoS to outperform software load balancers, but still relies on host CPU cycles to prepare and enqueue work descriptors. To fully offload this overhead, we introduce AccDirect, which leverages PCIe peer-to-peer communication so that the NIC can enqueue descriptors directly into DLB without touching the host CPU (Lou et al., 2025 ).

In-Memory Analytics Accelerator (IAA)

We present the Intel In-Memory Analytics Accelerator (IAA), an on-chip accelerator integrated into Intel's 4th Gen Xeon Scalable Processors. Our work introduces the hardware and software architecture of IAA, highlights its unique features, and offers practical guidelines for effectively leveraging IAA in analytic applications based on our evaluations (Kang et al., 2025 ).

QuickAssist Technology (QAT)

Intel QuickAssist Technology (QAT) offloads data compression and decompression for datacenter workloads. We thoroughly benchmark the latest on-chip QAT against its previous off-chip generation, revealing significant improvements in throughput and latency. Additionally, by re-implementing Linux’s zswap with QAT, we demonstrate marked performance gains of QAT-based zswap over the traditional CPU-based implementation (Xia† et al., 2025 ).

More technical refences can be found in our ISCA 2023 tutorial: On-chip Accelerators in 4th Gen Intel® Xeon® Scalable Processors: Features, Performance, Use Cases, and Future!

References

2025

CAL

Cooperative Memory Deduplication with Intel Data Streaming Accelerator

Houxiang Ji, Minho Kim, Seonmu Oh, Daehoon Kim, and Nam Sung Kim

IEEE Computer Architecture Letters , 2025

Accelerator System

Abs DOI

Memory deduplication plays a critical role in reducing memory consumption and the total cost of ownership (TCO) in hyperscalers, particularly as the advent of large language models imposes unprecedented demands on memory resources. However, conventional CPU-based memory deduplication can interfere with co-running applications, significantly impacting the performance of time-sensitive workloads. Intel introduced the on-chip Data Streaming Accelerator (DSA), providing high-performance data movement and transformation capabilities, including comparison and checksum calculation, which are heavily utilized in the deduplication. In this work, we enhance a widely-used kernel-space memory deduplication feature, Kernel Samepage Merging (ksm), by selectively offloading these operations to the DSA. Our evaluation demonstrates that CPU-based ksm can lead to 5.0–10.9× increase in the tail latency of co-running applications while DSA-based ksm limits the latency increase to just 1.6× while achieving comparable memory savings.
ISCA

Dynamic Load Balancer in Intel® Xeon® Scalable Processor: Performance analyses, enhancements, and guidelines

Jiaqi Lou, Srikar Vanavasam, Yifan Yuan, Ren Wang, and Nam Sung Kim

IEEE/ACM International Symposium on Computer Architecture (ISCA) , 2025
ISPASS

Intel In-Memory Analytics Accelerator: performance characterization and guidelines

Jaeyoung Kang, Qirong Xia, Ipoom Jeong, Yongjoo Park, and Nam Sung Kim

IEEE International Symposium on Performance Analysis of Systems and Software (ISPASS) , 2025

Best Paper Honorable Mention and Distinguished Artifact Award

Architecture Accelerator
CAL

Hardware-accelerated kernel-space memory compression using Intel QAT

Qirong Xia^†, Houxiang Ji^†, Yang Zhou, and Nam Sung Kim

Jun 2025

System Accelerator

2024

ASPLOS

A quantitative analysis and guidelines of data streaming accelerator in modern intel xeon scalable processors

Reese Kuper, Ipoom Jeong, Yifan Yuan, Ren Wang, Narayan Ranganathan, Nikhil Rao, Jiayu Hu, Sanjay Kumar, Philip Lantz, and Nam Sung Kim

29th ACM International Conference on Architectural Support for Programming Languages and Operating Systems , Jun 2024

Architecture CPU Accelerator

Abs DOI

As semiconductor power density is no longer constant with the technology process scaling down, we need different solutions if we are to continue scaling application performance. To this end, modern CPUs are integrating capable data accelerators on the chip, aiming to improve performance and efficiency for a wide range of applications and usages. One such accelerator is the Intel® Data Streaming Accelerator (DSA) introduced since Intel® 4th Generation Xeon® Scalable CPUs (Sapphire Rapids). DSA targets data movement operations in memory that are common sources of overhead in datacenter workloads and infrastructure. In addition, it supports a wider range of operations on streaming data, such as CRC32 calculations, computation of deltas between data buffers, and data integrity field (DIF) operations. This paper aims to introduce the latest features supported by DSA, dive deep into its versatility, and analyze its throughput benefits through a comprehensive evaluation with both microbenchmarks and real use cases. Along with the analysis of its characteristics and the rich software ecosystem of DSA, we summarize several insights and guidelines for the programmer to make the most out of DSA, and use an in-depth case study of DPDK Vhost to demonstrate how these guidelines benefit a real application.