Publications

2026

1. MICRO

   CXL AnySSD: A composable CXL SSD using a CXL Type-2 device and any SSDs

   Yang Zhou, Houxiang Ji, Bikrant Sharma, Jiyuan Zhang, Yu Li, Linjie Ma, Jaeyong Lee, Myoungjun Chun, Jihong Kim, Sudarsun Kannan, and Nam Sung Kim

   IEEE/ACM International Symposium on Microarchitecture (MICRO), Oct 2026
2. MICRO

   AMG: AMX-GPU cooperative acceleration of billion-scale ANNS via GEMM reformulation

   Minho Kim, Houxiang Ji, Hwanjun Lee, Jaeyoung Kang, Sungju Kim, Chunkyun Shin, Daehoon Kim, and Nam Sung Kim

   IEEE/ACM International Symposium on Microarchitecture (MICRO), Oct 2026
3. MICRO

   Elastic LLM serving on FPGAs via partial reconfiguration: a case study using the Altera FPGA AI Suite

   Nachuan Wang, Mike Montano, Hyungyo Kim, Ren Wang, Phillip Swart, and Nam Sung Kim

   IEEE/ACM International Symposium on Microarchitecture (MICRO), Oct 2026
4. MICRO

   NiF: hybrid in-flash and in-storage acceleration for approximate nearest neighbor search

   Myoungjun Chun, Jaeyong Lee, Gwuiin Kim, Minhyeong Kim, Inhyuk Choi, Myungsuk Kim, Nam Sung Kim, and Jihong Kim

   IEEE/ACM International Symposium on Microarchitecture (MICRO), Oct 2026
5. MICRO

   ARC: adaptive reconfigurable CXL hotness monitoring

   Chihun Song, Hwayong Nam, Haijian Wang, Eojin Na, Haneul Park, Taehyung Lee, Jung Ho Ahn, and Nam Sung Kim

   IEEE/ACM International Symposium on Microarchitecture (MICRO), Oct 2026
6. MICRO

   Rethinking compression for CXL memory expanders at hyperscale

   Haneul Park, Grant Ayers^*, Nam Sung Kim^*, Philip Levis^*, and Brian Morris^*

   IEEE/ACM International Symposium on Microarchitecture (MICRO), Oct 2026

   *The alphabetical order
7. SOSP

   Borges: A low-latency distributed shared log on a CXL memory/SSD hybrid

   Haowei Chen, Yiming Xiang, Zhipeng Jia, Yan Sun, Nam Sung Kim, and Emmett Witchel

   ACM Symposium on Operating Systems Principles (SOSP), Sep 2026
8. CAL

   HyGIN: Hybrid CPU/GPU-initiated communication for mixture-of-experts training

   Jinghan Huang and Nam Sung Kim

   July–December 2026
9. TOCS

   Tigon: A distributed database for a CXL pod

   Yibo Huang, Haowei Chen, Newton Ni, Yan Sun, Nam Sung Kim, Vijay Chidambaram, Dixin Tang, and Emmett Witchel

   ACM Transactions on Computer Systems (TOCS), to appear, 2026
10. OSDI

    MAC: Metadata acceleration for sustainable performance in big-data systems with CXL DRAM

    Dusol Lee, Yan Sun, Houxiang Ji, Vinit Gupta, Austin Antony Cruz, Inhyuk Choi, Nam Sung Kim, and Jihong Kim

    USENIX Symposium on Operating Systems Design and Implementation (OSDI), Jul 2026
11. HCS

    XCENA MX1 CXL computational memory device

    Harry Kim, Jinin So, Dohun Kim, Sungwoo Chang, Wonjae Lee, Seunghak Lee, Hojin Nam, Jinoh Ahn, Grant Mackey, Brian Hirano, and Nam Sung Kim

    IEEE Hot Chips Symposium (HCS) , Aug 2026
12. CAL

    A Simulator for LLM inference systems exploiting CXL memory pools

    Jinghan Huang, Hongkun Zeng, Mike Montano, Jaehong Cho, Hyunmin Choi, Jinin So, Junhyeok Im, Handeok Lee, Jongse Park, and Nam Sung Kim

    July–December 2026
13. CAL

    KiF: Accelerating low-batch LLM inference using in-flash KV cache

    Haeeun Jung, Jaeyong Lee, Sanggu Lee, Huiwon Yun, Dongsuk Jeon, Nam Sung Kim, and Jihong Kim

    July–December 2026
14. ISCA

    PVAC: A RowHammer mitigation architecture exploiting per-victim-row counting

    Jumin Kim, Seungmin Baek, Hwayong Nam, Minbok Wi, Nam Sung Kim, and Jung Ho Ahn

    IEEE/ACM International Symposium on Computer Architecture (ISCA) , Jun 2026
15. CAL

    MMC: Metadata migration for efficient memory management in CXL DRAM systems

    Dusol Lee, Taehyung Lee, Hanseon Lee, Nam Sung Kim, and Jihong Kim

    July–December 2026
16. CAL

    Characterizing system-level trade-offs of Intel DSA for user-space IPC offloading

    Misun Park, Richi Dubey, Yifan Yuan, Nam Sung Kim, and Ada Gavrilovska

    July–December 2026
17. CAL

    Capacity-latency tradeoffs in CXL memory expander at hyperscale

    Haneul Park, Grant Ayers, Nam Sung Kim, Philip Levis, and Brian Morris

    July–December 2026
18. CAL

    CXL-Tracer: Accurate, full-system memory tracing on commercial hardware via CXL memory

    Srikar Reddy Vanavasam, Austin Antony Cruz, and Nam Sung Kim

    July–December 2026
19. ISPASS

    Compiler and system optimizations for gem5 simulator

    Haneul Park^†, Siddharth Agarwal^†, Pradyun Narkadamilli, Kiung Jung, Yongjun Park, Ipoom Jeong, and Nam Sung Kim

    IEEE International Symposium on Performance Analysis of System and Software (ISPASS) , Apr 2026

    Best Paper Nomination

    
20. HPCA

    ReScue: Reliable and Secure CXL memory

    Chihun Song, Austin Antony Cruz, Michael Jaemin Kim, Minbok Wi, Gaohan Ye, Kyungsan Kim, Sangyeol Lee, Jung Ho Ahn, and Nam Sung Kim

    IEEE International Symposium on High-Performance Computer Architecture , Jan 2026

    
21. HPCA

    LILo: Harnessing the on-chip accelerators in Intel CPUs for compressed LLM inference acceleration

    Hyungyo Kim^†, Qirong Xia^†, Jinghan Huang, Nachuan Wang, Jung Ho Ahn, Younjoo Lee, Wajdi K Feghali, Ren Wang, and Nam Sung Kim

    IEEE International Symposium on High-Performance Computer Architecture , Jan 2026

    
22. HPCA

    MemSOS: OS-guided selective memory mirroring

    Junghoon Kim, Jongheon Jeong, Seokwon Moon, Seong Hoon Seo, Yeonhong Park, Jinkyu Jeong, Nam Sung Kim, and Jae W. Lee

    IEEE International Symposium on High-Performance Computer Architecture , Jan 2026
23. TCAS

    Characterizing the intrinsic bank-level accuracy vs. energy trade-off of SRAM-based analog in-memory computing architectures in 28 nm CMOS

    Shuo Li, Chihun Song, Hyungyo Kim, Nam Sung Kim, and Naresh R. Shanbhag

    IEEE Transactions on Circuits and Systems I: Regular Papers , 2026
24. S&P

    SoK: Systematizing a decade of architectural RowHammer defenses through the lens of streaming algorithms

    Michael Jaemin Kim, Seungmin Baek, Jumin Kim, Hwayong Nam, Nam Sung Kim, and Jung Ho Ahn

    IEEE Symposium on Security and Privacy (S&P) , Mar 2026
25. ASPLOS

    TiNA: Tiered network buffer architecture for fast networking in chiplet-based CPU

    Siddharth Agarwal^†, Tianchen Wang^†, Jinghan Huang, Saksham Agarwal, and Nam Sung Kim

    ACM International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS) , Apr 2026

    

2025

1. EPEPS

   Temperature-Dependent SPICE Models for UCIe Interconnects

   Ram Krishna, Ashita Victor, Srujan Penta, Atom Watanabe, Xu Chen, Muhannad S. Bakir, Nam Sung Kim, and Elyse Rosenbaum

   IEEE Conference on Electrical Performance of Electronics Packaging and Systems , Oct 2025
2. MICRO

   NetZIP: algorithm/hardware co-design of in-network lossless compression for distributed large model

   Jinghan Huang, Hyungyo Kim, Nachuan Wang, Jaeyoung Kang, Hrishi Shah, Eun Kyung Lee, Minjia Zhang, Fan Lai, and Nam Sung Kim

   2025 58th IEEE/ACM International Symposium on Microarchitecture (MICRO) , Oct 2025

   
3. MICRO

   Re-architecting end-host networking with CXL: coherence, memory, and offloading

   Houxiang Ji, Yifan Yuan, Yang Zhou, Ipoom Jeong, Ren Wang, Saksham Agarwal, and Nam Sung Kim

   2025 58th IEEE/ACM International Symposium on Microarchitecture (MICRO) , Oct 2025

   
4. MICRO

   DRAM fault classification through large-scale field monitoring for robust memory RAS management

   Hoeju Chung, Euisang Oh, Seungmin Baek, Hyeongshin Yoon, Jaesung Yoo, Sanghwan Lee, Yongjun Lee, Arhatha Bramhanand, Brett Dodds, Yang Zhou, and Nam Sung Kim

   2025 58th IEEE/ACM International Symposium on Microarchitecture (MICRO) , Oct 2025

   Best Paper Finalist
5. MICRO

   Stratum: system-hardware co-design with tiered monolithic 3D-DRAM for efficient MoE serving

   Yue Pan, Zihan Xia, Po-Kai Hsu, Lanxiang Hu, Hyungyo Kim, Janak Sharda, Minxuan Zhou, Nam Sung Kim, Shimeng Yu, Tajana Rosing, and Mingu Kang

   2025 58th IEEE/ACM International Symposium on Microarchitecture (MICRO) , Oct 2025
6. CAL

   CABANA: Cluster-aware query batching for accelerating billion-scale ANNS with Intel AMX

   Minho Kim^†, Houxiang Ji^†, Jaeyoung Kang, Hwanjun Lee, Daehoon Kim, and Nam Sung Kim

   Dec 2025
7. CAL

   HINT: A hardware platform for intra-host NIC traffic and SmartNIC emulation

   Jiaqi Lou^†, Yu Li^†, Srikar Vanavasam, and Nam Sung Kim

   Dec 2025
8. CAL

   Time series machine learning models for precise SSD access latency prediction

   Bikrant Das Sharma, Houxiang Ji, Ipoom Jeong, and Nam Sung Kim

   Dec 2025
9. ATC

   Para-ksm: Parallelized memory deduplication with data streaming accelerator

   Houxiang Ji, Minho Kim, Seonmu Oh, Daehoon Kim, and Nam Sung Kim

   USENIX Annual Technical Conference (ATC) , Jul 2025

   
10. CAL

    srNAND: A novel NAND flash organization for Enhanced Small Read Throughput in SSDs

    Jeongho Lee, Sanjun Kim, Jaeyong Lee, Jaeyong Kang, Sungjin Lee, Nam Sung Kim, and Jihong Kim

    2025

    Architecture Memory

    Abs DOI

    Emerging data-intensive applications with frequent small random read operations challenge the throughput capabilities of conventional SSD architectures. Although Compute Express Link enabled SSDs allow for fine-grained data access with reduced latency, their read throughput remains limited by legacy block-oriented designs. To address this, we propose srNAND, an advanced NAND flash architecture for CXL SSDs. It uses a twostage ECC decoding mechanism to reduce read amplification, an optimized read command sequence to boost parallelism, and a request merging module to eliminate redundant operations. Our evaluation shows that srSSD can improve read throughput by up to 10.4× compared to conventional CXL SSDs.

11. ISCA

    Dynamic Load Balancer in Intel® Xeon® Scalable Processor: Performance analyses, enhancements, and guidelines

    Jiaqi Lou, Srikar Vanavasam, Yifan Yuan, Ren Wang, and Nam Sung Kim

    IEEE/ACM International Symposium on Computer Architecture (ISCA) , 2025

    Architecture Accelerator System Network

    
12. ISCA

    A4: Microarchitecture-aware LLC management for datacenter servers with emerging I/O devices

    Haneul Park, Jiaqi Lou, Sangjin Lee, Yifan Yuan, KyoungSoo Park, Yongseok Son, Ipoom Jeong, and Nam Sung Kim

    IEEE/ACM International Symposium on Computer Architecture (ISCA) , 2025

    Microarchitecture System

    
13. ISCA

    LIA: A single-GPU LLM inference acceleration with cooperative AMX-enabled CPU-GPU computation and CXL offloading

    Hyungyo Kim, Nachuan Wang, Qirong Xia, Jinghan Huang, Amir Yazdanbakhsh, and Nam Sung Kim

    IEEE/ACM International Symposium on Computer Architecture (ISCA) , 2025

    Architecture Accelerator System

    
14. ISCA

    Universal predicate pushdown to smart storage

    Ipoom Jeong, Jinghan Huang, Chuxuan Hu, Dohyun Park, Jaeyoung Kang, Nam Sung Kim, and Yongjoo Park

    IEEE/ACM International Symposium on Computer Architecture (ISCA) , 2025

    Architecture Storage Accelerator

    
15. ISCA

    Hybrid SLC-MLC RRAM mixed-signal processing-in-memory architecture for Transformer acceleration via gradient redistribution

    Chang Eun Song, Priyansh Bhatnagar, Zihan Xia, Nam Sung Kim, Tajana S Rosing, and Mingu Kang

    IEEE/ACM International Symposium on Computer Architecture (ISCA) , 2025

    Architecture Memory Accelerator

    
16. ISPASS

    Intel In-Memory Analytics Accelerator: performance characterization and guidelines

    Jaeyoung Kang, Qirong Xia, Ipoom Jeong, Yongjoo Park, and Nam Sung Kim

    IEEE International Symposium on Performance Analysis of Systems and Software (ISPASS) , 2025

    Best Paper Honorable Mention and Distinguished Artifact Award

    Architecture Accelerator

    
17. DAC

    A full-system, programmable, and extensible in-memory computing simulation framework for deep learning

    Kaining Zhou, Jian Huang, Nam Sung Kim, and Naresh Shanbhag

    IEEE/ACM Design Automation Conference (DAC) , 2025

    Architecture Accelerator Infrastructure
18. TVLSI

    Exploiting chiplet integration technology for fast high-capacity DRAM modules

    Zihan Xia, Chihun Song, Ram Krishna, Ashita Victor, Srujan Penta, Muhannad S. Bakir, Elyse Rosenbaum, Nam Sung Kim, and Mingu Kang

    IEEE Transactions on Very Large Scale Integration Systems (TVLSI) , 2025

    Architecture DRAM
19. ASPLOS

    Marionette: A RowHammer Attack via Row Coupling

    Seungmin Baek, Minbok Wi, Seonyong Park, Hwayong Nam, Michael Jaemin Kim, Nam Sung Kim, and Jung Ho Ahn

    Proceedings of the 30th ACM International Conference on Architectural Support for Programming Languages and Operating Systems , 2025

    System Security Architecture Memory

    Abs DOI

    A body of recent work has revealed that two different rows in a DRAM bank, from the perspective of a processor-memory interface, are connected to the same wordline but two separate row buffers (bitline sense amplifiers) in certain DRAM chips. Such a pair of rows is referred to as a ”coupled-row pair.” Coupled-row pairs pose a substantial security threat as RowHammer bitflips can be caused not only by the conventional, adjacent aggressor rows but also by their coupled rows that are distant in physical address We investigate the impact of a coupled row on both FPGA-based infrastructure and server systems. In RowHammer attacks, coupled rows have hammering strength nearly identical to aggressor rows, with these attacks invisible to conventional, processor-side mitigation solutions. By exploiting these observations, we present Marionette, a new type of RowHammer attack that exploits coupled rows to extend the existing RowHammer attack surface. First, coupled rows enable an attacker to evade two types of existing software-based RowHammer defenses: tracking- and isolation-based defenses. We induce RowHammer bitflips successfully against tracking-based RowHammer defenses by silently hammering coupled rows. We also identify the feasibility of RowHammer bitflips in an isolation-based inter-VM RowHammer defense by breaking DRAM-subarray-level isolation. Second, we successfully conduct an existing RowHammer exploit in a server under the tracking-based RowHammer defense. In a native server system, Marionette enhances the success rate of the RowHammer exploit by up to 1.66x. Lastly, we explore lightweight mitigation schemes for Marionette by exposing the coupled-row relationship to systems.

20. CAL

    Hardware-Accelerated Kernel-Space Memory Compression Using Intel QAT

    Qirong Xia^†, Houxiang Ji^†, Yang Zhou, and Nam Sung Kim

    IEEE Computer Architecture Letters , 2025

    System Accelerator

    Abs DOI

    Data compression has been widely used by datacenters to decrease the consumption of not only the memory and storage capacity but also the interconnect bandwidth. Nonetheless, the CPU cycles consumed for data compression notably contribute to the overall datacenter taxes. To provide a cost-efficient data compression capability for datacenters, Intel has introduced QuickAssist Technology (QAT), a PCIe-attached data-compression accelerator. In this work, we first comprehensively evaluate the compression/decompression performance of the latest on-chip QAT accelerator and then compare it with that of the previous-generation off-chip QAT accelerator. Subsequently, as a compelling application for QAT, we take a Linux memory optimization kernel feature: compressed cache for swap pages (zswap), re-implement it to use QAT efficiently, and then compare the performance of QAT-based zswap with that of CPU-based zswap. Our evaluation shows that the deployment of CPU-based zswap increases the tail latency of a co-running latency-sensitive application, Redis by 3.2-12.1×, while that of QAT-based zswap does not notably increase the tail latency compared to no deployment of zswap.

21. ASPLOS

    M5: Mastering Page Migration and Memory Management for CXL-based Tiered Memory Systems

    Yan Sun, Jongyul Kim, Zeduo Yu, Jiyuan Zhang, Siyuan Chai, Michael Jaemin Kim, Hwayong Nam, Jaehyun Park, Eojin Na, Yifan Yuan, Ren Wang, Jung Ho Ahn, Tianyin Xu, and Nam Sung Kim

    2025

    System Architecture Memory

    Abs DOI

    

    CXL has emerged as a promising memory interface that can cost-effectively expand the capacity and bandwidth of a memory system, complementing the traditional DDR interface. However, CXL DRAM presents 2-3x longer access latency than DDR DRAM, forming a tiered-memory system that demands an effective and efficient page-migration solution. Although many page-migration solutions have been proposed for past tiered-memory systems, they have achieved limited success. To tackle the challenge of managing tiered-memory systems, this work first presents a CXL-driven profiling solution to precisely and transparently count the number of accesses to every 4KB page and 64B word in CXL DRAM. Second, using the profiling solution, this work uncovers that (1) widely used CPU-driven page-migration solutions often identify warm pages as hot pages, and (2) certain applications have sparse hot pages, where only a small percentage of words in each of these pages are frequently accessed. Besides, this work demonstrates that the performance overhead of identifying hot pages is sometimes high enough to degrade application performance. Lastly, this work presents M5, a platform designed to facilitate the development of effective CXL-driven page-migration solutions, providing hardware-based hot-page and hot-word trackers in the CXL controller. On average, M5 can identify 47% hotter pages and offer 14% higher performance than the best CPU-driven page-migration solution, even with a simple policy.

22. CAL

    X-PPR: Post Package Repair for CXL Memory

    Chihun Song, Michael Jaemin Kim, Yan Sun, Houxiang Ji, Kyungsan Kim, TaeKyeong Ko, Jung Ho Ahn, and Nam Sung Kim

    IEEE Computer Architecture Letters , 2025

    Architecture DRAM

    Abs DOI

    CXL is an emerging interface that can cost-efficiently expand the capacity and bandwidth of servers, recycling DRAM modules from retired servers. Such DRAM modules, however, will likely have many uncorrectable faulty words due to years of strenuous use in datacenters. To repair faulty words in the field, a few solutions based on Post Package Repair (PPR) and memory offlining have been proposed. Nonetheless, they are either unable to fix thousands of faulty words or prone to causing severe memory fragmentation, as they operate at the granularity of DRAM row and memory page addresses, respectively. In this work, for cost-efficient use of recycled DRAM modules with thousands of faulty words, we propose CXL-PPR (X-PPR), exploiting the CXL’s support for near-memory processing and variable memory access latency. We demonstrate that X-PPR implemented in a commercial CXL device with DDR4 DRAM modules can handle a faulty bit probability that is 3.3×104 higher than ECC for a 512GB DRAM module. Meanwhile, X-PPR negligibly degrades the performance of popular memory-intensive benchmarks, which is achieved through two mechanisms designed in X-PPR to minimize the performance impact of additional DRAM accesses required for repairing faulty words.

23. Special Issue on Interconnects for Chiplet Integration Technologies

    Debendra Das Sharma and Nam Sung Kim

    IEEE Micro , 2025

    Abs DOI

    The next three articles explore UCIe as a chiplet interface. Onufryk and Choudhary, A2 from Intel, provide an in-depth look at UCIe specification, covering physical layer, channel, link layer reliability, protocol mapping, usage models, form factors, and compliance for both planar and vertical chiplets. They also discuss the system-level integration challenges around manageability and debug that UCIe addresses. They envision composable systems at the rack and pod level, enabled by copackaged optics with UCIe-based retimers. Huynh et al., A3 from Synopsys, present their UCIe implementation, design flow, bump map, package routing, and silicon measurements. Carusone et al., A4 from Alphawave, focus on UCIe IP implementation, detailing design flows and solutions for signal integrity at both silicon and package levels.

24. HPCA

    Warped-Compaction: Maximizing GPU Register File Bandwidth Utilization via Operand Compaction

    Eunbi Jeong, Ipoom Jeong, Myung Kuk Yoon, and Nam Sung Kim

    2025 IEEE International Symposium on High Performance Computer Architecture (HPCA) , 2025

    Architecture DRAM

    Abs DOI

    The GPU has been successfully used for diverse emerging compute-intensive applications, including imaging, computer vision, and more recently, deep learning, to name a few. To offer high performance for such applications, it is provisioned with massive Register Files (RFs) to exploit high Thread-Level Parallelism (TLP). As RFs are designed to store thousands of contexts and provide very high bandwidth for uninterrupted supply of operands to hundreds of compute units for high TLP, they have become one of the most power-hungry components in the GPU. Meanwhile, faced with the end of Dennard scaling, it is more important than ever to innovate the microarchitecture of (power-constrained) GPUs to continuously improve the performance of future compute-intensive applications.In this work, we propose a GPU microarchitecture, WarpedCompaction, designed to utilize given RFs more efficiently, instead of relying on larger-capacity and/or higher-bandwidth RFs to further improve GPU performance. Specifically, first, we reverse-engineer the latest GPU’s RF organization through microbenchmarking. This uncovers that each sub-core within a streaming multiprocessor contains only two dual-ported RF banks, and accesses to these banks are arbitrated solely based on register IDs. We also reveal that, despite the modest configurations, RF banks are largely underutilized, staying inactive for 33.5% of the time. Second, we observe that previously proposed RF optimization techniques, data forwarding and dead register elimination, cannot address this underutilization problem. This is mainly due to the (R1) insufficient RF access requests from a limited number of Operand Collector Units (OCUs) and (R2) inefficient operand distribution by the conventional RF bank arbitration. Third, we present two architectural solutions to tackle the observed inefficiency: (S1) OCU sharing and (S2) skewed arbitrator. Building on enhanced OCU early allocation for partially ready instructions and operand forwarding to OCUs, OCU sharing allows two different warp instructions to share a single OCU, and skewed arbitrator evenly distributes register accesses across RF banks, maximizing the utilization of given RF bandwidth. The synergistic integration of these techniques, forming Warped-Compaction, results in 26.1% higher performance and 21.7% better energy efficiency of RF and OCU compared to baseline high-end GPU.

25. PATENT

    Application-transparent near-memory processing architecture with memory channel network

    Nam Sung Kim and Mohammad Alian

    2025

    Abs

    A computing device includes a host processor to execute a host driver to create a host-side interface, the host-side interface emulating a first Ethernet interface, assign the host-side interface a first medium access control (MAC) address and a first Internet Protocol (IP) address. Memory components are disposed on a substrate. A memory channel network (MCN) processor is disposed on the substrate and coupled between the memory components and the host processor. The MCN processor is to execute an MCN driver to create a MCN-side interface, the MCN-side interface emulating a second Ethernet interface. The MCN processor is to assign the MCN-side interface a second MAC address and a second IP address, which identify the MCN processor as a MCN network node to the host processor.

26. CAL

    Cooperative Memory Deduplication with Intel Data Streaming Accelerator

    Houxiang Ji, Minho Kim, Seonmu Oh, Daehoon Kim, and Nam Sung Kim

    IEEE Computer Architecture Letters , 2025

    Accelerator System

    Abs DOI

    Memory deduplication plays a critical role in reducing memory consumption and the total cost of ownership (TCO) in hyperscalers, particularly as the advent of large language models imposes unprecedented demands on memory resources. However, conventional CPU-based memory deduplication can interfere with co-running applications, significantly impacting the performance of time-sensitive workloads. Intel introduced the on-chip Data Streaming Accelerator (DSA), providing high-performance data movement and transformation capabilities, including comparison and checksum calculation, which are heavily utilized in the deduplication. In this work, we enhance a widely-used kernel-space memory deduplication feature, Kernel Samepage Merging (ksm), by selectively offloading these operations to the DSA. Our evaluation demonstrates that CPU-based ksm can lead to 5.0–10.9× increase in the tail latency of co-running applications while DSA-based ksm limits the latency increase to just 1.6× while achieving comparable memory savings.

2024

1. ASPLOS

   AttAcc! Unleashing the power of PIM for batched transformer-based generative model inference

   Jaehyun Park, Jaewan Choi, Kwanhee Kyung, Michael Jaemin Kim, Yongsuk Kwon, Nam Sung Kim, and Jung Ho Ahn

   29th ACM International Conference on Architectural Support for Programming Languages and Operating Systems , 2024

   Architecture Accelerator Memory ML/AI

   Abs DOI

   The Transformer-based generative model (TbGM), comprising summarization (Sum) and generation (Gen) stages, has demonstrated unprecedented generative performance across a wide range of applications. However, it also demands immense amounts of compute and memory resources. Especially, the Gen stages, consisting of the attention and fully-connected (FC) layers, dominate the overall execution time. Meanwhile, we reveal that the conventional system with GPUs used for TbGM inference cannot efficiently execute the attention layer, even with batching, due to various constraints. To address this inefficiency, we first propose AttAcc, a processing-in-memory (PIM) architecture for efficient execution of the attention layer. Subsequently, for the end-to-end acceleration of TbGM inference, we propose a novel heterogeneous system architecture and optimizations that strategically use xPU and PIM together. It leverages the high memory bandwidth of AttAcc for the attention layer and the powerful compute capability of the conventional system for the FC layer. Lastly, we demonstrate that our GPU-PIM system outperforms the conventional system with the same memory capacity, improving performance and energy efficiency of running a 175B TbGM by up to 2.81× and 2.67×, respectively.

2. HPCA

   An lpddr-based cxl-pnm platform for tco-efficient inference of transformer-based large language models

   Sang-Soo Park, KyungSoo Kim, Jinin So, Jin Jung, Jonggeon Lee, Kyoungwan Woo, Nayeon Kim, Younghyun Lee, Hyungyo Kim, Yongsuk Kwon, Jinhyun Kim, Jieun Lee, YeonGon Cho, Yongmin Tai, Jeonghyeon Cho, Hoyoung Song, Jung Ho Ahn, and Nam Sung Kim

   2024 IEEE International Symposium on High-Performance Computer Architecture (HPCA) , 2024

   Architecture Memory Accelerator ML/AI

   Abs DOI

   Transformer-based large language models (LLMs) such as Generative Pre-trained Transformer (GPT) have become popular due to their remarkable performance across diverse applications, including text generation and translation. For LLM training and inference, the GPU has been the predominant accelerator with its pervasive software development ecosystem and powerful computing capability. However, as the size of LLMs keeps increasing for higher performance and/or more complex applications, a single GPU cannot efficiently accelerate LLM training and inference due to its limited memory capacity, which demands frequent transfers of the model parameters needed by the GPU to compute the current layer(s) from the host CPU memory/storage. A GPU appliance may provide enough aggregated memory capacity with multiple GPUs, but it suffers from frequent transfers of intermediate values among GPU devices, each accelerating specific layers of a given LLM. As the frequent transfers of these model parameters and intermediate values are performed over relatively slow device-to-device interconnects such as PCIe or NVLink, they become the key bottleneck for efficient acceleration of LLMs. Focusing on accelerating LLM inference, which is essential for many commercial services, we develop CXL-PNM, a processing near memory (PNM) platform based on the emerging interconnect technology, Compute eXpress Link (CXL). Specifically, we first devise an LPDDR5X-based CXL memory architecture with 512GB of capacity and 1.1TB/s of bandwidth, which boasts 16× larger capacity and 10× higher bandwidth than GDDR6and DDR5-based CXL memory architectures, respectively, under a module form-factor constraint. Second, we design a CXLPNM controller architecture integrated with an LLM inference accelerator, exploiting the unique capabilities of such CXL memory to overcome the disadvantages of competing technologies such as HBM-PIM and AxDIMM. Lastly, we implement a CXLPNM software stack that supports seamless and transparent use of CXL-PNM for Python-based LLM programs. Our evaluation shows that a CXL-PNM appliance with 8 CXL-PNM devices offers 23% lower latency, 31% higher throughput, and 2.8× higher energy efficiency at 30% lower hardware cost than a GPU appliance with 8 GPU devices for an LLM inference service.

3. ASPLOS

   A quantitative analysis and guidelines of data streaming accelerator in modern intel xeon scalable processors

   Reese Kuper, Ipoom Jeong, Yifan Yuan, Ren Wang, Narayan Ranganathan, Nikhil Rao, Jiayu Hu, Sanjay Kumar, Philip Lantz, and Nam Sung Kim

   29th ACM International Conference on Architectural Support for Programming Languages and Operating Systems , 2024

   Architecture CPU Accelerator

   Abs DOI

   

   As semiconductor power density is no longer constant with the technology process scaling down, we need different solutions if we are to continue scaling application performance. To this end, modern CPUs are integrating capable data accelerators on the chip, aiming to improve performance and efficiency for a wide range of applications and usages. One such accelerator is the Intel® Data Streaming Accelerator (DSA) introduced since Intel® 4th Generation Xeon® Scalable CPUs (Sapphire Rapids). DSA targets data movement operations in memory that are common sources of overhead in datacenter workloads and infrastructure. In addition, it supports a wider range of operations on streaming data, such as CRC32 calculations, computation of deltas between data buffers, and data integrity field (DIF) operations. This paper aims to introduce the latest features supported by DSA, dive deep into its versatility, and analyze its throughput benefits through a comprehensive evaluation with both microbenchmarks and real use cases. Along with the analysis of its characteristics and the rich software ecosystem of DSA, we summarize several insights and guidelines for the programmer to make the most out of DSA, and use an in-depth case study of DPDK Vhost to demonstrate how these guidelines benefit a real application.

4. ASPLOS

   Tandem processor: Grappling with emerging operators in neural networks

   Soroush Ghodrati, Sean Kinzer, Hanyang Xu, Rohan Mahapatra, Yoonsung Kim, Byung Hoon Ahn, Dong Kai Wang, Lavanya Karthikeyan, Amir Yazdanbakhsh, Jongse Park, Nam Sung Kim, and Hadi Esmaeilzadeh

   29th ACM International Conference on Architectural Support for Programming Languages and Operating Systems , 2024

   IEEE Micro Top Picks, Honorable Mention, 2025

   Architecture Accelerator ML/AI

   Abs DOI

   With the ever increasing prevalence of neural networks and the upheaval from the language models, it is time to rethink neural acceleration. Up to this point, the broader research community, including ourselves, has disproportionately focused on GEneral Matrix Multiplication (GEMM) operations. The supporting argument was that the large majority of the neural operations are GEMM. This argument guided the research in Neural Processing Units (NPUs) for the last decade. However, scant attention was paid to non-GEMM operations and they are rather overlooked. As deep learning evolved and progressed, these operations have grown in diversity and also large variety of structural patterns have emerged that interweave them with the GEMM operations. However, conventional NPU designs have taken rather simplistic approaches by supporting these operations through either a number of dedicated blocks or fall back to general-purpose processors. This work sets out to challenge the conventional wisdom in neural accelerator design and explore the architecture of an on-chip companion, dubbed Tandem Processor, that complements the rather optimized GEMM unit in neural accelerators. This processor needs to be specialized to keep up with the GEMM unit; and yet needs to be programmable to address the (1) structural and (2) operational variations. To strike a balance between specialization and programmability, on the one hand, we specialize its memory access logic with a novel ISA/microarchitecture that alleviates the register file and its associated load/store operations. On the other hand, the calculations of the non-GEMM layers are only supported through primitive arithmetic/logic vector operations. Therefore, programmability is offered at the mathematical level. The enhancements due to the specialization of the memory access logic in the Tandem Processor and its tight integration with the GEMM unit sustain the throughput and the utilization of the neural accelerator. Comprehensive evaluations of the proposed design based on the end-to-end execution of seven diverse DNNs including emerging language models show significant performance improvements and energy reduction enabled by leveraging the Tandem Processor. We provide the RTL code that is synthesizable both for FPGA and ASIC implementations in addition to the associated compiler as part of the open-source GeneSys project (https://actlab-genesys.github.io/). We also present the chip floorplan and post-layout analysis. This work is the result of 10 years of effort in building real NPUs that support end-to-end neural network execution.

5. CAL

   Exploiting Intel® Advanced Matrix Extensions (AMX) for Large Language Model Inference

   Hyungyo Kim, Gaohan Ye, Nachuan Wang, Amir Yazdanbakhsh, and Nam Sung Kim

   IEEE Computer Architecture Letters , 2024

   2024 IEEE Computer Architecture Letter (CAL) Best Paper Award

   Architecture Accelerator System ML/AI

   Abs DOI

   The ever-increasing number of parameters in Large Language Models (LLMs) demands many expensive GPUs for both inference and training. This is because even such a high-end GPU such as NVIDIA A100 can store only a subset of parameters due to its limited memory capacity. To reduce the number of required GPUs, especially for inference, we may exploit the large memory capacity of (host) CPU to store not only all the model parameters but also intermediate outputs which also require a substantial memory capacity. However, this necessitates frequent data transfers between CPU and GPU over the slow PCIe interface, creating a bottleneck that hinders the accomplishment of both low latency and high throughput in inference. To address such a challenge, we first propose CPU-GPU cooperative computing that exploits the Advanced Matrix Extensions (AMX) capability of the latest Intel CPU, codenamed Sapphire Rapids (SPR). Second, we propose an adaptive model partitioning policy that determines the layers of a given LLM to be run on CPU and GPU, respectively, based on their memory capacity requirement and arithmetic intensity. As CPU executes the layers with large memory capacity but low arithmetic intensity, the amount of data transferred through the PCIe interface is significantly reduced, thereby improving the LLM inference performance. Our evaluation demonstrates that CPU-GPU cooperative computing, based on this policy, delivers 12.1× lower latency and 5.4× higher throughput than GPU-only computing for OPT-30B inference when both CPU-GPU and GPU-only computing store the model in CPU memory.

6. Scalecache: A scalable page cache for multiple solid-state drives

   Kiet Tuan Pham, Seokjoo Cho, Sangjin Lee, Lan Anh Nguyen, Hyeongi Yeo, Ipoom Jeong, Sungjin Lee, Nam Sung Kim, and Yongseok Son

   2024

   Systems Storage

   Abs DOI

   This paper presents a scalable page cache called ScaleCache for improving SSD scalability. Specifically, we first propose a concurrent data structure of page cache based on XArray (ccXArray) to enable access and update the page cache concurrently. Second, we introduce a direct page flush (dflush) which directly flushes pages to storage devices in a parallel and opportunistic manner. We implement ScaleCache with two techniques in the Linux kernel and evaluate it on a 64-core machine with eight NVMe SSDs. Our evaluations show that ScaleCache improves the performance of Linux file systems by up to 6.81× and 4.50× compared with the existing scheme and scalable scheme for multiple SSDs, respectively.

7. ISCA

   DRAMScope: Uncovering DRAM Microarchitecture and Characteristics by Issuing Memory Commands

   Hwayong Nam, Seungmin Baek, Minbok Wi, Michael Jaemin Kim, Jaehyun Park, Chihun Song, Nam Sung Kim, and Jung Ho Ahn

   2024 ACM/IEEE 51st Annual International Symposium on Computer Architecture (ISCA) , 2024

   Architecture Memory

   Abs DOI

   

   The demand for precise information on DRAM microarchitectures and error characteristics has surged, driven by the need to explore processing in memory, enhance reliability, and mitigate security vulnerability. Nonetheless, DRAM manufacturers have disclosed only a limited amount of information, making it difficult to find specific information on their DRAM microarchitectures. This paper addresses this gap by presenting more rigorous findings on the microarchitectures of commodity DRAM chips and their impacts on the characteristics of activate-induced bitflips (AIBs), such as RowHammer and RowPress. The previous studies have also attempted to understand the DRAM microarchitectures and associated behaviors, but we have found some of their results to be misled by inaccurate address mapping and internal data swizzling, or lack of a deeper understanding of the modern DRAM cell structure. For accurate and efficient reverse-engineering, we use three tools: AIBs, retention time test, and RowCopy, which can be cross-validated. With these three tools, we first take a macroscopic view of modern DRAM chips to uncover the size, structure, and operation of their subarrays, memory array tiles (MATs), and rows. Then, we analyze AIB characteristics based on the microscopic view of the DRAM microarchitecture, such as 6F2 cell layout, through which we rectify misunderstandings regarding AIBs and discover a new data pattern that accelerates AIBs. Lastly, based on our findings at both macroscopic and microscopic levels, we identify previously unknown AIB vulnerabilities and propose a simple yet effective protection solution.

8. ISCA

   Intel accelerators ecosystem: An soc-oriented perspective: Industry product

   Yifan Yuan, Ren Wang, Narayan Ranganathan, Nikhil Rao, Sanjay Kumar, Philip Lantz, Vivekananthan Sanjeepan, Jorge Cabrera, Atul Kwatra, Rajesh Sankaran, Ipoom Jeong, and Nam Sung Kim

   2024 ACM/IEEE 51st Annual International Symposium on Computer Architecture (ISCA) , 2024

   Architecture CPU Accelerator System

   Abs DOI

   A growing demand for hyperscale services has compelled hyperscalers to deploy more compute resources at an unprecedented pace, further accelerated by the demise of Dennard scaling. Meanwhile, a considerable portion of the compute resources are consumed to execute common functions present across the hyperscale services, i.e., datacenter taxes. These challenges motivated many to explore specialized accelerators for these functions. Leading such a technology trend, Intel has integrated diverse on-chip accelerators into its recent flagship datacenter CPU products. Furthermore, to support the easy and efficient use of these accelerators for successful deployment in production hyperscale services, Intel has developed a hardware/software ecosystem. In this paper, we first focus on Intel’s holistic efforts to build the hardware/software ecosystem, presenting key SoC-level features that facilitate efficient CPU-accelerator interaction, effortless programming and use, and scalable accelerator sharing and virtualization. Next, we delve into the functions, microarchitectures, and software stacks of three new on-chip accelerators: Data Streaming Accelerator (DSA), In-memory Analytics Accelerator (IAA), and Dynamic Load Balancer (DLB). Lastly, we demonstrate that Intel’s on-chip accelerators can not only significantly reduce the datacenter taxes but also accelerate data-intensive applications essential for hyperscale services, with little effort to use the accelerators.

9. ASPLOS

   TAROT: A CXL SmartNIC-Based Defense Against Multi-bit Errors by Row-Hammer Attacks

   Chihun Song, Michael Jaemin Kim, Tianchen Wang, Houxiang Ji, Jinghan Huang, Ipoom Jeong, Jaehyun Park, Hwayong Nam, Minbok Wi, Jung Ho Ahn, and Nam Sung Kim

   Proceedings of the 29th ACM International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS) , Apr 2024

   Architecture Memory Accelerator Security

   Abs DOI

   

   Row Hammer (RH) has been demonstrated as a security vulnerability in modern systems. Although commodity CPUs can handle RH-induced single-bit errors in DRAM through ECC, RH can still give rise to multi-bit uncorrectable errors (UEs) and crash the systems. Meanwhile, recent work has indicated that the DRAM cells vulnerable to RH are determined by manufacturing imperfections and resulting defects. Taking one step further from the recent work, we first conduct RH experiments on contemporary DRAM modules for 3 weeks. This demonstrates that RH-induced UEs occur only at specific DRAM addresses (RH-UE-vulnerable addresses) and the percentage of such addresses is small in these DRAM modules. Second, to protect the systems from RH-induced UEs, we propose two RH defense solutions: H- and S-TAROT (TArgeted ROw-Hammer Therapy). H-TAROT is a software-based solution running on the host CPU. It obtains RH-UE-vulnerable addresses during the system boot and then periodically accesses such addresses before UEs may occur. Since it accesses only a small percentage of addresses, it does not incur a notable performance penalty for throughput applications (e.g., a 1.5% increase in execution time of the SPECrate 2017 benchmark suite running on a system even with 128GB of DRAM). Yet, it imposes a significant performance penalty on latency-sensitive applications (e.g., a 28.2% increase in tail latency of Redis). To minimize the performance penalty, for a system with a SmartNIC (SNIC), S-TAROT offloads H-TAROT from the host CPU to the SNIC CPU. Our experiment shows that S-TAROT increases the execution time and tail latency of the SPECrate 2017 benchmark suite and Redis by only 0.1% and 1.0%, respectively.

10. HPCA

    Spade: Sparse pillar-based 3d object detection accelerator for autonomous driving

    Minjae Lee, Seongmin Park, Hyungmin Kim, Minyong Yoon, Janghwan Lee, Jun Won Choi, Nam Sung Kim, Mingu Kang, and Jungwook Choi

    2024 IEEE International Symposium on High-Performance Computer Architecture (HPCA) , Feb 2024

    Architecture Accelerator ML/AI

    Abs DOI

    3D object detection using point cloud (PC) data is essential for perception pipelines of autonomous driving, where efficient encoding is key to meeting stringent resource and latency requirements. PointPillars, a widely adopted bird’s-eye view (BEV) encoding, aggregates 3D point cloud data into 2D pillars for fast and accurate 3D object detection. However, the stateof-the-art methods employing PointPillars overlook the inherent sparsity of pillar encoding where only a valid pillar is encoded with a vector of channel elements, missing opportunities for significant computational reduction. Meanwhile, current sparse convolution accelerators are designed to handle only elementwise activation sparsity and do not effectively address the vector sparsity imposed by pillar encoding. In this paper, we propose SPADE, an algorithm-hardware codesign strategy to maximize vector sparsity in pillar-based 3D object detection and accelerate vector-sparse convolution commensurate with the improved sparsity. SPADE consists of three components: (1) a dynamic vector pruning algorithm balancing accuracy and computation savings from vector sparsity, (2) a sparse coordinate management hardware transforming 2D systolic array into a vector-sparse convolution accelerator, and (3) sparsityaware dataflow optimization tailoring sparse convolution schedules for hardware efficiency. Taped-out with a commercial technology, SPADE saves the amount of computation by 36.3–89.2% for representative 3D object detection networks and benchmarks, leading to 1.3–10.9 × speedup and 1.5–12.6 × energy savings compared to the ideal dense accelerator design. These sparsityproportional performance gains equate to 4.1–28.8 × speedup and 90.2–372.3 × energy savings compared to the counterpart server and edge platforms.

11. Transforming the Hybrid Cloud for Emerging AI Workloads

    Deming Chen, Alaa Youssef, Ruchi Pendse, André Schleife, Bryan K Clark, Hendrik Hamann, Jingrui He, Teodoro Laino, Lav Varshney, Yuxiong Wang, Avirup Sil, Reyhaneh Jabbarvand, Tianyin Xu, Volodymyr Kindratenko, Carlos Costa, Sarita Adve, Charith Mendis, Minjia Zhang, Santiago Núñez-Corrales, Raghu Ganti, Mudhakar Srivatsa, Nam Sung Kim, Josep Torrellas, Jian Huang, Seetharami Seelam, Klara Nahrstedt, Tarek Abdelzaher, Tamar Eilam, Huimin Zhao, Matteo Manica, Ravishankar Iyer, Martin Hirzel, Vikram Adve, Darko Marinov, Hubertus Franke, Hanghang Tong, Elizabeth Ainsworth, Han Zhao, Deepak Vasisht, Minh Do, Fabio Oliveira, Giovanni Pacifici, Ruchir Puri, and Priya Nagpurkar

    arXiv preprint arXiv:2411.13239 , 2024

    Architecture System ML

    Abs

    This white paper, developed through close collaboration between IBM Research and UIUC researchers within the IIDAI Institute, envisions transforming hybrid cloud systems to meet the growing complexity of AI workloads through innovative, full-stack co-design approaches, emphasizing usability, manageability, affordability, adaptability, efficiency, and scalability. By integrating cutting-edge technologies such as generative and agentic AI, cross-layer automation and optimization, unified control plane, and composable and adaptive system architecture, the proposed framework addresses critical challenges in energy efficiency, performance, and cost-effectiveness. Incorporating quantum computing as it matures will enable quantum-accelerated simulations for materials science, climate modeling, and other high-impact domains. Collaborative efforts between academia and industry are central to this vision, driving advancements in foundation models for material design and climate solutions, scalable multimodal data processing, and enhanced physics-based AI emulators for applications like weather forecasting and carbon sequestration. Research priorities include advancing AI agentic systems, LLM as an Abstraction (LLMaaA), AI model optimization and unified abstractions across heterogeneous infrastructure, end-to-end edge-cloud transformation, efficient programming model, middleware and platform, secure infrastructure, application-adaptive cloud systems, and new quantum-classical collaborative workflows. These ideas and solutions encompass both theoretical and practical research questions, requiring coordinated input and support from the research community. This joint initiative aims to establish hybrid clouds as secure, efficient, and sustainable platforms, fostering breakthroughs in AI-driven applications and scientific discovery across academia, industry, and society.

12. Lupin: Tolerating Partial Failures in a CXL Pod

    Zhiting Zhu, Newton Ni, Yibo Huang, Yan Sun, Zhipeng Jia, Nam Sung Kim, and Emmett Witchel

    2024

    Architecture System ML

    Abs DOI

    A compute express link (CXL) pod is a collection of hosts attached to a CXL memory module. It provides an opportunity to port single-host shared-memory programs to execute on multiple hosts in a CXL pod, where the ported application achieves higher performance than a distributed application that uses network for coordination. The cost of performance scaling on a CXL pod is that applications should tolerate partial failures, where one process or operating system fails or reboots. Lupin is system software that includes kernel modifications and user-level libraries to help applications remain available while they recover from partial failures using the contents of CXL memory.

13. ISCA

    Hal: Hardware-assisted load balancing for energy-efficient snic-host cooperative computing

    Jinghan Huang, Jiaqi Lou, Srikar Vanavasam, Xinhao Kong, Houxiang Ji, Ipoom Jeong, Danyang Zhuo, Eun Kyung Lee, and Nam Sung Kim

    2024 ACM/IEEE 51st Annual International Symposium on Computer Architecture (ISCA) , 2024

    Architecture Accelerator System Network

    Abs DOI

    

    A typical SmartNIC (SNIC) integrates a processor comprising Arm CPU and accelerators with a conventional NIC. The processor is designed to energy-efficiently execute network functions frequently used by datacenter applications. With such a processor, the SNIC has promised to notably improve the system-wide energy efficiency of datacenter servers. Nevertheless, the latest trend of integrating accelerators into server CPUs for these functions sparks a question on the SNIC processor’s superiority over a host processor (i.e., server CPU with accelerators) in system-wide energy efficiency, especially under given tail latency constraints. Answering this question, we first take an Intel Xeon processor, integrated with various accelerators (e.g., QuickAssist Technology), as a host processor, and then compare it to an NVIDIA BlueField-2 SNIC processor. This uncovers that (1) the host accelerator, coupled with a more powerful memory subsystem, can outperform the SNIC accelerator, and (2) the SNIC processor can improve system-wide energy efficiency only at low packet rates for most functions under tail latency constraints. To provide high system-wide energy efficiency without compromising tail latency at any packet rates, we propose HAL, consisting of a hardware-based load balancer and an intelligent load balancing policy implemented inside the SNIC. When HAL determines that the SNIC processor cannot efficiently process a given function beyond a specific packet rate, it limits the rate of packets to the SNIC processor and lets the host processor handle the excess. We implement a HAL-enabled SNIC with a commodity FPGA and a BlueField-2 SNIC, plug it into a commodity server, and run 10 popular network functions. Our evaluation shows that HAL can improve the system-wide energy efficiency and throughput of the server running these functions by 31% and 10%, respectively, without notably increasing the tail latency.

14. NSDI

    Harmonic: Hardware-assisted RDMA Performance Isolation for Public Clouds

    Jiaqi Lou^†, Xinhao Kong^†, Jinghan Huang, Wei Bai, Nam Sung Kim, and Danyang Zhuo

    21st USENIX Symposium on Networked Systems Design and Implementation (NSDI 24) , 2024

    System Network

    Abs

    Performance isolation is essential for sharing resources in multi-tenant public clouds. Compared with traditional kernel-based networking, RDMA presents unique challenges especially because RDMA NIC’s complex microarchitecture resources are often hidden from users. Current RDMA isolation methods overlook these microarchitecture resources, leading to insufficient performance isolation. Consequently, a faulty/malicious tenant can exploit these microarchitecture resources to compromise well-behaved tenants’ network performance. In this paper, we introduce the first microarchitecture-resource-aware RDMA performance isolation solution for public clouds, Harmonic. It consists of two key components designed to be conscious of the RDMA NIC’s microarchitectural resources:(1) a programmable intelligent PCIe switch (prototyped with FPGA) and (2) an RDMA-friendly rate limiter. At runtime, these two components allow us to accurately monitor and modulate the RDMA NIC resource usage per tenant. We evaluate Harmonic with a state-of-the-art RDMA performance isolation test suite (Husky) and a popular in-memory database application (Redis). We demonstrate that Harmonic can not only successfully pass Husky but also provide Redis with 1.4× higher throughput than the best alternative isolation solution.

15. MICRO

    Demystifying a CXL Type-2 Device: A Heterogeneous Cooperative Computing Perspective

    Houxiang Ji, Srikar Vanavasam, Yang Zhou, Qirong Xia, Jinghan Huang, Yifan Yuan, Ren Wang, Pekon Gupta, Bhushan Chitlur, Ipoom Jeong, and Nam Sung Kim

    2024 57th IEEE/ACM International Symposium on Microarchitecture (MICRO) , 2024

    System Accelerator

    Abs DOI

    

    CXL is the latest interconnect technology built on PCIe, providing three protocols to facilitate three distinct types of devices, each with unique capabilities. Among these devices, a CXL Type-2 device has become commercially available, followed by CXL Type-3 devices. Therefore, it is timely to understand capabilities and characteristics of the CXL Type-2 device, as well as explore suitable applications. In this work, first, we delve into three key features of a CXL Type-2 device: cache-coherent device accelerator to host memory, device accelerator to device memory, and host CPU to device memory accesses. Second, using microbenchmarks, we comprehensively characterize the latency and bandwidth of these memory accesses with a CXL Type-2 device, and then compare them with those of equivalent memory accesses with comparable devices, such as emulated CXL Type-2, CXL Type-3, and PCIe devices. Lastly, as applications that exploit the unique capabilities of a CXL Type-2 device, we propose two CXL-based Linux memory optimization features: compressed RAM cache for swap (zswap) and memory deduplication (ksm). Our evaluation shows that Redis, when running with traditional CPU-based zswap and ksm, suffers from a tail latency increase of 4.5-10.3× compared to Redis running alone. While PCIe-based zswap and ksm still experience a tail latency increase of up to 8.1×, CXL-based zswap and ksm practically eliminate the tail latency increase with faster and more efficient host-device communication than PCIe-based zswap and ksm.

16. PATENT

    Computer Architecture Having Selectable Parallel and Serial Communication Channels Between Processors and Memory

    Hao Wang and Nam Sung Kim

    2024

    Architecture DRAM

    Abs

    A computer architecture provides both a parallel memory bus and serial memory bus between a processor system and memory. Latency-tolerant memory access requests are steered to the serial memory bus which operates to increase the available memory bus bandwidth on the parallel memory. The invention also provides integrated circuit computer memory suitable for this application.

17. PATENT

    Computer architecture having selectable parallel and serial communication channels between processors and memory

    Hao Wang and Nam Sung Kim

    2024

    Abs DOI

    A computer architecture provides both a parallel memory bus and serial memory bus between a processor system and memory. Latency-tolerant memory access requests are steered to the serial memory bus which operates to increase the available memory bus bandwidth on the parallel memory. The invention also provides integrated circuit computer memory suitable for this application.

18. FriendlyFoe: Adversarial Machine Learning as a Practical Architectural Defense against Side Channel Attacks

    Hyoungwook Nam, Raghavendra Pradyumna Pothukuchi, Bo Li, Nam Sung Kim, and Josep Torrellas

    2024

    Architecture CPU Security ML/AI

    Abs DOI

    Machine learning (ML)-based side channel attacks have become prominent threats to computer security. These attacks are often powerful, as ML models easily find patterns in signals. To address this problem, this paper proposes dynamically applying Adversarial Machine Learning (AML) to obfuscate side channels. The rationale is that it has been shown that intelligently injecting an adversarial perturbation can confuse ML classifiers. We call this approach FriendlyFoe and the neural network we introduce to perturb signals FriendlyFoe Defender. FriendlyFoe is a practical, effective, and general architectural technique to obfuscate signals. We show a workflow to design Defenders with low overhead and information leakage, and to customize them for different environments. Defenders are transferable, i.e., they thwart attacker classifiers that are different from those used to train the Defenders. They also resist adaptive attacks, where attackers train using the obfuscated signals collected while the Defender is active. Finally, the approach is general enough to be applicable to different environments. We demonstrate FriendlyFoe against two side channel attacks: one based on memory contention and one on system power. The first example uses a hardware Defender with ns-level response time that, for the same level of security as a Pad-to-Constant scheme, has 27% and 64% lower performance overhead for single- and multi-threaded workloads, respectively. The second example uses a software Defender with ms-level response time that reduces leakage by 3.7 × over a state-of-the-art scheme while reducing the energy overhead by 22.5%.

19. Yield-Aware Interposer Design for UCIe Interconnects

    Ram Krishna, Ashita Victor, Srujan Penta, Xu Chen, Muhannad S Bakir, Nam Sung Kim, and Elyse Rosenbaum

    2024 IEEE 33rd Conference on Electrical Performance of Electronic Packaging and Systems (EPEPS) , 2024

    Circuit Interconnect

    Abs DOI

    This work presents an interposer design methodol-ogy for UCle die-to-die interfaces that maximizes yield given a set of signal integrity specifications. Four different routing configurations on a silicon interposer are considered, and the cost, performance, and yield tradeoffs are elucidated. The fabrication steps for the D2D interconnects are outlined and sources of yield-limiting variability are identified. The yield analysis is expedited by the use of a Gaussian Process Regression surrogate model.

20. PATENT

    Systems and methods for hardware-based asynchronous persistence

    Ahmed Abulila, Nam Sung Kim, and Izzat El Hajj

    2024

    Abs

    Systems and methods for hardware-based asynchronous logging include: initiating first and second atomic regions on first and second cores of a central processing unit (CPU); and asynchronously logging data for the first atomic region and the second atomic region using the CPU by: asynchronously performing log persist operations (LPOs) to log an old data value from each atomic region; updating the old data value to a new data value from each atomic region; tracking dependencies between the first atomic region and the second atomic region using a memory controller; asynchronously performing data persist operations (DPOs) to persist the new data value for each atomic region; and committing the first atomic region and the second atomic region based on the dependencies using the memory controller of the CPU.

21. Tandem Processor: Grappling with Emerging Operators in Neural Networks

    Soroush Ghodrati Sean Kinzer Hanyang Xu, Rohan Mahapatra Yoonsung Kim, Byung Hoon Ahn Dong Kai Wang, Lavanya Karthikeyan Amir Yazdanbakhsh, Jongse Park, Nam Sung Kim, and Hadi Esmaeilzadeh

    2024

    Architecture Accelerator ML/AI

    Abs DOI

    With the ever increasing prevalence of neural networks and the upheaval from the language models, it is time to rethink neural acceleration. Up to this point, the broader research community, including ourselves, has disproportionately focused on GEneral Matrix Multiplication (GEMM) operations. The supporting argument was that the large majority of the neural operations are GEMM. This argument guided the research in Neural Processing Units (NPUs) for the last decade. However, scant attention was paid to non-GEMM operations and they are rather overlooked. As deep learning evolved and progressed, these operations have grown in diversity and also large variety of structural patterns have emerged that interweave them with the GEMM operations. However, conventional NPU designs have taken rather simplistic approaches by supporting these operations through either a number of dedicated blocks or fall back to general-purpose processors. This work sets out to challenge the conventional wisdom in neural accelerator design and explore the architecture of an on-chip companion, dubbed Tandem Processor, that complements the rather optimized GEMM unit in neural accelerators. This processor needs to be specialized to keep up with the GEMM unit; and yet needs to be programmable to address the (1) structural and (2) operational variations. To strike a balance between specialization and programmability, on the one hand, we specialize its memory access logic with a novel ISA/microarchitecture that alleviates the register file and its associated load/store operations. On the other hand, the calculations of the non-GEMM layers are only supported through primitive arithmetic/logic vector operations. Therefore, programmability is offered at the mathematical level. The enhancements due to the specialization of the memory access logic in the Tandem Processor and its tight integration with the GEMM unit sustain the throughput and the utilization of the neural accelerator. Comprehensive evaluations of the proposed design based on the end-to-end execution of seven diverse DNNs including emerging language models show significant performance improvements and energy reduction enabled by leveraging the Tandem Processor. We provide the RTL code that is synthesizable both for FPGA and ASIC implementations in addition to the associated compiler as part of the open-source GeneSys project (https://actlab-genesys.github.io/). We also present the chip floorplan and post-layout analysis. This work is the result of 10 years of effort in building real NPUs that support end-to-end neural network execution.

2023

1. MICRO

   Demystifying cxl memory with genuine cxl-ready systems and devices

   Yan Sun, Yifan Yuan, Zeduo Yu, Reese Kuper, Chihun Song, Jinghan Huang, Houxiang Ji, Siddharth Agarwal, Jiaqi Lou, Ipoom Jeong, Ren Wang, Jung Ho Ahn, Tianyin Xu, and Nam Sung Kim

   2023 56th IEEE/ACM International Symposium on Microarchitecture (MICRO) , 2023

   Systems Memory

   Abs DOI

   The ever-growing demands for memory with larger capacity and higher bandwidth have driven recent innovations on memory expansion and disaggregation technologies based on Compute eXpress Link (CXL). Especially, CXL-based memory expansion technology has recently gained notable attention for its ability not only to economically expand memory capacity and bandwidth but also to decouple memory technologies from a specific memory interface of the CPU. However, since CXL memory devices have not been widely available, they have been emulated using DDR memory in a remote NUMA node. In this paper, for the first time, we comprehensively evaluate a true CXL-ready system based on the latest 4th-generation Intel Xeon CPU with three CXL memory devices from different manufacturers. Specifically, we run a set of microbenchmarks not only to compare the performance of true CXL memory with that of emulated CXL memory but also to analyze the complex interplay between the CPU and CXL memory in depth. This reveals important differences between emulated CXL memory and true CXL memory, some of which will compel researchers to revisit the analyses and proposals from recent work. Next, we identify opportunities for memory-bandwidth-intensive applications to benefit from the use of CXL memory. Lastly, we propose a CXL-memory-aware dynamic page allocation policy, Caption to more efficiently use CXL memory as a bandwidth expander. We demonstrate that Caption can automatically converge to an empirically favorable percentage of pages allocated to CXL memory, which improves the performance of memory-bandwidth-intensive applications by up to 24% when compared to the default page allocation policy designed for traditional NUMA systems.

2. HPCA

   Shadow: Preventing row hammer in dram with intra-subarray row shuffling

   Minbok Wi, Jaehyun Park, Seoyoung Ko, Michael Jaemin Kim, Nam Sung Kim, Eojin Lee, and Jung Ho Ahn

   2023 IEEE International Symposium on High-Performance Computer Architecture (HPCA) , 2023

   Architecture Memory Security

   Abs DOI

   As Row Hammer (RH) attacks have been a critical threat to computer systems, numerous hardware-based (HWbased) RH mitigation strategies have been proposed. However, the advent of non-adjacent RH attacks and lower RH thresholds significantly increase the area and performance overhead of these prior solutions due to their conservative design characteristics.We propose a new in-DRAM RH protection solution named Shuffling Aggressor DRAM Rows (SHADOW). SHADOW dynamically randomizes DRAM row mapping information, preventing an attacker from targeting a specific victim row that may hold critical data. SHADOW is robust against non-adjacent RH attacks because it utilizes the in-DRAM row-shuffle technique. To realize the in-DRAM row-shuffle operation with low performance and energy overhead, we use a novel DRAM microarchitecture optimization technique. We also utilize the recently introduced JEDEC RFM interface to enable in-DRAM RH mitigation without any DRAM interface changes. By exploiting an additional DRAM row per subarray, SHADOW does not require costly SRAM- or CAM-based tracking structures other than intrinsic counters for the RFM interface. We demonstrate the strong probabilistic protection of SHADOW against RH attacks through adversarial pattern analysis and highlight the compelling performance, area, and energy overheads compared to those of state-of-the-art HW-based RH prevention solutions.

3. HPCA

   Rambda: Rdma-driven acceleration framework for memory-intensive µs-scale datacenter applications

   Yifan Yuan, Jinghan Huang, Yan Sun, Tianchen Wang, Jacob Nelson, Dan RK Ports, Yipeng Wang, Ren Wang, Charlie Tai, and Nam Sung Kim

   2023 IEEE International Symposium on High-Performance Computer Architecture (HPCA) , 2023

   Systems Network Accelerator

   Abs DOI

   Responding to the "datacenter tax" and "killer microseconds" problems for memory-intensive datacenter applications, diverse solutions including Smart NIC-based ones have been proposed. Nonetheless, they often suffer from high overhead of communications over network and/or PCIe links. To tackle the limitations of the current solutions, this paper proposes RAMBDA, a holistic network and architecture co-design solution that leverages current RDMA and emerging cache-coherent off-chip interconnect technologies. Specifically, RAMBDA consists of four hardware and software components: (1) unified abstraction of inter- and intra-machine communications synergistically managed by one-sided RDMA write and cache-coherent memory write; (2) efficient notification of requests to accelerators assisted by cache coherence; (3) cache-coherent accelerator architecture directly interacting with NIC; and (4) adaptive device-to-host data transfer for modern server memory systems comprising both DRAM and NVM exploiting state-of-the-art features in CPUs and PCIe. We prototype RAMBDA with a commercial system and evaluate three popular datacenter applications: (1) in-memory key-value store, (2) chain replication-based distributed transaction system, and (3) deep learning recommendation model inference. The evaluation shows that RAMBDA provides 30.1 69.1% lower latency, 0.2 2.5× throughput, and   3× higher energy efficiency than the current state-of-the-art solutions, including Smart NIC. For those cases where Rambda performs poorly, we also envision future architecture to improve it.

4. CAL

   Unleashing the potential of pim: Accelerating large batched inference of transformer-based generative models

   Jaewan Choi, Jaehyun Park, Kwanhee Kyung, Nam Sung Kim, and Jung Ho Ahn

   IEEE Computer Architecture Letters , 2023

   IEEE Computer Architecture Letters (CAL) Best Paper Award

   Abs DOI

   Transformer-based generative models, such as GPT, summarize an input sequence by generating key/value (KV) matrices through attention and generate the corresponding output sequence by utilizing these matrices once per token of the sequence. Both input and output sequences tend to get longer, which improves the understanding of contexts and conversation quality. These models are also typically batched for inference to improve the serving throughput. All these trends enable the models’ weights to be reused effectively, increasing the relative importance of sequence generation, especially in processing KV matrices through attention. We identify that the conventional computing platforms (e.g., GPUs) are not efficient at handling this attention part for inference because each request generates different KV matrices, it has a low operation per byte ratio regardless of the batch size, and the aggregate size of the KV matrices can even surpass that of the entire model weights. This motivates us to propose AttAcc, which exploits the fact that the KV matrices are written once during summarization but used many times (proportional to the output sequence length), each multiplied by the embedding vector corresponding to an output token. The volume of data entering/leaving AttAcc could be more than orders of magnitude smaller than what should be read internally for attention. We design AttAcc with multiple processing-in-memory devices, each multiplying the embedding vector with the portion of the KV matrices within the devices, saving external (inter-device) bandwidth and energy consumption.

5. ATC

   STYX: Exploiting SmartNIC capability to reduce datacenter memory tax

   Houxiang Ji, Mark Mansi, Yan Sun, Yifan Yuan, Jinghan Huang, Reese Kuper, Michael M Swift, and Nam Sung Kim

   2023 USENIX Annual Technical Conference (USENIX ATC 23) , 2023

   Systems Network Accelerator

   Abs DOI

   Memory optimization kernel features, such as memory deduplication, are designed to improve the overall efficiency of systems like datacenter servers, and they have proven to be effective. However, when invoked, these kernel features notably disrupt the execution of applications, intensively consuming the server CPU’s cycles and polluting its caches. To minimize such disruption, we propose STYX, a framework for offloading the intensive operations of these kernel features to SmartNIC (SNIC). STYX first RDMA-copies the server’s memory regions, on which these kernel features intend to operate, to an SNIC’s memory region, exploiting SNIC’s RDMA capability. Subsequently, leveraging SNIC’s (underutilized) compute capability, STYX makes the SNIC CPU perform the intensive operations of these kernel features. Lastly, STYX RDMA-copies their results back to a server’s memory region, based on which it performs the remaining operations of the kernel features. To demonstrate the efficacy of STYX, we re-implement two memory optimization kernel features in Linux:(1) memory deduplication (ksm) and (2) compressed cache for swap pages (zswap), using the STYX framework. We then show that a system with STYX provides a 55-89% decrease in 99th-percentile latency of co-running applications, compared to a system without STYX, while preserving the benefits of these kernel features.

6. MICRO

   How to kill the second bird with one ecc: The pursuit of row hammer resilient dram

   Michael Jaemin Kim, Minbok Wi, Jaehyun Park, Seoyoung Ko, Jaeyoung Choi, Hwayoung Nam, Nam Sung Kim, Jung Ho Ahn, and Eojin Lee

   2023

   Architecture Memory Security

   Abs DOI

   Error-correcting code (ECC) has been widely used in DRAM-based memory systems to address the exacerbating random errors following the fabrication process scaling. However, ECCs including the strong form of Chipkill have not been so effective against Row Hammer (RH), which incurs bursts of errors discretely corrupting the whole row beyond the ECC correction capability. We propose Cube, a novel chip-wise physical to DRAM address randomization scheme that leverages the abundant detection capability of on-die-ECC (OECC) and correction capability of Chipkill against RH. Cube allows for synergistic cooperation between ECC, probabilistic RH-protection schemes, and the system with minimal to no modification for each. First, Cube scrambles the rows of each chip using a boot-time key in a way that distributes RH victims to multiple Chipkill codewords. Second, Cube utilizes the newly observed distinct RH error characteristics from real DRAM chips, to swiftly diagnose the RH victim rows using the error profile from OECC scrubbing, and even correct it leveraging Chipkill. When combined, Cube decreases the failure probability of PARA and a state-of-the-art RH protection scheme SRS by up to 10− 25. At a target failure probability of 10− 10 per year on a DDR5 rank under the RH threshold of 2K, Cube reduces the performance and table size overhead of SRS by up to 24.3% and 39.9%, respectively.

7. CAL

   X-ray: Discovering dram internal structure and error characteristics by issuing memory commands

   Hwayong Nam, Seungmin Baek, Minbok Wi, Michael Jaemin Kim, Jaehyun Park, Chihun Song, Nam Sung Kim, and Jung Ho Ahn

   IEEE Computer Architecture Letters , 2023

   Abs DOI

8. HotOS

   Towards a Manageable Intra-Host Network

   Xinhao Kong, Jiaqi Lou, Wei Bai, Nam Sung Kim, and Danyang Zhuo

   2023

   Systems Network

   Abs DOI

   Intra-host networks, including heterogeneous devices and interconnect fabrics, have become increasingly complex and crucial. However, intra-host networks today do not provide sufficient manageability. This prevents data center operators from running a reliable and efficient end-to-end network, especially for multi-tenant clouds. In this paper, we analyze the main manageability deficiencies of intra-host networks and argue that a systematic solution should be implemented to bridge this function gap. We propose two key building blocks for a manageable intra-host network: a fine-grained monitoring system and a holistic resource manager. We discuss the research questions associated with realizing these two building blocks.

9. IISWC

   Making sense of using a smartnic to reduce datacenter tax from slo and tco perspectives

   Jinghan Huang, Jiaqi Lou, Yan Sun, Tianchen Wang, Eun Kyung Lee, and Nam Sung Kim

   2023 IEEE International Symposium on Workload Characterization (IISWC) , 2023

   Best Paper Runner-up Award

   Systems Network Accelerator

   Abs DOI

   The speed of network interfaces has rapidly increased, while the performance and energy efficiency of CPUs have not, due to the demise of Dennard scaling. As a result, functions processing network packets have become responsible for a rapidly increasing portion of the datacenter tax. To tackle this problem, the industry has developed SmartNICs (SNICs) integrating conventional NICs with inexpensive and energy-efficient processors that can efficiently execute functions widely used by network-intensive datacenter applications. With such processors, the SNICs promise to reduce the total cost of ownership (TCO) for datacenters by increasing energy efficiency of servers and/or decreasing the number of expensive server CPU cores. In this paper, to make sense of using SNICs, we focus on analyzing energy efficiency of a server with an SNIC, especially under service level objective (SLO) constraints which matter for many datacenter applications. To this end, we not only measure the system-wide power consumption of a server but also devise a custom hardware setup that allows us to isolate the power consumption of the SNIC from that of the server. This helps us better understand the impact of using SNICs on energy efficiency of servers. Second, we take popular TCP/UDP, DPDK, and RDMA-based functions, and prepare them to run on an SNIC processor and a server CPU. Then, we measure maximum throughput, tail latency, system-wide energy efficiency of executing the functions on the SNIC processor and the server CPU, respectively. Lastly, based on analyses of the measurements, we make five key observations and three strategies to better use and design SmartNICs in the future.

10. ISPASS

    Analyzing Energy Efficiency of a Server with a SmartNIC under SLO Constraints

    Jinghan Huang, Jiaqi Lou, Yan Sun, Tianchen Wang, Eun Kyung Lee, and Nam Sung Kim

    2023 IEEE International Symposium on Performance Analysis of Systems and Software (ISPASS) , 2023

    System Network Accelerator

    Abs DOI

    Network processing has become a fast-increasing portion of the datacenter tax. To tackle this problem, the industry introduced SmartNICs comprising energy-efficient CPU cores, FPGA and/or accelerators. With architectures optimized for network-intensive applications, they promise to notably reduce the total cost of ownership (TCO) of datacenters primarily through improving energy efficiency. As such, this paper sets out to investigate the overall energy efficiency of an Intel Xeon CPU based server with an NVIDIA BlueField-2 SmartNIC, especially under service level objective (SLO) constraints which matter for many datacenter services.

11. CAL

    Nohammer: Preventing row hammer with last-level cache management

    Seunghak Lee, Ki-Dong Kang, Gyeongseo Park, Nam Sung Kim, and Daehoon Kim

    IEEE Computer Architecture Letters , 2023

    Abs DOI

    Row Hammer (RH) is a circuit-level phenomenon where repetitive activation of a DRAM row causes bit-flips in adjacent rows. Prior studies that rely on extra refreshes to mitigate RH vulnerability demonstrate that bit-flips can be prevented effectively. However, its implementation is challenging due to the significant performance degradation and energy overhead caused by the additional extra refresh for the RH mitigation. To overcome challenges, some studies propose techniques to mitigate the RH attack without relying on extra refresh. These techniques include delaying the activation of an aggressor row for a certain amount of time or swapping an aggressor row with another row to isolate it from victim rows. Although such techniques do not require extra refreshes to mitigate RH, the activation delaying technique may result in high-performance degradation in false-positive cases, and the swapping technique requires high storage overheads to track swap information. We propose NoHammer, an efficient RH mitigation technique to prevent the bit-flips caused by the RH attack by utilizing Last-Level Cache (LLC) management. NoHammer temporarily extends the associativity of the cache set that is being targeted by utilizing another cache set as the extended set and keeps the cache lines of aggressor rows on the extended set under the eviction-based RH attack. Along with the modification of the LLC replacement policy, NoHammer ensures that the aggressor row’s cache lines are not evicted from the LLC under the RH attack. In our evaluation, we demonstrate that NoHammer gives 6% higher performance than a baseline without any RH mitigation technique by replacing excessive cache misses caused by the RH attack with LLC hits through sophisticated LLC management, while requiring 45% less storage than prior proposals.

12. Defensive ml: Defending architectural side-channels with adversarial obfuscation

    Hyoungwook Nam, Raghavendra Pradyumna Pothukuchi, Bo Li, Nam Sung Kim, and Josep Torrellas

    arXiv preprint arXiv:2302.01474 , 2023

    Abs

    Side-channel attacks that use machine learning (ML) for signal analysis have become prominent threats to computer security, as ML models easily find patterns in signals. To address this problem, this paper explores using Adversarial Machine Learning (AML) methods as a defense at the computer architecture layer to obfuscate side channels. We call this approach Defensive ML, and the generator to obfuscate signals, defender. Defensive ML is a workflow to design, implement, train, and deploy defenders for different environments. First, we design a defender architecture given the physical characteristics and hardware constraints of the side-channel. Next, we use our DefenderGAN structure to train the defender. Finally, we apply defensive ML to thwart two side-channel attacks: one based on memory contention and the other on application power. The former uses a hardware defender with ns-level response time that attains a high level of security with half the performance impact of a traditional scheme; the latter uses a software defender with ms-level response time that provides better security than a traditional scheme with only 70% of its power overhead.

13. PATENT

    Computer architecture having selectable parallel and serial communication channels between processors and memory

    Hao Wang and Nam Sung Kim

    2023

    Abs

    G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, eg interface arrangements

14. CAL

    LADIO: Leakage-aware direct I/O for I/O-intensive workloads

    Ipoom Jeong, Jiaqi Lou, Yongseok Son, Yongjoo Park, Yifan Yuan, and Nam Sung Kim

    IEEE Computer Architecture Letters , 2023

    Abs DOI

    The advancement in I/O technology has posed an unprecedented demand for high-performance processing on I/O data, leading to the development of Data Direct I/O (DDIO) technology. DDIO improves I/O processing efficiency by directly injecting all inbound I/O data into the last-level cache (LLC) in cooperation with any type of I/O device. Nonetheless, in certain scenarios with more than one I/O applications, DDIO may have sub-optimal performance caused by interference inside the LLC, resulting in the degradation of system performance. Especially, in this paper, we demonstrate that storage I/O on modern high-performance NVMe SSDs hardly benefits from DDIO, sometimes causing inefficient use of the shared LLC due to the “leaky DMA problem”. To address this problem, we propose LADIO, an adaptive approach that mitigates inter-application interference by dynamically controlling the DDIO functionality and reallocating LLC ways based on the leakage and locality of storage I/O data, respectively. In scenarios with heavy I/O interference, LADIO improves the throughput of network-intensive applications by 20% while maintaining that of storage-intensive applications.

15. ISCA

    Mesa: Microarchitecture extensions for spatial architecture generation

    Dong Kai Wang, Jiaqi Lou, Naiyin Jin, Edwin Mascarenhas, Rohan Mahapatra, Sean Kinzer, Soroush Ghodrati, Amir Yazdanbakhsh, Hadi Esmaeilzadeh, and Nam Sung Kim

    2023

    System Network Storage CPU

    Abs DOI

    Modern heterogeneous CPUs incorporate hardware accelerators to enable domain-specialized execution and achieve improved efficiency. A well-known class among them, spatial accelerators, are designed with reconfigurability to accelerate a wide range of compute-heavy and data-parallel applications. Unlike CPU cores, however, they tend to require specialized compilers and software stacks, libraries, or languages to operate and cannot be utilized with ease by all applications. As a result, the accelerator’s large pool of compute and memory resources sit wastefully idle when it is not explicitly programmed. Our goal is to dismantle this CPU-accelerator barrier by monitoring CPU threads for acceleration opportunities during execution and, if viable, dynamically reconfigure the accelerator to allow transparent offloading. We develop MESA (Microarchitecture Extensions for Spatial Architecture Generation), a hardware block on the CPU that translates machine code to build an accelerator configuration specialized for the running program. While such a dynamic translation/reconfiguration approach is challenging, it has a key advantage over ahead-of-time compilers: access to runtime information, revealing not only dynamic dependencies but also performance characteristics. MESA maintains a real-time performance model of the program mapped on the accelerator in the form of a spatial dataflow graph with nodes weighted by operation latency and edges weighted by data transfer latency. Features of this dataflow graph are continuously updated with runtime information captured by performance counters, allowing a feedback loop of optimization, reconfiguration, and acceleration. This performance model allows MESA to identify the accelerator’s critical paths and pinpoint its bottlenecks, upon which we implement in hardware a data-driven instruction mapping algorithm that locally minimizes latency. Backed by a synthesized RTL implementation, we evaluate the feasibility of our microarchitectural solution with different accelerator configurations. Across the Rodinia benchmarks, results demonstrate an average 1.3× speedup in performance and 1.8× gain in energy efficiency against a multicore CPU baseline.

16. Special issue on emerging system interconnects

    John Kim and Nam Sung Kim

    IEEE Micro , 2023

    Abs DOI

    This special issue on emerging system interconnects addresses recent advances in system interconnect technologies, with special emphasis on compute express link (cxl) as it becomes more widely adopted in the industry.

17. PATENT

    Dataflow-based general-purpose processor architectures

    Nam Sung Kim and Dong Kai Wang

    2023

    Abs DOI

    2023-05-05 Assigned to THE BOARD OF TRUSTEES OF THE UNIVERSITY OF ILLINOIS reassignment THE BOARD OF TRUSTEES OF THE UNIVERSITY OF ILLINOIS ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: KIM, NAM SUNG, WANG, DONG KAI

18. Triple-A: Early Operand Collector Allocation for Maximizing GPU Register Bank Utilization

    Ipoom Jeong, Eunbi Jeong, Nam Sung Kim, and Myung Kuk Yoon

    IEEE Embedded Systems Letters , 2023

    Abs DOI

    Recent GPUs provisioned with large register files (RFs) cannot fully utilize the bandwidth between the RFs and execution pipelines, as the current policy for allocating operand (OP) collectors defers the RF accesses until all the source OPs become ready. To tackle this issue, this letter introduces a new OP collector allocation mechanism called Triple-A. Triple-A comprises four key operations. First, Triple-A proactively allocates an OP collector (OC) to a warp instruction even if one of its source OPs is not yet ready, taking advantage of GPUs’ in-order execution. Second, a computation result can be directly forwarded to an early allocated OC along with a data dependence, reducing OP loading time from the RFs. Third, Triple-A bypasses RF write operations if the forwarded data is not consumed by any other instruction. Finally, the early allocation is further enhanced with latency-aware optimization, alleviating the potential performance degradation caused by allocating OCs aggressively. Together, these techniques synergistically improve the register bank utilization, demonstrating a 14.1% improvement in performance and an 11.8% reduction in RF energy consumption compared to the state-of-the-art GPUs.

2022

1. MLSys

   Bns-gcn: Efficient full-graph training of graph convolutional networks with partition-parallelism and random boundary node sampling

   Cheng Wan, Youjie Li, Ang Li, Nam Sung Kim, and Yingyan Lin

   Proceedings of Machine Learning and Systems , 2022

   Abs DOI

   Graph Convolutional Networks (GCNs) have emerged as the state-of-the-art method for graph-based learning tasks. However, training GCNs at scale is still challenging, hindering both the exploration of more sophisticated GCN architectures and their applications to real-world large graphs. While it might be natural to consider graph partition and distributed training for tackling this challenge, this direction has only been slightly scratched the surface in the previous works due to the limitations of existing designs. In this work, we first analyze why distributed GCN training is ineffective and identify the underlying cause to be the excessive number of boundary nodes of each partitioned subgraph, which easily explodes the memory and communication costs for GCN training. Furthermore, we propose a simple yet effective method dubbed BNS-GCN that adopts random Boundary-Node-Sampling to enable efficient and scalable distributed GCN training. Experiments and ablation studies consistently validate the effectiveness of BNS-GCN, eg, boosting the throughput by up to 16.2× and reducing the memory usage by up to 58%, while maintaining a full-graph accuracy. Furthermore, both theoretical and empirical analysis show that BNS-GCN enjoys a better convergence than existing sampling-based methods. We believe that our BNS-GCN has opened up a new paradigm for enabling GCN training at scale. The code is available at https://github. com/RICE-EIC/BNS-GCN.

2. Pipegcn: Efficient full-graph training of graph convolutional networks with pipelined feature communication

   Cheng Wan, Youjie Li, Cameron R Wolfe, Anastasios Kyrillidis, Nam Sung Kim, and Yingyan Lin

   arXiv preprint arXiv:2203.10428 , 2022

   Abs

   Graph Convolutional Networks (GCNs) is the state-of-the-art method for learning graph-structured data, and training large-scale GCNs requires distributed training across multiple accelerators such that each accelerator is able to hold a partitioned subgraph. However, distributed GCN training incurs prohibitive overhead of communicating node features and feature gradients among partitions for every GCN layer during each training iteration, limiting the achievable training efficiency and model scalability. To this end, we propose PipeGCN, a simple yet effective scheme that hides the communication overhead by pipelining inter-partition communication with intra-partition computation. It is non-trivial to pipeline for efficient GCN training, as communicated node features/gradients will become stale and thus can harm the convergence, negating the pipeline benefit. Notably, little is known regarding the convergence rate of GCN training with both stale features and stale feature gradients. This work not only provides a theoretical convergence analysis but also finds the convergence rate of PipeGCN to be close to that of the vanilla distributed GCN training without any staleness. Furthermore, we develop a smoothing method to further improve PipeGCN’s convergence. Extensive experiments show that PipeGCN can largely boost the training throughput (1.7x 28.5x) while achieving the same accuracy as its vanilla counterpart and existing full-graph training methods. The code is available at https://github.com/RICE-EIC/PipeGCN.

3. Aquabolt-XL HBM2-PIM LPDDR5-PIM with in-memory processing and AXDIMM with acceleration buffer

   Jin Hyun Kim, Shin-Haeng Kang, Sukhan Lee, Hyeonsu Kim, Yuhwan Ro, Seungwon Lee, David Wang, Jihyun Choi, Jinin So, YeonGon Cho, JoonHo Song, Jeonghyeon Cho, Kyomin Sohn, and Nam Sung Kim

   IEEE Micro , 2022

   Abs DOI

   Processing-in-memory (PIM) has been proposed to improve the performance of bandwidth-intensive workloads as well as save energy due to reduced compute-memory data movement. To realize PIM, programmable computing units were integrated with memory cores on an HBM2 device to enable parallel processing and minimize data movement. A graphics processing unit (GPU) system equipped with Samsung Aquabolt-XL HBM2-PIM devices improved microkernel general matrix–vector multiplication and speech recognition applications by 8.9× and 3.5×, respectively, and reduced energy consumption by over 60%. In a Xilinx AlveoU280 system, microkernel GEMV and ADD workload performances improved by 2.8×, and long short-term memory workload improved by 2.54×. Simulations show that a performance gain of over 2.3× may be attained in a system with LP5-PIM for certain transformer-based speech recognition with an energy reduction of 86%. In addition, AXDIMM, a DIMM-level PIM with acceleration buffers, exhibits an 80% performance improvement and a 42.6% energy savings over a regular RDIMM system.

4. NSDI

   Unlocking the power of inline Floating-Point operations on programmable switches

   Yifan Yuan, Omar Alama, Jiawei Fei, Jacob Nelson, Dan RK Ports, Amedeo Sapio, Marco Canini, and Nam Sung Kim

   19th USENIX Symposium on Networked Systems Design and Implementation (NSDI 22) , 2022

   Systems Network Accelerator

   Abs

   The advent of switches with programmable dataplanes has enabled the rapid development of new network functionality, as well as providing a platform for acceleration of a broad range of application-level functionality. However, existing switch hardware was not designed with application acceleration in mind, and thus applications requiring operations or datatypes not used in traditional network protocols must resort to expensive workarounds. Applications involving floating point data, including distributed training for machine learning and distributed query processing, are key examples.

5. Harmony: Overcoming the hurdles of GPU memory capacity to train massive DNN models on commodity servers

   Youjie Li, Amar Phanishayee, Derek Murray, Jakub Tarnawski, and Nam Sung Kim

   48th International Conference on Very Large Databases , 2022

   Systems ML/AI

   Abs DOI

   Deep neural networks (DNNs) have grown exponentially in size over the past decade, leaving only those who have massive datacenter-based resources with the ability to develop and train such models. One of the main challenges for the long tail of researchers who might have only limited resources (e.g., a single multi-GPU server) is limited GPU memory capacity compared to model size. The problem is so acute that the memory requirement of training massive DNN models can often exceed the aggregate capacity of all available GPUs on a single server; this problem only gets worse with the trend of ever-growing model sizes. Current solutions that rely on virtualizing GPU memory (by swapping to/from CPU memory) incur excessive swapping overhead. In this paper, we present a new training framework, Harmony, and advocate rethinking how DNN frameworks schedule computation and move data to push the boundaries of training massive models efficiently on a single commodity server. Across various massive DNN models, Harmony is able to reduce swap load by up to two orders of magnitude and obtain a training throughput speedup of up to 7.6x over highly optimized baselines with virtualized memory.

6. MICRO

   IDIO: Network-driven inbound network data orchestration on server processors

   Mohammad Alian, Siddharth Agarwal, Jongmin Shin, Neel Patel, Yifan Yuan, Daehoon Kim, Ren Wang, and Nam Sung Kim

   2022 55th IEEE/ACM International Symposium on Microarchitecture (MICRO) , 2022

   Architecture CPU Network

   Abs DOI

   High-bandwidth network interface cards (NICs), each capable of transferring 100s of Gigabits per second, are making inroads into the servers of next-generation datacenters. Such unprecedented data delivery rates impose immense pressure, especially on the server’s memory subsystem, as NICs first transfer network data to DRAM before processing. To alleviate the pressure, the cache hierarchy has evolved, supporting a direct data I/O (DDIO) technology to directly place network data in the last-level cache (LLC). Subsequently, various policies have been explored to manage such LLC and have proven to effectively reduce service latency and memory bandwidth consumption of network applications. However, the more recent evolution of the cache hierarchy decreased the size of LLC per core but significantly increased that of midlevel cache (MLC) with a non-inclusive policy. This calls for a re-examination of the aforementioned DDIO technology and management policies. In this paper, first, we identify three shortcomings of the current static data placement policy placing network data to LLC first and the non-inclusive policy with a commercial server system: (1) ineffectively using large MLC, (2) suffering from high rates of writebacks from MLC to LLC, and (3) breaking the isolation between application and network data enforced by limiting cache ways for DDIO. Second, to tackle the three shortcomings, we propose an intelligent direct I/O (IDIO) technology that extends DDIO to MLC and provides three synergistic mechanisms: (1) self-invalidating I/O buffer, (2) network-driven MLC prefetching, and (3) selective direct DRAM access. Our detailed experiments using a full-system simulator — capable of running modern DPDK userspace network functions while sustaining 100Gbps + network bandwidth — show that IDIO significantly reduces data movement (up to 84% MLC and LLC writeback reduction), provides LLC isolation (up to 22% performance improvement), and improves tail latency (up to 38% reduction in 99th latency) for receive-intensive network applications.

7. An FPGA-based RNN-T Inference Accelerator with PIM-HBM

   Shinhaeng Kang, Sukhan Lee, Byeongho Kim, Hweesoo Kim, Kyomin Sohn, Nam Sung Kim, and Eojin Lee

   2022

   Architecture Memory Accelerator ML/AI

   Abs DOI

   In this paper, we implemented a world-first RNN-T inference accelerator using FPGA with PIM-HBM that can multiply the internal bandwidth of the memory. The accelerator offloads matrix-vector multiplication (GEMV) operations of LSTM layers in RNN-T into PIM-HBM, and PIM-HBM reduces the execution time of GEMV significantly by exploiting HBM internal bandwidth. To ensure that the memory commands are issued in a pre-defined order, which is one of the most important constraints in exploiting PIM-HBM, we implement a direct memory access (DMA) module and change configuration of the on-chip memory controller by utilizing the flexibility and reconfigurability of the FPGA. In addition, we design the other hardware modules for acceleration such as non-linear functions (i.e., sigmoid and hyperbolic tangent), element-wise operation, and ReLU module, to operate these compute-bound RNN-T operations on FPGA. For this, we prepare FP16 quantized weight and MLPerf input datasets, and modify the PCIe device driver and C++ based control codes. On our evaluation, our accelerator with PIM-HBM reduces the execution time of RNN-T by 2.5 × on average with 11.09% reduced LUT size and improves energy efficiency up to 2.6 × compared to the baseline.

8. Asap: architecture support for asynchronous persistence

   Ahmed Abulila, Izzat El Hajj, Myoungsoo Jung, and Nam Sung Kim

   2022

   Architecture CPU Memory

   Abs DOI

   Supporting atomic durability of updates for persistent memories is typically achieved with Write-Ahead Logging (WAL). WAL flushes log entries to persistent memory before making the actual data persistent to ensure that a consistent state can be recovered if a crash occurs. Performing WAL in hardware is attractive because it makes most aspects of log management transparent to software, and it completes log persist operations (LPOs) and data persist operations (DPOs) in the background, overlapping them with the execution of other instructions. Prior hardware logging solutions commit atomic regions synchronously. That is, once the end of a region is reached, all outstanding persist operations required for the region to commit must complete before instruction execution may proceed. For undo logging, LPOs and DPOs are both performed synchronously to ensure that the region commits synchronously. For redo logging, DPOs can be performed asynchronously, but LPOs are performed synchronously to ensure that the region commits synchronously. In both cases, waiting for synchronous persist operations (LPO or DPO) at the end of an atomic region causes atomic regions to incur high latency. To tackle this limitation, we propose ASAP, a hardware logging solution that allows atomic regions to commit asynchronously. That is, once the end of an atomic region is reached, instruction execution may proceed without waiting for outstanding persist operations to complete. As such, both LPOs and DPOs can be performed asynchronously. The challenge with allowing atomic regions to commit asynchronously is that it can lead to control and data dependence violations in the commit order of the atomic regions, leaving data in an unrecoverable state in case of a crash. To address this issue, ASAP tracks and enforces control and data dependencies between atomic regions in hardware to ensure that the regions commit in the proper order. Our evaluation shows that ASAP outperforms the state-of-the-art hardware undo and redo logging techniques by 1.41X and 1.53X, respectively, while achieving 0.96X the ideal performance when no persistence is enforced, at a small hardware cost (< 3%). ASAP also reduces memory traffic to persistent memory by 38% and 48%, compared with the state-of-the-art hardware undo and redo logging techniques, respectively. ASAP is robust against increasing persistent memory latency, making it suitable for both fast and slow persistent memory technologies.

9. Rethinking DRAM’s page mode with STT-MRAM

   Byoungchan Oh, Nilmini Abeyratne, Nam Sung Kim, Jeongseob Ahn, Ronald G Dreslinski, and Trevor Mudge

   IEEE Transactions on Computers , 2022

   Abs DOI

   Spin torque magnetic random access memory (STT-MRAM) is a promising candidate for drop-in replacement for DRAM-based main memory because of its higher energy efficiency and similar latency to dynamic random access memory (DRAM). However, simply replacing DRAM with STT-MRAM without optimizations severely limits STT-MRAM from exploiting its full potential. STT-MRAM employs costly sense amplifiers that demand an order of magnitude more area and power than DRAM. To manage the high cost, STT-MRAM shares one sense amplifier across multiple bit-lines, exploiting the non-destructive nature of its read operation. This sense amplifier sharing reduces the size of row buffers; as a result, it incurs higher activation energy and lower performance. Other issues arise if STT-MRAM is required to be compatible with DRAM interfaces and policies. To address these challenges in a cost-effective manner, we propose STT-MRAM ARchiTecture supporting smart activation and sensing (SMART) that, unlike DRAM and conventional STT-MRAM, waits to do bit-line sensing until after receiving a column access command instead of a row activation command. This results in several benefits: larger pages, fewer sense amplifiers, lower activation power, higher bank-level parallelism, shorter latency, fewer address pins, and more efficient repairing of defective columns than conventional STT-MRAM. Our evaluation shows that SMART consumes lower energy while providing higher performance than conventional STT-MRAM and DRAM. Additionally, SMART consumes less area compared to conventional STT-MRAM.

10. PATENT

    Application-transparent near-memory processing architecture with memory channel network

    Nam Sung Kim and Mohammad Alian

    2022

    Abs

    (57) ABSTRACT A system includes a printed circuit board (PCB) on which is disposed memory components and a processor disposed on the PCB and coupled between the memory components and a host memory controller. The processor comprises a memory channel network (MCN) memory controller to handle memory requests associated with the memory com ponents; a local buffer; and a core coupled to the MCN memory controller and the local buffer. The core executes an operating system (OS) running a network software layer and a distributed computing framework; and an MCN driver to: receive a network packet from the network software layer; store the network packet in the local buffer; and assert a transmit polling field of the local buffer to signal to the host (Continued)

11. PATENT

    Network-driven packet context-aware power management for client-server architecture

    Nam Sung Kim and Mohammad Alian

    2022

    Abs

    A server system including an enhanced Network Interface Controller (NIC) within a client-server architecture is pro vided. The server system includes a memory for storing data from one or more network packets and one or more proces sors for processing network requests based on the one or more network packets. The enhanced NIC is configured to receive the one or more network packets and transfer the data from the one or more network packets to the memory.During a latency period defined from the time required to transfer the network packet to memory, the enhanced NIC performs a Network-driven, packet Context Aware Power (NCAP) management process in order to actively transition a power management state of the one or more processors to

12. Coordinated Science Laboratory 70th Anniversary Symposium: The Future of Computing

    Klara Nahrstedt, Naresh Shanbhag, Vikram Adve, Nancy Amato, Romit Roy Choudhury, Carl Gunter, Nam Sung Kim, Olgica Milenkovic, Sayan Mitra, Lav Varshney, Yurii Vlasov, Sarita Adve, Rashid Bashir, Andreas Cangellaris, James DiCarlo, Katie Driggs-Campbell, Nick Feamster, Mattia Gazzola, Karrie Karahalios, Sanmi Koyejo, Paul Kwiat, Bo Li, Negar Mehr, Ravish Mehra, Andrew Miller, Daniela Rus, Alex Schwing, and Anshumali Shrivastava

    arXiv preprint arXiv:2210.08974 , 2022
13. HAMS: Hardware Automated Memory-over-Storage for Large-scale Memory Expansion

    Jie Zhang, Miryeong Kwon, Donghyun Gouk, Sungjoon Koh, Nam Sung Kim, Mahmut Taylan Kandemir, and Myoungsoo Jung

    13rd Annual Non-Volatile Memories Workshop (NVMW), 2022 , 2022

    Architecture GPU

    Abs

    II. RELATED WORK AND CHALLENGES Baseline architecture for persistent memory expansion. One common solution to build a large and scalable, yet persistent memory space is to use a type of NVDIMM (ie, NVDIMM-N) together with SSD and memory-mapped files (MMFs), which can be implemented in an OS memory manager or a file system (cf. Figure 1). Figure 2 illustrates the software support and storage stack that user applications require for expanding NVDIMM with SSD. The memory-mapped file (MMF) module in Linux, also referred to as mmap, can be used to expand the persistent memory space of NVDIMM with SSD. If a process calls mmap with a file descriptor (fd) for SSD (1), the MMF creates a new mapping in its process address space, represented by a memory management structure (mm_struct), by allocating a virtual memory area (VMA) to the structure (2). In other words, the MMF links fd to the VMA, by establishing a mapping between the process memory and the target file. When the process accesses the memory designated by the VMA (3 4 5), this triggers a page fault (if the data is not available in NVDIMM). When a page fault occurs, the page fault handler is invoked and allocates a new page to the VMA. Since the VMA is linked to the target file, the page fault handler retrieves the file metadata (inode) associated with fd and acquires a lock for its access (6). The MMU interacts with a fault handler of the file system to read a page from the SSD (7 8 9). Once the actual data is loaded into a new region of the allocated page memory,

2021

1. ISCA

   Hardware architecture and software stack for PIM based on commercial DRAM technology: Industrial product

   Sukhan Lee, Shin-haeng Kang, Jaehoon Lee, Hyeonsu Kim, Eojin Lee, Seungwoo Seo, Hosang Yoon, Seungwon Lee, Kyounghwan Lim, Hyunsung Shin, Jinhyun Kim, O Seongil, Anand Iyer, David Wang, Kyomin Sohn, and Nam Sung Kim

   2021 ACM/IEEE 48th Annual International Symposium on Computer Architecture (ISCA) , 2021

   IEEE Micro Top Picks Honorable Mention

   Abs DOI

2. 25.4 a 20nm 6gb function-in-memory dram based on hbm2 with a 1.2 tflops programmable computing unit using bank-level parallelism for machine learning applications

   Young-Cheon Kwon, Suk Han Lee, Jaehoon Lee, Sang-Hyuk Kwon, Je Min Ryu, Jong-Pil Son, O Seongil, Hak-Soo Yu, Haesuk Lee, Soo Young Kim, Youngmin Cho, Jin Guk Kim, Jongyoon Choi, Hyun-Sung Shin, Jin Kim, BengSeng Phuah, HyoungMin Kim, Myeong Jun Song, Ahn Choi, Daeho Kim, SooYoung Kim, Eun-Bong Kim, David Wang, Shinhaeng Kang, Yuhwan Ro, Seungwoo Seo, JoonHo Song, Jaeyoun Youn, Kyomin Sohn, and Nam Sung Kim

   2021 IEEE International Solid-State Circuits Conference (ISSCC) , 2021

   World’s First Professing-In-Memory Chip from a Major Memory Manufacturer

   Circuit Memory

   Abs DOI

3. Near-memory processing in action: Accelerating personalized recommendation with axdimm

   Liu Ke, Xuan Zhang, Jinin So, Jong-Geon Lee, Shin-Haeng Kang, Sukhan Lee, Songyi Han, YeonGon Cho, Jin Hyun Kim, Yongsuk Kwon, KyungSoo Kim, Jin Jung, Ilkwon Yun, Sung Joo Park, Hyunsun Park, Joonho Song, Jeonghyeon Cho, Kyomin Sohn, Nam Sung Kim, and Hsien-Hsin S Lee

   IEEE Micro , 2021

   Abs DOI

4. ISCA

   Don’t forget the I/O when allocating your LLC

   Yifan Yuan, Mohammad Alian, Yipeng Wang, Ren Wang, Ilia Kurakin, Charlie Tai, and Nam Sung Kim

   2021 ACM/IEEE 48th Annual International Symposium on Computer Architecture (ISCA) , 2021

   Architecture CPU Network

   Abs DOI

   In modern server CPUs, last-level cache (LLC) is a critical hardware resource that exerts significant influence on the performance of the workloads, and how to manage LLC is a key to the performance isolation and QoS in the cloud with multi-tenancy. In this paper, we argue that in addition to CPU cores, high-speed I/O is also important for LLC management. This is because of an Intel architectural innovation – Data Direct I/O (DDIO) – that directly injects the inbound I/O traffic to (part of) the LLC instead of the main memory. We summarize two problems caused by DDIO and show that (1) the default DDIO configuration may not always achieve optimal performance, (2) DDIO can decrease the performance of non-I/O workloads that share LLC with it by as high as 32%.We then present, the first LLC management mechanism that treats the I/O as the first-class citizen. Iat monitors and analyzes the performance of the core/LLC/DDIO using CPU’s hardware performance counters and adaptively adjusts the number of LLC ways for DDIO or the tenants that demand more LLC capacity. In addition, Iat dynamically chooses the tenants that share its LLC resource with DDIO to minimize the performance interference by both the tenants and the I/O. Our experiments with multiple microbenchmarks and real-world applications demonstrate that with minimal overhead, Iat can effectively and stably reduce the performance degradation caused by DDIO.

5. 25.2 a 16gb sub-1v 7.14 gb/s/pin lpddr5 sdram applying a mosaic architecture with a short-feedback 1-tap dfe an fss bus with low-level swing and an adaptively controlled body biasing in a 3 rd-generation 10nm dram

   Yong-Hun Kim, Hyung-Jin Kim, Jaemin Choi, Min-Su Ahn, Dongkeon Lee, Seung-Hyun Cho, Dong-Yeon Park, Young-Jae Park, Min-Soo Jang, Yong-Jun Kim, Jinyong Choi, Sung-Woo Yoon, Jae-Woo Jung, Jae-Koo Park, Jae-Woo Lee, Dae-Hyun Kwon, Hyung-Seok Cha, Si-Hyeong Cho, Seong-Hoon Kim, Jihwa You, Kyoung-Ho Kim, Dae-Hyun Kim, Byung-Cheol Kim, Young-Kwan Kim, Jun-Ho Kim, Seouk-Kyu Choi, Chan-Young Kim, Byong-Wook Na, Hye-In Choi, Reum Oh, Jeong-Don Ihm, Seung-Jun Bae, Nam Sung Kim, and Jung-Bae Lee

   2021 IEEE International Solid-State Circuits Conference (ISSCC) , 2021

   Abs DOI

6. Nmap: Power management based on network packet processing mode transition for latency-critical workloads

   Ki-Dong Kang, Gyeongseo Park, Hyosang Kim, Mohammad Alian, Nam Sung Kim, and Daehoon Kim

   2021

   Abs DOI

7. Dml: Dynamic partial reconfiguration with scalable task scheduling for multi-applications on fpgas

   Ashutosh Dhar, Edward Richter, Mang Yu, Wei Zuo, Xiaohao Wang, Nam Sung Kim, and Deming Chen

   IEEE Transactions on Computers , 2021

   Abs DOI

8. ISCA

   Revamping storage class memory with hardware automated memory-over-storage solution

   Jie Zhang, Miryeong Kwon, Donghyun Gouk, Sungjoon Koh, Nam Sung Kim, Mahmut Taylan Kandemir, and Myoungsoo Jung

   2021 ACM/IEEE 48th Annual International Symposium on Computer Architecture (ISCA) , 2021

   Abs DOI

9. PATENT

   Network-centric architecture and algorithms to accelerate distributed training of neural networks

   Nam Sung Kim, LI Youjie, and Alexander Gerhard Schwing

   2021

   Architecture Accelerator Systems ML/AI

   Abs DOI

   A distributed network includes a first group of computing devices. Each computing device is to be coupled to two neighbor computing devices of the first group of computing device and is to:(i) aggregate gradient values received from a first neighbor computing device with local gradient values to generate a partial aggregate of gradient values that are to train a neural network model;(ii) transfer the partial aggre gate of gradient values to a second neighbor computing device; and repeat (i) and (ii) until a first aggregate of gradient values from the first group of computing devices is buffered at a first computing device of the first group of computing devices. The first computing device is to transfer the first aggregate of gradient values to a second group of computing devices of the distributed network for further aggregation.

10. ASPLOS

    Diag: a dataflow-inspired architecture for general-purpose processors

    Dong Kai Wang and Nam Sung Kim

    26th ACM International Conference on Architectural Support for Programming Languages and Operating Systems , 2021

    Microarchitecture CPU

    Abs DOI

11. Osc: An online self-configuring big data framework for optimization of qos

    Zhendong Bei, Nam Sung Kim, Kai HWang, and Zhibin Yu

    IEEE Transactions on Computers , 2021

    Abs DOI

    Big-data frameworks such as MapReduce/Hadoop or Spark have many performance-critical configuration parameters which may interact with each other in a complex way. Their optimal values for an application on a given cluster are affected by not only the application itself but also its input data. This makes offline auto-configuration approaches hard to be used in practice because the input data of an application may change at each run. To address this issue, we propose an Online Self-Configuring (OSC) approach that automatically determines the optimal parameter values for a given application. OSC synergistically integrates three key techniques. First, OSC leverages ensemble learning to build a precise performance model for a given application. Second, it quantifies the importance of the parameters and interaction intensity between them to accelerate the genetic algorithm for searching optimal configuration parameters. Third, OSC supports an incremental modeling approach to achieve low overhead of the models for online needs. These techniques allow OSC to effectively learn the characteristics of an application and optimize its performance by automatically adjusting the configurations at runtime. Our implementation of OSC atop MapReduce/Hadoop 2.6 improves performance by 60 percent on average and up to 120 percent compared with the state-of-the-art approach. Lastly, the performance benefit of an application running on OSC generally increases along with its input data size.

12. HPCA

    QEI: Query acceleration can be generic and efficient in the cloud

    Yifan Yuan, Yipeng Wang, Ren Wang, Rangeen Basu Roy Chowhury, Charlie Tai, and Nam Sung Kim

    2021 IEEE International Symposium on High-Performance Computer Architecture (HPCA) , 2021

    Architecture Accelerator Network

    Abs DOI

13. A Journey to a Commercial-Grade Processing-In-Memory (PIM) Chip Development HPCA 2021

    Nam Sung Kim

    The 27th IEEE International Symposium on High-Performance Computer Architecture (PCA-27), Seoul, South Korea, URL: https://hpca-conf. org/2021/keynotes/, dated Mar , 2021

    Abs DOI

14. Greendimm: Os-assisted dram power management for dram with a sub-array granularity power-down state

    Seunghak Lee, Ki-Dong Kang, Hwanjun Lee, Hyungwon Park, Younghoon Son, Nam Sung Kim, and Daehoon Kim

    2021

    Architecture Systems Memory

    Abs DOI

15. PATENT

    In-Memory Near-Data Approximate Acceleration

    Nam Sung Kim, Hadi Esmaeilzadeh, and Amir Yazdanbakhsh

    2021
16. Virtual-Cache: A cache-line borrowing technique for efficient GPU cache architectures

    Bingchao Li, Jizeng Wei, and Nam Sung Kim

    Microprocessors and Microsystems , 2021

    Abs DOI

    GPUs provide megabytes of registers and shared memories to maintain the contexts for thousands of threads and enable fast data sharing amongst threads of a thread block, respectively. Besides, GPUs employ L1 cache to provide the high bandwidth service for memory requests. However, the average L1 cache capacity per thread is very limited, resulting in cache thrashing which in turn impairs the performance. Meanwhile, many registers and shared memories are unassigned to any warps or thread blocks. Moreover, registers and shared memories that are assigned can be idle when warps or thread blocks are finished. Exploiting the above insights, we propose Virtual-Cache to cost-effectively increase the effective size of L1 cache by utilizing the unassigned and released registers and shared memories as cache-lines in this paper. Specifically, we leverage the unassigned registers and shared memories to serve cache requests directly. Regarding the registers assigned to a warp, they can work as cache-lines after the warp completes the execution and before they are accessed again by a new launched warp. Regarding the shared memories of a thread block, they are enabled to serve cache requests when the thread block is finished till they are referenced by shared memory instructions of the relaunched thread block. The register file, shared memory and L1 cache are physically independent but logically unified as a large virtual cache with redesigned cache-line management. We develop the control and data path for the register file, making the register file accessible for cache requests by borrowing an operand collector to serve the cache requests. We also expand the control and data path for the shared memory to serve the cache requests. Our evaluation results show that Virtual-Cache makes the performance improved by 28% over the previously proposed cache management technique for cache-sensitive applications.

17. PATENT

    Memory module memory device and processing device having a processor mode and memory system

    O Seong-il, Nam Sung Kim, SON Young-Hoon, Chan-kyung Kim, Ho-young Song, Jung Ho Ahn, and Sang-joon Hwang

    2021

    Abs

    A memory module includes a memory device, a command/address buffering device, and a processing data buffer. The memory device includes a memory cell array, a first set of input/output terminals, each terminal configured to receive first command/address bits, and a second set of input/output terminals, each terminal configured to receive both data bits and second command/address bits. The command address buffering device is configured to output the first command/address bits to the first set of input/output terminals. The processing data buffer is configured to output the data bits and second command/address bits to the second set of input/output terminals. The memory device is configured such that the first command/address bits, second command/address bits, and data bits are all used to access the memory

18. PATENT

    FlatFlash system for byte granularity accessibility of memory in a unified memory-storage hierarchy

    Ahmed AbulilaAbulila, Vikram Sharma Mailthody, Zaid Qures, Hijian Huang, Nam Sung Kim, Jinjun Xiong, and Wen-mei Hwu

    2021

2020

1. MICRO

   Planaria: Dynamic architecture fission for spatial multi-tenant acceleration of deep neural networks

   Soroush Ghodrati, Byung Hoon Ahn, Joon Kyung Kim, Sean Kinzer, Brahmendra Reddy Yatham, Navateja Alla, Hardik Sharma, Mohammad Alian, Eiman Ebrahimi, Nam Sung Kim, Cliff Young, and Hadi Esmaeilzadeh

   2020 53rd Annual IEEE/ACM International Symposium on Microarchitecture (MICRO) , 2020

   Abs DOI

2. A 16-GB 640-GB/s HBM2E DRAM with a data-bus window extension technique and a synergetic on-die ECC scheme

   Ki Chul Chun, Yong Ki Kim, Yesin Ryu, Jaewon Park, Chi Sung Oh, Young Yong Byun, So Young Kim, Dong Hak Shin, Jun Gyu Lee, Byung-Kyu Ho, Min-Sang Park, Seong-Jin Cho, Seunghan Woo, Byoung Mo Moon, Beomyong Kil, Sungoh Ahn, Jae Hoon Lee, Soo Young Kim, Seouk-Kyu Choi, Jae-Seung Jeong, Sung-Gi Ahn, Jihye Kim, Jun Jin Kong, Kyomin Sohn, Nam Sung Kim, and Jung-Bae Lee

   IEEE Journal of Solid-State Circuits , 2020

   World’s Highest Bandwidth DRAM

   Abs DOI

3. Mixed-signal charge-domain acceleration of deep neural networks through interleaved bit-partitioned arithmetic

   Soroush Ghodrati, Hardik Sharma, Sean Kinzer, Amir Yazdanbakhsh, Jongse Park, Nam Sung Kim, Doug Burger, and Hadi Esmaeilzadeh

   2020

   Abs DOI

4. Data direct I/O characterization for future I/O system exploration

   Mohammad Alian, Yifan Yuan, Jie Zhang, Ren Wang, Myoungsoo Jung, and Nam Sung Kim

   2020 IEEE International Symposium on Performance Analysis of Systems and Software (ISPASS) , 2020

   Abs DOI

5. Bit-Parallel Vector Composability for Neural Acceleration

   Hadi Esmaeilzadeh Soroush Ghodrati, Hardik Sharma, Cliff Young, and Nam Sung Kim

   IEEE/ACM Design Automation Conference , 2020

   Abs DOI

6. ISCA

   Babelfish: Fusing address translations for containers

   Dimitrios Skarlatos, Umur Darbaz, Bhargava Gopireddy, Nam Sung Kim, and Josep Torrellas

   2020 ACM/IEEE 47th Annual International Symposium on Computer Architecture (ISCA) , 2020

   IEEE Micro Top Picks

   Architecture CPU

   Abs DOI

7. An 8.5-Gb/s/Pin 12-Gb LPDDR5 SDRAM with a hybrid-bank architecture low power and speed-boosting techniques

   Chang-Kyo Lee, Hyung-Joon Chi, Jin-Seok Heo, Jung-Hwan Park, Jin-Hun Jang, Dongkeon Lee, Jae-Hoon Jung, Dong-Hun Lee, Dae-Hyun Kim, Kihan Kim, Sang-Yun Kim, Dukha Park, Youngil Lim, Geuntae Park, Seung-Jun Lee, Seungki Hong, Dae-Hyun Kwon, Isak Hwang, Byongwook Na, Kyung-Ryun Kim, Seouk-Kyu Choi, Hyein Choi, Won-Il Bae, Jeong-Don Ihm, Seung-Jun Bae, Nam Sung Kim, and Jung-Bae Lee

   IEEE Journal of Solid-State Circuits , 2020

   Abs DOI

8. MICRO

   Freac cache: Folded-logic reconfigurable computing in the last level cache

   Ashutosh Dhar, Xiaohao Wang, Hubertus Franke, Jinjun Xiong, Jian Huang, Wen-mei Hwu, Nam Sung Kim, and Deming Chen

   2020 53rd Annual IEEE/ACM International Symposium on Microarchitecture (MICRO) , 2020

   Architecture Memory

   Abs DOI

9. Leveraging dynamic partial reconfiguration with scalable ILP based task scheduling

   Ashutosh Dhar, Mang Yu, Wei Zuo, Xiaohao Wang, Nam Sung Kim, and Deming Chen

   2020 33rd International Conference on VLSI Design and 2020 19th International Conference on Embedded Systems (VLSID) , 2020

   Abs DOI

10. CAL

    IDIO: Orchestrating Inbound Network Data on Server Processors

    Mohammad Alian, Jongmin Shin, Ki-Dong Kang, Ren Wang, Alexandros Daglis, Daehoon Kim, and Nam Sung Kim

    IEEE Computer Architecture Letters , 2020

    Abs DOI

    Network bandwidth demand in datacenters is doubling every 12 to 15 months. In response to this demand, high-bandwidth network interface cards, each capable of transferring 100s of Gigabits of data per second, are making inroads into the servers of next-generation datacenters. Such unprecedented data delivery rates on server endpoints raise new challenges, as inbound network traffic placement decisions within the memory hierarchy have a direct impact on end-to-end performance. Modern server-class Intel processors leverage DDIO technology to steer all inbound network data into the last-level cache (LLC), regardless of the network traffic’s nature. This static data placement policy is suboptimal, both from a performance and an energy efficiency standpoint. In this work, we design IDIO, a framework that—unlike DDIO—dynamically decides where to place inbound network traffic within a server’s multi-level memory hierarchy. IDIO dynamically monitors system behavior and distinguishes between different traffic classes to determine and periodically re-evaluate the best placement location for each flow: LLC, mid-level (L2) cache or DRAM. Our results show that IDIO increases a server’s maximum sustainable load by up to ∼33.3% across various network functions.

11. BDS-GCN: Efficient full-graph training of graph convolutional nets with partition-parallelism and boundary sampling

    Cheng Wan, Youjie Li, Nam Sung Kim, and Yingyan Lin

    2020

    Architecture CPU Network

    Abs

    Graph Convolutional Networks (GCNs) have emerged as the state-of-the-art model for graph-based learning tasks. However, it is still challenging to train GCNs at scale, limiting their applications to real-world large graphs and hindering the exploration of deeper and more sophisticated GCN architectures. While it can be natural to leverage graph partition and distributed training for tackling this challenge, this direction has only been slightly touched on previously due to the unique challenge posed by the GCN structures, especially the excessive amount of boundary nodes in each partitioned subgraph, which can easily explode the required memory and communications for distributed training of GCNs. To this end, we propose BDS-GCN, a method that adopts unbiased boundary sampling strategy to enable efficient and scalable distributed GCN training while maintaining the full-graph accuracy. Empirical evaluations and ablation studies validate the effectiveness of the proposed BDS-GCN, e.g., boosting the throughput by up-to 500% and reducing the memory usage by up-to 58% for distributed GCN training, while achieving the same accuracy, as compared with the state-of-the-art methods. We believe our BDS-GCN would open up a new paradigm for enabling GCN training at scale. All code will be released publicly upon acceptance.

12. CAL

    FastDrain: Removing page victimization overheads in NVMe storage stack

    Jie Zhang, Miryeong Kwon, Sanghyun Han, Nam Sung Kim, Mahmut Kandemir, and Myoungsoo Jung

    IEEE Computer Architecture Letters , 2020

    Abs DOI

13. PATENT

    Graphic processor unit providing reduced storage costs for similar operands

    Nam Sung Kim and Zhenhong Liu

    2020

    Architecture Memory Security
14. Errata to “Exploring Fault-Tolerant Erasure Codes for Scalable All-Flash Array Clusters”

    Sungjoon Koh, Jie Zhang, Miryeong Kwon, Jungyeon Yoon, David Donofrio, Nam Sung Kim, and Myoungsoo Jung

    IEEE Transactions on Parallel & Distributed Systems , 2020

    Abs DOI

    [1] S. Koh, et al.,“Exploring fault-tolerant erasure codes for scalable all-flash array clusters,” IEEE Trans. Parallel Distrib. Syst., vol. 30, no. 6, pp. 1312–1330, Jun. 2019.

2019

1. ASPLOS

   Flatflash: Exploiting the byte-accessibility of ssds within a unified memory-storage hierarchy

   Ahmed Abulila, Vikram Sharma Mailthody, Zaid Qureshi, Jian Huang, Nam Sung Kim, Jinjun Xiong, and Wen-mei Hwu

   24th ACM International Conference on Architectural Support for Programming Languages and Operating Systems , 2019

   Circuit Memory

   Abs DOI

2. Netdimm: Low-latency near-memory network interface architecture

   Mohammad Alian and Nam Sung Kim

   2019

   Abs DOI

3. PATENT

   Memory module memory device and processing device having a processor mode and memory system

   O Seong-il, Nam Sung Kim, SON Young-Hoon, Chan-kyung Kim, Ho-young Song, Jung Ho Ahn, and Sang-joon Hwang

   2019

   Abs DOI

   A memory module includes a memory device, a command/address buffering device, and a processing data buffer. The memory device includes a memory cell array, a first set of input/output terminals, each terminal configured to receive first command/address bits, and a second set of input/output terminals, each terminal configured to receive both data bits and second command/address bits. The command/address buffering device is configured to output the first command/address bits to the first set of input/output terminals. The processing data buffer is configured to output the data bits and second command/address bits to the second set of input/output terminals. The memory device is configured such that the first command/address bits, second command/address bits, and data bits are all used to access the memory cell array.(51) Int. CI. G06F 12/00 G06F 3/06 G06F 13/16

4. LL-PCM: Low-latency phase change memory architecture

   Nam Sung Kim, Choungki Song, Woo Young Cho, Jian Huang, and Myoungsoo Jung

   2019

   Abs DOI

5. An efficient GPU cache architecture for applications with irregular memory access patterns

   Bingchao Li, Jizeng Wei, Jizhou Sun, Murali Annavaram, and Nam Sung Kim

   ACM Transactions on Architecture and Code Optimization (TACO) , 2019

   Abs DOI

6. AxMemo: Hardware-compiler co-design for approximate code memoization

   Zhenhong Liu, Amir Yazdanbakhsh, Dong Kai Wang, Hadi Esmaeilzadeh, and Nam Sung Kim

   2019

   Abs DOI

7. Near-memory and in-storage fpga acceleration for emerging cognitive computing workloads

   Ashutosh Dhar, Sitao Huang, Jinjun Xiong, Damir Jamsek, Bruno Mesnet, Jian Huang, Nam Sung Kim, Wen-mei Hwu, and Deming Chen

   2019 IEEE Computer Society Annual Symposium on VLSI (ISVLSI) , 2019

   Abs DOI

   The ever-increasing number of parameters in Large Language Models (LLMs) demands many expensive GPUs for both inference and training. This is because even such a high-end GPU such as NVIDIA A100 can store only a subset of parameters due to its limited memory capacity. To reduce the number of required GPUs, especially for inference, we may exploit the large memory capacity of (host) CPU to store not only all the model parameters but also intermediate outputs which also require a substantial memory capacity. However, this necessitates frequent data transfers between CPU and GPU over the slow PCIe interface, creating a bottleneck that hinders the accomplishment of both low latency and high throughput in inference. To address such a challenge, we first propose CPU-GPU cooperative computing that exploits the Advanced Matrix Extensions (AMX) capability of the latest Intel CPU, codenamed Sapphire Rapids (SPR). Second, we propose an adaptive model partitioning policy that determines the layers of a given LLM to be run on CPU and GPU, respectively, based on their memory capacity requirement and arithmetic intensity. As CPU executes the layers with large memory capacity but low arithmetic intensity, the amount of data transferred through the PCIe interface is significantly reduced, thereby improving the LLM inference performance. Our evaluation demonstrates that CPU-GPU cooperative computing, based on this policy, delivers 12.1× lower latency and 5.4× higher throughput than GPU-only computing for OPT-30B inference when both CPU-GPU and GPU-only computing store the model in CPU memory.

8. CAL

   Exploiting os-level memory offlining for dram power management

   Seunghak Lee, Nam Sung Kim, and Daehoon Kim

   IEEE Computer Architecture Letters , 2019

   Abs DOI

9. Practical near-data processing to evolve memory and storage devices into mainstream heterogeneous computing systems

   Nam Sung Kim and Pankaj Mehra

   2019

   Invited

   Abs DOI

10. An energy-efficient programmable mixed-signal accelerator for machine learning algorithms

    Mingu Kang, Prakalp Srivastava, Vikram Adve, Nam Sung Kim, and Naresh R Shanbhag

    IEEE micro , 2019

    Abs DOI

    We propose PROMISE, the first end-to-end design of a PROgrammable MIxed-Signal accElerator from Instruction Set Architecture to high-level language compiler for acceleration of diverse machine learning algorithms by exploiting the advantage of the superior energy efficiency from analog/mixed-signal processing.

11. CAL

    Network packet processing mode-aware power management for data center servers

    Ki-Dong Kang, Gyeongseo Park, Nam Sung Kim, and Daehoon Kim

    IEEE Computer Architecture Letters , 2019

    Abs DOI

    In data center servers, power management (PM) exploiting Dynamic Voltage and Frequency Scaling (DVFS) for processors can play a crucial role to improve energy efficiency. However, we observe that current PM policies (i.e., governors) not only considerably increase tail response time (i.e., violate a given Service Level Objective (SLO)) but also hurt energy efficiency. Tackling limitations of current PM governors, we propose NMAP, Network packet processing Mode-Aware Power management. NMAP improves energy efficiency while satisfying given SLOs, considering packet processing status on a core for PM by monitoring transitions between network packet processing modes - interrupt and polling. Tracking the transitions, NMAP detects moments that a core cannot process packets fast enough and forces the core to immediately raise the voltage and frequency (V/F) state. As a result, NMAP can provide not only low response time but also low energy consumption. Our experiment shows that NMAP improves tail response time by up to 4.1× compared with the ondemand governor, reducing energy by up to 44.6 percent compared with the performance governor.

12. SMART: STT-MRAM architecture for smart activation and sensing

    Byoungchan Oh, Nilmini Abeyratne, Nam Sung Kim, Ronald G Dreslinski, and Trevor Mudge

    2019

    Abs DOI

    STT-MRAM is a promising drop-in replacement for DRAM for main memory, because it can offer higher energy efficiency than DRAM with comparable latency. Implementing STT-MRAM similar to traditional DRAM memory, that is keeping the same policies while only replacing the technology, however, severely limits the goodness that this new technology can offer. STT-MRAM needs to employ current sense amps which require an order of magnitude more space and power than typical DRAM voltage sense amps. To manage the high cost of sense amps, STT-MRAM decouples bit-lines from sense amps and shares one sense amp across 16 to 128 bit-lines, exploiting the non-destructive nature of its read operation. This sense amp sharing reduces the size of row buffers and, as a result, incur more row-buffer misses (i.e., higher activation energy and lower performance). Other issues arise if STT-MRAM is required to be compatible with current DRAM interfaces and policies. To cost-effectively address these issues, we propose SMART, which, unlike DRAM and conventional STT-MRAM, proposes sensing bit-lines after receiving a column access command instead of a row activation command. This results in several benefits: larger pages, fewer sense amps, lower activation power, higher bank-level parallelism, shorter latency, fewer address pins, and more efficient repairing of defective columns than conventional STT-MRAM. Our evaluation shows that SMART consumes 11% (39%) lower energy while providing 9% (5%) higher performance than conventional STT-MRAM (DRAM) on average. In addition to these benefits, SMART is 6% smaller than conventional STT-MRAM.

13. A2M: Approximate Algebraic Memory Using Polynomials Rings

    Dong Kai Wang and Nam Sung Kim

    2019 IEEE/ACM International Symposium on Low Power Electronics and Design (ISLPED) , 2019

    Abs DOI

14. Ghost routers: energy-efficient asymmetric multicore processors with symmetric NoCs

    Hyojun Son, Hanjoon Kim, Hao Wang, Nam Sung Kim, and John Kim

    2019

    Abs DOI

2018

1. Pipe-SGD: A decentralized pipelined SGD framework for distributed deep net training

   Youjie Li, Mingchao Yu, Songze Li, Salman Avestimehr, Nam Sung Kim, and Alexander Schwing

   Advances in Neural Information Processing Systems , 2018

   Abs DOI

   Distributed training of deep nets is an important technique to address some of the present day computing challenges like memory consumption and computational demands. Classical distributed approaches, synchronous or asynchronous, are based on the parameter server architecture, ie, worker nodes compute gradients which are communicated to the parameter server while updated parameters are returned. Recently, distributed training with AllReduce operations gained popularity as well. While many of those operations seem appealing, little is reported about wall-clock training time improvements. In this paper, we carefully analyze the AllReduce based setup, propose timing models which include network latency, bandwidth, cluster size and compute time, and demonstrate that a pipelined training with a width of two combines the best of both synchronous and asynchronous training. Specifically, for a setup consisting of a four-node GPU cluster we show wall-clock time training improvements of up to 5.4 x compared to conventional approaches.

2. OSDI

   FlashShare: Punching Through Server Storage Stack from Kernel to Firmware for Ultra-Low Latency SSDs

   Jie Zhang, Miryeong Kwon, Donghyun Gouk, Sungjoon Koh, Changlim Lee, Mohammad Alian, Myoungjun Chun, Mahmut Taylan Kandemir, Nam Sung Kim, Jihong Kim, and Myoungsoo Jung

   13th USENIX Symposium on Operating Systems Design and Implementation (OSDI 18) , 2018

   Abs DOI

   A modern datacenter server aims to achieve high energy efficiency by co-running multiple applications. Some of such applications (eg, web search) are latency sensitive. Therefore, they require low-latency I/O services to fast respond to requests from clients. However, we observe that simply replacing the storage devices of servers with Ultra-Low-Latency (ULL) SSDs does not notably reduce the latency of I/O services, especially when co-running multiple applications. In this paper, we propose FlashShare to assist ULL SSDs to satisfy different levels of I/O service latency requirements for different co-running applications. Specifically, FlashShare is a holistic cross-stack approach, which can significantly reduce I/O interferences among co-running applications at a server without any change in applications. At the kernel-level, we extend the data structures of the storage stack to pass attributes of (co-running) applications through all the layers of the underlying storage stack spanning from the OS kernel to the SSD firmware. For given attributes, the block layer and NVMe driver of FlashShare differently manage the I/O scheduler and interrupt handler of NVMe. We also enhance the NVMe controller and cache layer at the SSD firmware-level, by dynamically partitioning DRAM in the ULL SSD and adjusting its caching strategies to meet diverse user requirements. The evaluation results demonstrate that FlashShare can shorten the average and 99th-percentile turnaround response times of co-running applications by 22% and 31%, respectively.

3. ISCA

   Ganax: A unified mimd-simd acceleration for generative adversarial networks

   Amir Yazdanbakhsh, Hajar Falahati, Philip J Wolfe, Hadi Esmaeilzadeh, and Kambiz Samadi

   2018 ACM/IEEE 45th annual international symposium on computer architecture (ISCA) , 2018

   Abs DOI

4. MICRO

   A network-centric hardware/algorithm co-design to accelerate distributed training of deep neural networks

   Youjie Li, Jongse Park, Mohammad Alian, Yifan Yuan, Zheng Qu, Peitian Pan, Ren Wang, Alexander Schwing, Hadi Esmaeilzadeh, and Nam Sung Kim

   2018 51st Annual IEEE/ACM International Symposium on Microarchitecture (MICRO) , 2018

   Hardware Prototype Demonstration

   Architecture Systems Memory ML/AI

   Abs DOI

5. MICRO

   Amber: Enabling precise full-system simulation with detailed modeling of all SSD resources

   Donghyun Gouk, Miryeong Kwon, Jie Zhang, Sungjoon Koh, Wonil Choi, Nam Sung Kim, Mahmut Kandemir, and Myoungsoo Jung

   2018 51st Annual IEEE/ACM International Symposium on Microarchitecture (MICRO) , 2018

   Systems ML/AI

   Abs DOI

6. Flexigan: An end-to-end solution for fpga acceleration of generative adversarial networks

   Amir Yazdanbakhsh, Michael Brzozowski, Behnam Khaleghi, Soroush Ghodrati, Kambiz Samadi, Nam Sung Kim, and Hadi Esmaeilzadeh

   2018 IEEE 26th Annual International Symposium on Field-Programmable Custom Computing Machines (FCCM) , 2018

   Systems ML/AI

   Abs DOI

7. Gradiveq: Vector quantization for bandwidth-efficient gradient aggregation in distributed cnn training

   Mingchao Yu, Zhifeng Lin, Krishna Narra, Songze Li, Youjie Li, Nam Sung Kim, Alexander Schwing, Murali Annavaram, and Salman Avestimehr

   Advances in Neural Information Processing Systems , 2018

   Abs DOI

   Data parallelism can boost the training speed of convolutional neural networks (CNN), but could suffer from significant communication costs caused by gradient aggregation. To alleviate this problem, several scalar quantization techniques have been developed to compress the gradients. But these techniques could perform poorly when used together with decentralized aggregation protocols like ring all-reduce (RAR), mainly due to their inability to directly aggregate compressed gradients. In this paper, we empirically demonstrate the strong linear correlations between CNN gradients, and propose a gradient vector quantization technique, named GradiVeQ, to exploit these correlations through principal component analysis (PCA) for substantial gradient dimension reduction. GradiveQ enables direct aggregation of compressed gradients, hence allows us to build a distributed learning system that parallelizes GradiveQ gradient compression and RAR communications. Extensive experiments on popular CNNs demonstrate that applying GradiveQ slashes the wall-clock gradient aggregation time of the original RAR by more than 5x without noticeable accuracy loss, and reduce the end-to-end training time by almost 50%. The results also show that\GradiveQ is compatible with scalar quantization techniques such as QSGD (Quantized SGD), and achieves a much higher speed-up gain under the same compression ratio.

8. ISCA

   PROMISE: An end-to-end design of a programmable mixed-signal accelerator for machine-learning algorithms

   Prakalp Srivastava, Mingu Kang, Sujan K Gonugondla, Sungmin Lim, Jungwook Choi, Vikram Adve, Nam Sung Kim, and Naresh Shanbhag

   2018 ACM/IEEE 45th Annual International Symposium on Computer Architecture (ISCA) , 2018

   Abs DOI

9. MICRO

   Application-transparent near-memory processing architecture with memory channel network

   Mohammad Alian, Seung Won Min, Hadi Asgharimoghaddam, Ashutosh Dhar, Dong Kai Wang, Thomas Roewer, Adam McPadden, Oliver O’Halloran, Deming Chen, Jinjun Xiong, Daehoon Kim, Wen-mei Hwu, and Nam Sung Kim

   2018 51st Annual IEEE/ACM International Symposium on Microarchitecture (MICRO) , 2018

   Best Paper Nomination
10. In-dram near-data approximate acceleration for gpus

    Amir Yazdanbakhsh, Choungki Song, Jacob Sacks, Pejman Lotfi-Kamran, Hadi Esmaeilzadeh, and Nam Sung Kim

    2018

    Abs DOI

11. Leveraging power-performance relationship of energy-efficient modern DRAM devices

    Sukhan Lee, Hyunyoon Cho, Young Hoon Son, Yuhwan Ro, Nam Sung Kim, and Jung Ho Ahn

    IEEE Access , 2018

    Best Paper Award

    Abs DOI

12. SiMul: An algorithm-driven approximate multiplier design for machine learning

    Zhenhong Liu, Amir Yazdanbakhsh, Taejoon Park, Hadi Esmaeilzadeh, and Nam Sung Kim

    IEEE Micro , 2018

    Abs DOI

13. PATENT

    Computer architecture having selectable parallel and serial communication channels between processors and memory

    Hao Wang and Nam Sung Kim

    2018

    Abs

    A computer architecture provides both a parallel memory bus and serial memory bus between a processor system and memory. Latency-tolerant memory access requests are steered to the serial memory bus which operates to increase the available memory bus bandwidth on the parallel memory. The invention also provides integrated circuit computer memory suitable for this application.

14. Cta-aware prefetching and scheduling for gpu

    Gunjae Koo, Hyeran Jeon, Zhenhong Liu, Nam Sung Kim, and Murali Annavaram

    2018 IEEE International Parallel and Distributed Processing Symposium (IPDPS) , 2018

    Abs DOI

15. PATENT

    Memory system and method for error correction of memory

    Jung Ho Ahn and KIM Namsung

    2018

    Abs DOI

    A memory system and a method for the error correction of memory are disclosed herein. The method for the error correction of memory is performed by a memory system including a plurality of memory chips. The method for the error correction of memory may include reading, by a first ECC engine unit included in each of a plurality of memory chips, a chunk including a plurality of data bursts, first parity bits, and position bits from each of the plurality of memory chips; extracting, by the first ECC engine unit, a single data burst having an error from the plurality of data bursts using the position bits; and performing, by the first ECC engine unit, first error correction using the first parity bit corre sponding to the extracted error data burst.

16. Load-triggered warp approximation on GPU

    Zhenhong Liu, Daniel Wong, and Nam Sung Kim

    2018

    Abs DOI

    Value similarity of operands across warps have been exploited to improve energy efficiency of GPUs. Prior work, however, incurs significant overheads to check value similarity for every instruction and does not improve performance as it does not reduce the number of executed instructions. This work proposes Lock ’n Load (LnL) which triggers approximate execution of code regions by only checking similarity of values returned from load instructions and fuses multiple approximated warps into a single warp.

17. CIAO: Cache interference-aware throughput-oriented architecture and scheduling for GPUs

    Jie Zhang, Shuwen Gao, Nam Sung Kim, and Myoungsoo Jung

    2018 IEEE International Parallel and Distributed Processing Symposium (IPDPS) , 2018

    Abs DOI

18. Exploring fault-tolerant erasure codes for scalable all-flash array clusters

    Sungjoon Koh, Jie Zhang, Miryeong Kwon, Jungyeon Yoon, David Donofrio, Nam Sung Kim, and Myoungsoo Jung

    IEEE Transactions on Parallel and Distributed Systems , 2018

    Abs DOI

19. PATENT

    Shared row buffer system for asymmetric memory

    Hao Wang and Nam Sung Kim

    2018

    Abs DOI

    An architecture for improved memory access in asymmetric memories provides a set of shared row buffers that may be freely allocated between slow and fast memory banks of the asymmetric memory. This permits allocation of row buffers dynamically between the slow and fast memory banks to improve execution speeds and also permits a lightweight memory swap procedure for moving data between the slow and fast memory banks with low processor and memory channel overheads.(58)

20. 3D-Xpath: High-density managed dram architecture with cost-effective alternative paths for memory transactions

    Sukhan Lee, Kiwon Lee, Minchul Sung, Mohammad Alian, Chankyung Kim, Wooyeong Cho, Reum Oh, Seongil O, Jung Ho Ahn, and Nam Sung Kim

    2018

    Abs DOI

21. IISWC

    Simulating PCI-Express interconnect for future system exploration

    Mohammad Alian, Krishna Parasuram Srinivasan, and Nam Sung Kim

    2018 IEEE International Symposium on Workload Characterization (IISWC) , 2018

    Abs DOI

22. Ganax: A unified mimd-simd acceleration for generative adversarial networks. In 2018 ACM/IEEE 45th Annual International Symposium on Computer Architecture (ISCA)

    Amir Yazdanbakhsh, Kambiz Samadi, Nam Sung Kim, and Hadi Esmaeilzadeh

    2018

    Abs DOI

23. VIP: Virtual performance-state for efficient power management of virtual machines

    Ki-Dong Kang, Mohammad Alian, Daehoon Kim, Jaehyuk Huh, and Nam Sung Kim

    2018

    Abs DOI

24. Design and implementation of SSD-assisted backup and recovery for database systems

    Yongseok Son, Moonsub Kim, Sunggon Kim, Heon Young Yeom, Nam Sung Kim, and Hyuck Han

    IEEE Transactions on Knowledge and Data Engineering , 2018

    Abs DOI

25. Practical challenges in supporting function in memory

    Nam Sung Kim

    2018 IEEE Asian Solid-State Circuits Conference (A-SSCC) , 2018

    Invited

    Abs DOI

    The performance of computer systems is often limited by the bandwidth of their memory channels, but further increasing the bandwidth is challenging under the stringent pin and power constraints of packages. To further increase performance under these constraints, various processing-in-memory (or function-in-memory) architectures, which tightly integrate processing functions with DRAM devices using 3D/2.5D-stacking technology, have been proposed. However, they have not been successfully commercialized by the industry yet because of various technical and practical challenges. In this article for the plenary talk, I will briefly discuss what challenges have been overlooked by researchers to shed light on successful commercialization and wide adoption of processing-in-memory architecture.

26. A load balancing technique for memory channels

    Byoungchan Oh, Nam Sung Kim, Jeongseob Ahn, Bingchao Li, Ronald G Dreslinski, and Trevor Mudge

    2018

    Abs DOI

    The performance needs of memory systems caused by growing volumes of data from emerging applications, such as machine learning and big data analytics, have continued to increase. As a result, HBM has been introduced in GPUs and throughput oriented processors. HBM is a stack of multiple DRAM devices across a number of memory channels. Although HBM provides a large number of channels and high peak bandwidth, we observed that all channels are not evenly utilized and often only one or few channels are highly congested after applying the hashing technique to randomize the translated physical memory address. To solve this issue, we propose a cost-effective technique to improve load balancing for HBM channels. In the proposed memory system, a memory request from a busy channel can be migrated to other non-busy channels and serviced in the other channels. Moreover, this request migration reduces stalls by memory controllers, because the depth of a memory request queue in a memory controller is effectively increased by the migration. The improved load balancing of memory channels shows a 10.1% increase in performance for GPGPU workloads.

27. Approximate ultra-low voltage many-core processor design

    Nam Sung Kim and Ulya R Karpuzcu

    2018

    Abs DOI

    Computing at ultra-low voltages can increase the energy efficiency significantly, however, operating frequency and resilience to errors degrade as the operating voltage reaches the transistor threshold voltage. More parallelism can help prevent degradation in throughput performance arising from the lower frequency. More parallelism, however, makes more components subject to errors, which exacerbates the already intensified vulnerability to errors. This chapter is all about how to exploit the intrinsic noise tolerance of emerging R(ecognition), M(ining), and S(ynthesis) applications in addressing degraded resilience at ultra-low voltages by embracing errors.

28. CAL

    Semi-coherent dma: An alternative i/o coherency management for embedded systems

    Seungwon Min, Mohammad Alian, Wen-Mei Hwu, and Nam Sung Kim

    IEEE Computer Architecture Letters , 2018

    Abs DOI

    Many modern embedded CPUs adopt Non-Coherent DMA (NC-DMA) over Coherent DMA (C-DMA) because of simplicity. An NC-DMA design, however, requires a CPU device driver to explicitly invalidate or flush a wide range of cache space. When an I/O DMA device writes data to a main memory region, the CPU needs to invalidate the cache space corresponding to the same memory region twice: (1) to prevent dirty cache lines from overwriting the DMA data and (2) to remove any cache lines prefetched before the DMA is done. In this work, we first show that such explicit invalidations consume 31 percent of CPU cycles, limiting the data transfer throughput of a high-speed network interface card (NIC) when receiving network packets. Second, we propose a Semi-Coherent DMA (SC-DMA) architecture for improving the efficiency of NC-DMA with a slight modification to the hardware. Specifically, our SC-DMA records the DMA region and prohibits any data that is prefetched from the region from entering the cache, reducing nearly 50 percent of the unnecessary invalidations. Lastly, we identify several software optimizations that can substantially reduce excessive cache invalidations prevalent in NIC drivers. Our evaluation with NVIDIA Jetson TX2 shows that our proposed SC-DMA design with the NIC driver optimizations can improve the NIC data transfer throughput by up to 53.3 percent.

2017

1. dist-gem5: Distributed simulation of computer clusters

   Alian Mohammad, Umur Darbaz, Gabor Dozsa, Stephan Diestelhorst, Daehoon Kim, and Nam Sung Kim

   2017 IEEE International Symposium on Performance Analysis of Systems and Software (ISPASS) , 2017

   Abs DOI

2. HPCA

   Defect analysis and cost-effective resilience architecture for future DRAM devices

   Sanguhn Cha, O Seongil, Hyunsung Shin, Sangjoon Hwang, Kwangil Park, Seong Jin Jang, Joo Sun Choi, Gyo Young Jin, Young Hoon Son, Hyunyoon Cho, Jung Ho Ahn, and Nam Sung Kim

   2017 IEEE International Symposium on High Performance Computer Architecture (HPCA) , 2017

   Abs DOI

3. CAL

   SimpleSSD: Modeling solid state drives for holistic system simulation

   Myoungsoo Jung, Jie Zhang, Ahmed Abulila, Miryeong Kwon, Narges Shahidi, John Shalf, Nam Sung Kim, and Mahmut Kandemir

   IEEE Computer Architecture Letters , 2017

   Abs DOI

4. Smart gait-aid glasses for Parkinson’s disease patients

   DaeHan Ahn, Hyerim Chung, Ho-Won Lee, Kyunghun Kang, Pan-Woo Ko, Nam Sung Kim, and Taejoon Park

   IEEE Transactions on Biomedical Engineering , 2017

   Abs DOI

5. Elastic-cache: GPU cache architecture for efficient fine-and coarse-grained cache-line management

   Bingchao Li, Jizhou Sun, Murali Annavaram, and Nam Sung Kim

   2017 IEEE International Parallel and Distributed Processing Symposium (IPDPS) , 2017

   Abs DOI

   GPUs provide high-bandwidth/low-latency on-chip shared memory and L1 cache to efficiently service a large number of concurrent memory requests (to contiguous memory space). To support warp-wide accesses to L1 cache, GPU L1 cache lines are very wide. However, such L1 cache architecture cannot always be efficiently utilized when applications generate many memory requests with irregular access patterns especially due to branch and memory divergences. In this paper, we propose Elastic-Cache that can efficiently support both fine- and coarse-grained L1 cache-line management for applications with both regular and irregular memory access patterns. Specifically, it can store 32- or 64-byte words in non-contiguous memory space to a single 128-byte cache line. Furthermore, it neither requires an extra tag storage structure nor reduces the capacity of L1 cache since it stores auxiliary tags for fine-grained L1 cache-line managements in sharedmemory space that is not fully used in many applications. Our experiment shows that Elastic-Cache improves the geo-mean performance of applications with irregular memory access patterns by 58% without degrading performance of applications with regular memory access patterns.

6. Pageforge: a near-memory content-aware page-merging architecture

   Dimitrios Skarlatos, Nam Sung Kim, and Josep Torrellas

   2017

   Abs DOI

7. Heterogeneous computing meets near-memory acceleration and high-level synthesis in the post-moore era

   Nam Sung Kim, Deming Chen, Jinjun Xiong, and Wen-mei W Hwu

   IEEE Micro , 2017

   Abs DOI

8. Collaborative (cpu+ gpu) algorithms for triangle counting and truss decomposition on the minsky architecture: Static graph challenge: Subgraph isomorphism

   Ketan Date, Keven Feng, Rakesh Nagi, Jinjun Xiong, Nam Sung Kim, and Wen-Mei Hwu

   2017 IEEE High Performance Extreme Computing Conference (HPEC) , 2017

   IEEE/Amazon/DARPA Graph Challenge Champions, Honorable Mention

   Abs DOI

   In this paper, we present collaborative CPU + GPU algorithms for triangle counting and truss decomposition, the two fundamental problems in graph analytics. We describe the implementation details and present experimental evaluation on the IBM Minsky platform. The main contribution of this paper is a thorough benchmarking and comparison of the different memory management schemes offered by CUDA 8 and NVLink, which can be harnessed for tackling large problems where the limited GPU memory capacity is the primary bottleneck in traditional computing platform. We find that the collaborative algorithms achieve 28× speedup on average (180× max) for triangle counting, and 165× speedup on average (498× max) for truss decomposition, when compared with the baseline Python implementation provided by the Graph Challenge organizers.

9. HPCA

   Ncap: Network-driven packet context-aware power management for client-server architecture

   Mohammad Alian, Ahmed HMO Abulila, Lokesh Jindal, Daehoon Kim, and Nam Sung Kim

   2017 IEEE International Symposium on High Performance Computer Architecture (HPCA) , 2017

   Best Paper Nomination

   Abs DOI

10. HPCA

    G-scalar: Cost-effective generalized scalar execution architecture for power-efficient gpus

    Zhenhong Liu, Syed Gilani, Murali Annavaram, and Nam Sung Kim

    2017 IEEE International Symposium on High Performance Computer Architecture (HPCA) , 2017

    Abs DOI

11. Rebooting the data access hierarchy of computing systems

    W Hwu Wen-mei, Izzat El Hajj, Simon Garcia De Gonzalo, Carl Pearson, Nam Sung Kim, Deming Chen, Jinjun Xiong, and Zehra Sura

    2017 IEEE International Conference on Rebooting Computing (ICRC) , 2017

    Abs DOI

12. PATENT

    Voltage regulator control for improved computing power efficiency

    Nam Sung Kim

    2017

    Abs DOI

    (57) ABSTRACT A controller for Voltage regulators providing power to computer processors may control the number of active phases of each voltage regulator according to a determined electrical current demand from the processor. By relying on electrical current demand rather than a P-state, the latter generally indicating a power conservation status, improved regulator efficiencies may be had, in particular responding to situations where low current demand occurs under heavy processor demand because of C-state variations. 2005, 0046400 A1

13. PATENT

    Resource and core scaling for improving performance of power-constrained multi-core processors

    Nam Sung Kim

    2017

    Abs DOI

    (57) ABSTRACT A multi-core processor provides circuitry for jointly scaling the number of operating cores and the amount of resources per core in order to maximize processing performance in a power-constrained environment. Such scaling is advanta geously provided without the need for Scaling Voltage and frequency. Selection of the number of operating cores and the amount of resources per core is made by examining the degree of instruction and thread level parallelism available for a given application. Accordingly, performance counters (and other characteristics) implemented in by a processor may be sampled on-line (in real time) and/or performance counters for a given application may be profiled and char acterized off-line. As a result, improved processing perfor mance may be achieved despite decreases in core operating Voltages and increases in technology process variability over time.

14. IISWC

    Understanding System Characteristics of Online Erasure Coding on Scalable Distributed and Large-Scale SSD Array Systems

    Sungjoon Koh, Jie Zhang, Miryeong Kwon, Jungyeon Yoonz, David Donofrioy, Nam Sung Kim, and Myoungsoo Jung

    2017 IEEE International Symposium on Workload Characterization (IISWC) , 2017

    Abs DOI

15. PATENT

    Multiplication circuit providing dynamic truncation

    Srinivasan Narayanamoorthy and Nam Sung Kim

    2017

    Abs DOI

16. IISWC

    Understanding power-performance relationship of energy-efficient modern DRAM devices

    Sukhan Lee, Yuhwan Ro, Young Hoon Son, Hyunyoon Cho, Nam Sung Kim, and Jung Ho Ahn

    2017 IEEE International Symposium on Workload Characterization (IISWC) , 2017

    Abs DOI

17. Temporal codes in on-chip interconnects

    Michael Mishkin, Nam Sung Kim, and Mikko Lipasti

    2017 IEEE/ACM International Symposium on Low Power Electronics and Design (ISLPED) , 2017

    Abs DOI

    Dynamic power consumption associated with signal toggles over long distance wires accounts for a significant portion of on-chip interconnect power. Improving dynamic energy efficiency in highly capacitive interconnects can be achieved by reducing the toggle rates associated with data communication. Temporal coding schemes facilitate bounded activity factors by encoding information as placement of signal toggles in time and can thereby improve the energy efficiency of data communication by encoding multiple bits per toggle. We introduce two temporal protocol variants designed for traversal of the crossbars in on-chip networks. These protocols reduce peak power without loss of bandwidth and achieve energy efficient on-chip communication in high capacitance long distance interconnects. Extending these energy savings to a multi-hop mesh topology is achieved by router implementations equipped with bypassing mechanisms that elide per hop reencoding overheads. We demonstrate a four bit per transition temporal protocol with up to 75% communication energy reduction that can be achieved over a baseline serial bit stream protocol.

18. PATENT

    Memory controller for heterogeneous computer

    Hao Wang and Nam Sung Kim

    2017

    Abs

    The present inventors have recognized that current priority allocation systems for memory requests in heterogeneous processor Systems are Suboptimal for an important class of problems where the system executes a single parallel appli cation using both the CPU and the GPU, in this case, the system performance can be evaluated in view of the mea Sured completion of the entire application and fairness may not be a primary consideration. Accordingly, the present invention provides a memory access control system that measures the execution time of the single parallel applica tion running on the different processors and operates to minimize the execution time. The priority may be dynami cally adjusted during runtime. In one embodiment, the invention prevents high-priority memory access requests from being blocked behind more frequent but lower priority memory access requests, defeat

19. CNFET-based high throughput SIMD architecture

    Li Jiang, Tianjian Li, Naifeng Jing, Nam Sung Kim, Minyi Guo, and Xiaoyao Liang

    IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems , 2017

    Abs DOI

    Carbon nanotube field effect transistor (CNFET), using the carbon nanotubes (CNTs) as the material for conducting, is a promising alternative of CMOS technology to overcome the “power wall” issue. Recently, a microprocessor solely based on CNFETs was fabricated and demonstrated, which is a big step forward to the industrial practice. However, CNFETs are inherently subject to much larger process variation or manufacturing defects; thereby it may cause significant design cost to build high performance processors. This is exacerbated in the large register file (RF) architectures widely used in single instruction multiple data (SIMD) architectures, e.g., general public utilities style processors, where the number of critical paths are multiplied by the SIMD width and thread count. In this paper, we seek cost-effective approaches to address the issues by judiciously exploiting the strong asymmetric spatial correlation in the variation unique to the CNFET fabrication process. This paper presents a microarchitectural model to characterize CNFET delay variation and malfunction, under which we show that the RF organizations coupled with the architectural schemes are critical to the performance and power consumption of the SIMD processor. Therefore, we propose several architectural techniques to mitigate the performance degradation and the impact of CNT metallization, leveraging the distinctive CNFET characteristics and the unique features in the SIMD processors. Experimental results verify the effectiveness of the proposed techniques and demonstrate the great opportunity offered by this new device technology.

20. Energy‐Efficient Approximate Speech Signal Processing for Wearable Devices

    Taejoon Park, Kyoosik Shin, and Nam Sung Kim

    ETRI Journal , 2017

    Abs DOI

    As wearable devices are powered by batteries, they need to consume as little energy as possible. To address this challenge, in this article, we propose a synergistic technique for energy‐efficient approximate speech signal processing (ASSP) for wearable devices. More specifically, to enable the efficient trade‐off between energy consumption and sound quality, we synergistically integrate an approximate multiplier and a successive approximate register analog‐to‐digital converter using our enhanced conversion algorithm. The proposed ASSP technique provides  40% lower energy consumption with  5% higher sound quality than a traditional one that optimizes only the bit width of SSP.

21. Pageforge

    Dimitrios Skarlatos, Nam Sung Kim, and Josep Torrellas

    Proceedings of the 50th Annual IEEE/ACM International Symposium on Microarchitecture , 2017

    Abs DOI

22. Janus: Supporting heterogeneous power management in virtualized environments

    Daehoon Kim, Mohammad Alian, Jaehyuk Huh, and Nam Sung Kim

    2017

    Abs DOI

    The cloud servers have routinely adopted machine virtualization for high energy efficiency. Such virtualization notably improves energy efficiency not only through consolidation, but also through Dynamic Voltage/Frequency Scaling (DVFS). Thus, current hypervisors such as Xen and KVM support power management (PM) policies statically or dynamically setting a Voltage/Frequency (V/F) level, similar to ones deployed by the Linux. However, the current hypervisors can promote only a single PM policy (i.e., host governor) per physical core. This poses a unique challenge for VMs sharing a physical core and running applications with opposite runtime characteristics in a time-shared manner (i.e., heterogeneous VMs); note that the consolidation policy often encourages heterogeneous VMs to share a physical core, since such VMs use different resources in the system [2].

23. PATENT

    Method and apparatus for power reduction during lane divergence

    Nam Sung Kim, James M O’connor, Michael J Schulte, and Vijay Janapa Reddi

    2017

    Abs DOI

    (57) ABSTRACT A method and device for reducing power during an instruc tion lane divergence includes idling an inactive execution lane during the lane divergence.

24. PATENT

    Memory fault patching using pre-existing memory structures

    David John Palframan, Nam Sung Kim, and Mikko Lipasti

    2017

    Abs DOI

    (57) ABSTRACT A computer architecture addresses intermittent memory faults by exploiting redundancy inherent in a hierarchical memory structure, for example, as data moves through various cache levels and registers before use by the proces sor. Accesses to data from faulted memory areas is diverted to a secondary memory structure holding that data and the secondary memory structure is flagged to increase the per sistence of the stored data used for patching against normal updating policies.

25. Energy-efficient approximate audio signal processing engine for wearable devices

    Electronics and Telecommunications Research Institute (ETRI) Journal , 2017

    Abs DOI

26. PATENT

    Sequential circuit with error detection

    Keith A Bowman, James W Tschanz, Nam Sung Kim, Janice C Lee, Christopher B Wilkerson, Shih-Lien L Lu, Tanay Karnik, and Vivek K De

    2017

    Abs DOI

    Sequential circuits with error-detection are provided. They may, for example, be used to replace traditional master-slave flip-flops, eg, in critical path circuits to detect and initiate correction of late transitions at the input of the sequential. In Some embodiments, such sequentials may comprise a tran sition detector with a time borrowing latch.

27. PATENT

    Apparatus and method for adjusting bandwidth

    Ho-Young Kim, Nam-Sung Kim, and Daniel W Chang

    2017

    Abs DOI

    Foreign Application Priority Data A method for adjusting bandwidth, a bandwidth scaler and an apparatus are provided. The method for adjusting band Nov. 7, 2013 (KR)........................ 10-2013-O134908 width involves determining a dynamic context of a proces

2016

1. MICRO

   Chameleon: Versatile and practical near-DRAM acceleration architecture for large memory systems

   Hadi Asghari-Moghaddam, Young Hoon Son, Jung Ho Ahn, and Nam Sung Kim

   2016 49th annual IEEE/ACM international symposium on Microarchitecture (MICRO) , 2016

   Abs DOI

2. HPCA

   Approximating warps with intra-warp operand value similarity

   Daniel Wong, Nam Sung Kim, and Murali Annavaram

   2016 IEEE International Symposium on High Performance Computer Architecture (HPCA) , 2016

   Abs DOI

3. Near-DRAM acceleration with single-ISA heterogeneous processing in standard memory modules

   Hadi Asghari-Moghaddam, Amin Farmahini-Farahani, Katherine Morrow, Jung Ho Ahn, and Nam Sung Kim

   IEEE Micro , 2016

   Abs DOI

   Energy consumed for transferring data across the processor memory hierarchy constitutes a large fraction of total system energy consumption, and this fraction has steadily increased with technology scaling. This article presents a near-DRAM acceleration (NDA) architecture wherein lightweight processors (LWPs) with the same ISA as their host processor are 3D-stacked atop commodity DRAM devices in a standard memory module to efficiently process data. In contrast to previous architectures, the authors’ NDA architecture requires negligible changes to commodity DRAM device and standard memory module architectures. This allows the NDA to be more easily adopted in both existing and emerging systems. Experiments demonstrate that, on average, the authors’ NDA-based system consumes almost 65 percent less energy at nearly two times higher performance than the baseline system.

4. On effective and efficient quality management for approximate computing

   Ting Wang, Qian Zhang, Nam Sung Kim, and Qiang Xu

   2016

   Abs DOI

5. HPCA

   ScalCore: Designing a core for voltage scalability

   Bhargava Gopireddy, Choungki Song, Josep Torrellas, Nam Sung Kim, Aditya Agrawal, and Asit Mishra

   2016 IEEE International Symposium on High Performance Computer Architecture (HPCA) , 2016

   Abs DOI

6. VARIUS-TC: A modular architecture-level model of parametric variation for thin-channel switches

   S Karen Khatamifard, Michael Resch, Nam Sung Kim, and Ulya R Karpuzcu

   2016 IEEE 34th International Conference on Computer Design (ICCD) , 2016

   Abs DOI

   Under aggressive miniaturization, unconventional digital switches rapidly come to light, which introduce new sources of variation in design parameters, and hence challenge the manufacturing process further. As a result, performance and power of manufactured hardware becomes greatly unpredictable. Characterizing variation-incurred unpredictability at early stages of the design necessitates dependable architecture-level models of variation, which distill device- and circuit-level details to accurately evaluate system-level implications. In this paper, we introduce a modular architecture-level model of parametric variation to address this challenge. As a case study, we refine our discussion to a representative class of emerging thin-channel switches, FinFETs.

7. HPCA

   DUANG: Fast and lightweight page migration in asymmetric memory systems

   Hao Wang, Jie Zhang, Sharmila Shridhar, Gieseo Park, Myoungsoo Jung, and Nam Sung Kim

   2016 IEEE International Symposium on High Performance Computer Architecture (HPCA) , 2016

   Abs DOI

8. Exploring new features of high-bandwidth memory for gpus

   Bingchao Li, Choungki Song, Jizeng Wei, Jung Ho Ahn, and Nam Sung Kim

   IEICE Electronics Express , 2016

   Abs DOI

   Due to the off-chip I/O pin and power constraints of GDDR5, HBM has been proposed to provide higher bandwidth and lower power consumption for GPUs. In this paper, we first provide detailed comparison between HBM and GDDR5 and expose two unique features of HBM: dualcommand and pseudo channel mode. Second, we analyze the effectiveness of these two features and show that neither notably contributes to performance. However, by combining pseudo channel mode with cache architecture supporting fine-grained cache-line management such as Amoeba cache, we achieve high effciency for applications with irregular memory requests. Our experiment demonstrates that compared with Amoeba caches with legacy mode, Amoeba cache with pseudo channel mode improves GPU performance by 25% and reduces HBM energy consumption by 15%.

9. PATENT

   Multiplier circuit with dynamic energy consumption adjustment

   Nam Sung Kim

   2016

   Abs DOI

10. PATENT

    Dynamic error handling for on-chip memory structures

    Nam Sung Kim

    2016

    Abs DOI

    A memory structure is provided that controls the activation of error handling bits as a function of operating voltage. In this way, error correction can be used to offset errors when the memory structure is run at low voltage (and frequency). However, negative performance impacts for such error correction, such as additional access latencies, can be avoided when the memory structure is run at higher voltage (and frequency) and memory errors are less likely. In addition, increased latencies due to evaluating error handling bits may be hidden by reading digital data bits from the memory structures speculatively and assuming no errors. Also, certain portions of memory structures may have larger cells, and therefore larger areas, than other portions, which may provide not only higher reliability at low operating voltages, but also faster operation with reduced latency.

11. PATENT

    Memory controller for heterogeneous computer

    Hao Wang and Nam Sung Kim

    2016

    Abs DOI

12. PATENT

    Method and apparatus for soft error mitigation in computers

    David John Palframan, Nam Sung Kim, and Mikko Lipasti

    2016

    Abs DOI

    The present inventors have recognized that previous efforts to identify the sources of soft errors have detrimentally adopted a “binary” approach to the problem, that is, examin

13. PATENT

    High efficiency computer floating point multiplier unit

    Nam Sung Kim, Syed Gilani, and Michael Schulte

    2016

    Abs DOI

14. MICRO

    Snatch: Opportunistically reassigning power allocation between processor and memory in 3D stacks

    Dimitrios Skarlatos, Renji Thomas, Aditya Agrawal, Shibin Qin, Robert Pilawa-Podgurski, Ulya R Karpuzcu, Radu Teodorescu, Nam Sung Kim, and Josep Torrellas

    2016 49th Annual IEEE/ACM International Symposium on Microarchitecture (MICRO) , 2016

    Abs DOI

15. Spinwise: A practical energy-efficient synchronization technique for cmps

    Hadi Asgharimoghaddam and Nam Sung Kim

    ACM SIGARCH Computer Architecture News , 2016

    Abs DOI

16. VR-scale: Runtime dynamic phase scaling of processor voltage regulators for improving power efficiency

    Hadi Asghari-Moghaddam, Hamid Reza Ghasemi, Abhishek Arvind Sinkar, Indrani Paul, and Nam Sung Kim

    2016

    Abs DOI

17. Fine-Grained Task Migration for Graph Algorithms using Processing in Memory

    Paula Aguilera, Dong Ping Zhang, Nam Sung Kim, and Nuwan Jayasena

    2016 IEEE International Parallel and Distributed Processing Symposium Workshops (IPDPSW) , 2016

    Abs DOI

18. CNFET-based high throughput register file architecture

    Tianjian Li, Li Jiang, Naifeng Jing, Nam Sung Kim, and Xiaoyao Liang

    2016 IEEE 34th International Conference on Computer Design (ICCD) , 2016

    Abs DOI

    A Carbon Nanotube field-effect transistor (CNFET) is a promising alternative to a traditional metal-oxide-semiconductor field-effect transistor (MOSFET) to overcome the “Power Wall” challenge. However, CNFETs are inherently subject to much larger process variation and thereby they can incur a significant design cost to build high-performance processors. Particularly, the large register files (RF) of SIMD GPU-style processors suffer more from such process variations because the number of critical paths are multiplied by the SIMD width and thread count. In this paper, we first show that RF organizations coupled with architectural techniques are critical to RF performance under CNFET-specific variations. Second, we propose several architectural techniques to mitigate the performance degradation, leveraging distinctive characteristics of CNFETs and unique features of SIMD processors. Our experiments demonstrate that the average RF performance is 53% higher than the worst design under variation and only 7% lower than the design with no variation.

19. Write-after-read hazard prevention in GPGPUSIM

    Michael Mishkin, Nam Sung Kim, and Mikko Lipasti

    Workshop on Deplicating, Deconstructing, and Debunking (WDDD) , 2016

    Abs

    The GPGPUsim simulator has a detailed model of streaming multiprocessor microarchitecture including scheduling logic that facilitates both instruction level parallelism and thread level parallelism. However, the pipeline performance model lacks proper safeguards against write-after-read hazards. The absence of mechanisms to enforce strict ordering of register file accesses permits more instruction level parallelism than would be otherwise achievable. This biases simulation results for some benchmarks and is potentially misleading to users of the simulator. In this paper, a few implementations of hazard handling mechanisms are considered, each with its own implementation complexity and granularity of bookkeeping. Evaluation of each mechanism’s impact on performance is included as well as demonstration of the variation on conclusions that may be drawn from experiments depending on the choice of write-after-read hazard handling technique.

20. Bit Serializing a Microprocessor for Ultra-low-power

    Matthew Tomei, Henry Duwe, Nam Sung Kim, and Rakesh Kumar

    2016

    Abs DOI

    Many emerging sensor applications are powered by energy harvesters that impose strict power constraints. These applications often do not require high performance or energy efficiency. We explore a technique for minimizing power of a microprocessor for power constrained applications: bit serial computing. Bit serial computing promises power benefits up to the data width for fully bit serializable logic. We perform a best-effort bit serialization of the openMSP430 microprocessor without making instruction set architecture (ISA) modifications. Although it is very challenging to serialize much of the logic in the microprocessor, we show that power benefits of serialization exceed 42% when the serial and parallel designs synthesized for their maximum operating frequency are running at a low duty cycle. Benefits are expected to be higher when ISA modifications are allowed.

21. Guest Editors’ Introduction: Approximate Computing

    Qiang Xu, Todd Mytkowicz, and Nam Sung Kim

    IEEE Design & Test , 2016

    Abs DOI

22. pd-gem5: Simulation Infrastructure for Parallel/Distributed Computer Systems

    Mohammad Alian, Daehoon Kim, Nam Sung Kim, Y Kim, W Yang, O Mutlu, LE Olson, S Sethumadhavan, MD Hill, B Jacob, M Kleanthous, Y Sazeides, E Özer, C Nicopoulos, P Nikolaou, Z Hadjilambrou, BK Daya, LS Peh, and AP Chandrakasan

    no. IEEE , 2016

    Abs DOI

23. PATENT

    Energy-efficient multicore processor architecture for parallel processing

    Nam-Sung Kim

    2016

    Abs DOI

24. PATENT

    Signal processing circuit with multiple power modes

    Nam Sung Kim

    2016

    Abs DOI

    The present inventor has recognized that the task of optimizing an image for human perception may not align with the requirements of feature detection circuitry (for example, used for face recognition). Accordingly the present invention provides a signal processing system that may operate in at least two modes, one mode optimizing the signal for human vision and the other mode optimizing the signal for feature detection where this latter mode provides reduced power consumption. Generally the feature detection mode may operate with a perception-degraded image that

25. Heterogeneous Computing–A Path to Post-Moore Supercomputing: Architecture Circuits and Process

    Wayne Burleson, Shomit Das, Yasuko Eckert, and Nam Sung Kim

    International Workshop on Post Moore’s Era Supercomputing , 2016

    Abs DOI

    As the trends driven by Moore’s Law come to an end, increased heterogeneity at all levels of computing is required to continue supercomputing progress. Supercomputing, perhaps more than other domains of computing such as client devices or Internet of Things, has relied on ever increasing resources and efficient management. This paper describes the opportunities and challenges of heterogeneous technologies at the architecture, circuit, and manufacturing process levels that provide a foundation for Post-Moore Supercomputing.

2015

1. Approximate computing: A survey

   Qiang Xu, Todd Mytkowicz, and Nam Sung Kim

   IEEE Design & Test , 2015

   Abs DOI

   As one of the most promising energy-efficient computing paradigms, approximate computing has gained a lot of research attention in the past few years. This paper presents a survey of state-of-the-art work in all aspects of approximate computing and highlights future research challenges in this field.

2. HPCA

   NDA: Near-DRAM acceleration architecture leveraging commodity DRAM devices and standard memory modules

   Amin Farmahini-Farahani, Jung Ho Ahn, Katherine Morrow, and Nam Sung Kim

   2015 IEEE 21st International Symposium on High Performance Computer Architecture (HPCA) , 2015

   Abs DOI

   Energy consumed for transferring data across the processor memory hierarchy constitutes a large fraction of total system energy consumption, and this fraction has steadily increased with technology scaling. In this paper, we propose near-DRAM acceleration (NDA) architectures, which process data using accelerators 3D-stacked on DRAM devices comprising off-chip main memory modules. NDA transfers most data through high-bandwidth and low-energy 3D interconnects between accelerators and DRAM devices instead of low-bandwidth and high-energy off-chip interconnects between a processor and DRAM devices, substantially reducing energy consumption and improving performance. Unlike previous near-memory processing architectures, NDA is built upon commodity DRAM devices; apart from inserting through-silicon vias (TSVs) to 3D-interconnect DRAM devices and accelerators, NDA requires minimal changes to the commodity DRAM device and standard memory module architectures. This allows NDA to be more easily adopted in both existing and emerging systems. Our experiments demonstrate that, on average, our NDA-based system consumes 46% (68%) lower (data transfer) energy at 1.67× higher performance than a system that integrates the same accelerator logic within the processor itself.

3. GPU register file virtualization

   Hyeran Jeon, Gokul Subramanian Ravi, Nam Sung Kim, and Murali Annavaram

   2015

   Abs DOI

4. HPCA

   CiDRA: A cache-inspired DRAM resilience architecture

   Young Hoon Son, Sukhan Lee, O Seongil, Sanghyuk Kwon, Nam Sung Kim, and Jung Ho Ahn

   2015 IEEE 21st International Symposium on High Performance Computer Architecture (HPCA) , 2015

   Abs DOI

5. COP: To compress and protect main memory

   David J Palframan, Nam Sung Kim, and Mikko H Lipasti

   ACM SIGARCH Computer Architecture News , 2015

   Abs DOI

6. HPCA

   iPatch: Intelligent fault patching to improve energy efficiency

   David J Palframan, Nam Sung Kim, and Mikko H Lipasti

   2015 IEEE 21st International Symposium on High Performance Computer Architecture (HPCA) , 2015

   Abs DOI

7. Ultra‐low‐power image signal processor for smart camera applications

   Zhenhong Liu, T Park, HS Park, and NS Kim

   Electronics Letters , 2015

   Abs DOI

   Among thriving cyber physical systems, smart camera applications require to run both image sensors and image signal processors (ISPs) to capture images whenever necessary. Owing to the nature of such applications (i.e. constantly capturing images and analysing the images to detect any event of interest), the image sensor and ISP become the two most energy consuming components in smart camera applications. It is started with the intuition that the perceptive quality of images is not strongly correlated with the accuracy of object detection algorithms and a technique that requires only minor modifications to the baseline ISP is proposed but notably reduces the ISP energy consumption in object detection mode for smart camera applications. It is demonstrated that the ISP reduces energy consumption by 33% compared with the baseline ISP energy while degrading face detection accuracy by only 3%.

8. Decoupled control and data processing for approximate near-threshold voltage computing

   Ismail Akturk, Nam Sung Kim, and Ulya R Karpuzcu

   IEEE Micro , 2015

   Abs DOI

   Near-threshold voltage computing can significantly improve energy efficiency; however, both operating speed and resilience to parametric variation reduce as the operating voltage reaches the threshold voltage. To prevent degradation in throughput performance, more cores should contribute to computation. The corresponding expansion in the chip area, however, is likely to further exacerbate the already intensified vulnerability to variation. Variation itself results in slower cores and ample speed differences among the cores. Worse, variation-induced slowdown might prevent operation at the designated clock speed, leading to timing errors. In this article, the authors exploit the intrinsic error tolerance of emerging Recognition, Mining, and Synthesis applications to mitigate variation. RMS applications can tolerate errors emanating from data-intensive program phases as opposed to control. Accordingly, the authors reserve reliable cores for control, and they execute error-tolerant data-intensive phases on error-prone cores. They also provide a design space exploration for decoupled control and data processing to mitigate variation at near-threshold voltages.

9. ATC

   Bolt: Faster reconfiguration in operating systems

   Sankaralingam Panneerselvam, Michael Swift, and Nam Sung Kim

   2015 USENIX Annual Technical Conference (USENIX ATC 15) , 2015

   Abs DOI

   Dynamic resource scaling enables provisioning extra resources during peak loads and saving energy by reclaiming those resources during off-peak times. Scaling the number of CPU cores is particularly valuable as it allows power savings during low-usage periods. Current systems perform scaling with a slow hotplug mechanism, which was primarily designed to remove or replace faulty cores. The high cost of scaling is reflected in power management policies that perform scaling at coarser time scales to amortize the high reconfiguration latency.

10. vCache: Architectural support for transparent and isolated virtual LLCs in virtualized environments

    Daehoon Kim, Hwanju Kim, Nam Sung Kim, and Jaehyuk Huh

    2015

    Abs DOI

11. Workload-aware optimal power allocation on single-chip heterogeneous processors

    Jae Young Jang, Hao Wang, Euijin Kwon, Jae W Lee, and Nam Sung Kim

    IEEE Transactions on Parallel and Distributed Systems , 2015

    Abs DOI

12. HPCA

    Alloy: Parallel-serial memory channel architecture for single-chip heterogeneous processor systems

    Hao Wang, Chang-Jae Park, Gyung-su Byun, Jung Ho Ahn, and Nam Sung Kim

    2015 IEEE 21st International Symposium on High Performance Computer Architecture (HPCA) , 2015

    Abs DOI

13. PATENT

    Memory-link compression for graphic processor unit

    Nam Sung Kim

    2015

    Abs DOI

    (57) ABSTRACT A graphic processing unit having multiple computational elements flexibly interconnected to memory elements pro vides for data compressors/decompressors in the memory channels communicating between the computational ele ments and memory elements to provide an effective increase in bandwidth of those connections by the compression of data transferred thereon.

14. Comparison of single-isa heterogeneous versus wide dynamic range processors for mobile applications

    Hamid Reza Ghasemi, Ulya R Karpuzcu, and Nam Sung Kim

    2015 33rd IEEE International Conference on Computer Design (ICCD) , 2015

    Abs DOI

15. NDA: Near-DRAM acceleration architecture leveraging commodity DRAM devices and standard memory modules

    Amin Farmahini Farahani, Jung Ho Ahn, Katherine Morrow, and Nam Sung Kim

    HPCA , 2015

    Abs DOI

    Energy consumed for transferring data across the processor memory hierarchy constitutes a large fraction of total system energy consumption, and this fraction has steadily increased with technology scaling. In this paper, we propose near-DRAM acceleration (NDA) architectures, which process data using accelerators 3D-stacked on DRAM devices comprising off-chip main memory modules. NDA transfers most data through high-bandwidth and low-energy 3D interconnects between accelerators and DRAM devices instead of low-bandwidth and high-energy off-chip interconnects between a processor and DRAM devices, substantially reducing energy consumption and improving performance. Unlike previous near-memory processing architectures, NDA is built upon commodity DRAM devices; apart from inserting through-silicon vias (TSVs) to 3D-interconnect DRAM devices and accelerators, NDA requires minimal changes to the commodity DRAM device and standard memory module architectures. This allows NDA to be more easily adopted in both existing and emerging systems. Our experiments demonstrate that, on average, our NDA-based system consumes 46% (68%) lower (data transfer) energy at 1.67× higher performance than a system that integrates the same accelerator logic within the processor itself.

16. Joint optimisation of computational accuracy and algorithm parameters for energy‐efficient recognition algorithms

    Heesung Lim, Taejoon Park, and Nam Sung Kim

    Electronics Letters , 2015

    Abs DOI

    In this reported work, firstly, the artificial neural network (ANN) is taken as a target recognition algorithm and then jointly, the computational accuracy and an algorithm parameter (i.e. the number of hidden nodes) are optimised to minimise the overall energy consumption of ANN evaluations. This joint optimisation is motivated by the observation that both the computational accuracy and the algorithm parameter affect recognition accuracy and energy consumption. The evaluation shows that the jointly optimised computational accuracy and the algorithm parameter reduces the energy consumption of ANN evaluations by 79% at the same recognition target, compared with optimising only the algorithm parameter with precise computations. Furthermore, it is demonstrated that to evaluating ANNs with reduced computational accuracy, recognition accuracy is further improved by training the ANNs with reduced computational accuracy. This allows reduction of energy consumption by 86%.

17. Online and Operand-Aware Detection of Failures Utilizing False Alarm Vectors

    Amir Yazdanbakhsh, David Palframan, Azadeh Davoodi, Nam Sung Kim, and Mikko Lipasti

    2015

    Abs DOI

2014

1. VLSI

   Energy-efficient approximate multiplication for digital signal processing and classification applications

   Srinivasan Narayanamoorthy, Hadi Asghari Moghaddam, Zhenhong Liu, Taejoon Park, and Nam Sung Kim

   IEEE transactions on very large scale integration (VLSI) systems , 2014

   Abs DOI

   The need to support various digital signal processing (DSP) and classification applications on energy-constrained devices has steadily grown. Such applications often extensively perform matrix multiplications using fixed-point arithmetic while exhibiting tolerance for some computational errors. Hence, improving the energy efficiency of multiplications is critical. In this brief, we propose multiplier architectures that can tradeoff computational accuracy with energy consumption at design time. Compared with a precise multiplier, the proposed multiplier can consume 58% less energy/op with average computational error of %. Finally, we demonstrate that such a small computational error does not notably impact the quality of DSP and the accuracy of classification applications.

2. Sleepscale: Runtime joint speed scaling and sleep states management for power efficient data centers

   Yanpei Liu, Stark C Draper, and Nam Sung Kim

   ACM SIGARCH Computer Architecture News , 2014

   Abs DOI

3. CAL

   DRAMA: An architecture for accelerated processing near memory

   Amin Farmahini-Farahani, Jung Ho Ahn, Katherine Morrow, and Nam Sung Kim

   IEEE Computer Architecture Letters , 2014

   Abs DOI

   Improving energy efficiency is crucial for both mobile and high-performance computing systems while a large fraction of total energy is consumed to transfer data between storage and processing units. Thus, reducing data transfers across the memory hierarchy of a processor (i.e., off-chip memory, on-chip caches, and register file) can greatly improve the energy efficiency. To this end, we propose an architecture, DRAMA, that 3D-stacks coarse-grain reconfigurable accelerators (CGRAs) atop off-chip DRAM devices. DRAMA does not require changes to the DRAM device architecture, apart from through-silicon vias (TSVs) that connect the DRAM device’s internal I/O bus to the CGRA layer. We demonstrate that DRAMA can reduce the energy consumption to transfer data across the memory hierarchy by 66-95 percent while achieving speedups of up to 18× over a commodity processor.

4. QoS-aware dynamic resource allocation for spatial-multitasking GPUs

   Paula Aguilera, Katherine Morrow, and Nam Sung Kim

   2014 19th Asia and South Pacific Design Automation Conference (ASP-DAC) , 2014

   Abs DOI

5. Fair share: Allocation of GPU resources for both performance and fairness

   Paula Aguilera, Katherine Morrow, and Nam Sung Kim

   2014 IEEE 32nd International Conference on Computer Design (ICCD) , 2014

   Abs DOI

6. ISCA

   Row-buffer decoupling: A case for low-latency DRAM microarchitecture

   O Seongil, Young Hoon Son, Nam Sung Kim, and Jung Ho Ahn

   2014 ACM/IEEE 41st International Symposium on Computer Architecture (ISCA) , 2014

   Abs DOI

7. Process variation-aware workload partitioning algorithms for GPUs supporting spatial-multitasking

   Paula Aguilera, Jungseob Lee, Amin Farmahini-Farahani, Katherine Morrow, Michael Schulte, and Nam Sung Kim

   2014 Design, Automation & Test in Europe Conference & Exhibition (DATE) , 2014

   Abs DOI

8. HPCA

   Precision-aware soft error protection for GPUs

   David J Palframan, Nam Sung Kim, and Mikko H Lipasti

   2014 IEEE 20th International Symposium on High Performance Computer Architecture (HPCA) , 2014

   Abs DOI

9. HPCA

   Accordion: Toward soft near-threshold voltage computing

   Ulya R Karpuzcu, Ismail Akturk, and Nam Sung Kim

   2014 IEEE 20th International Symposium on High Performance Computer Architecture (HPCA) , 2014

   Abs DOI

10. Memory scheduling towards high-throughput cooperative heterogeneous computing

    Hao Wang, Ripudaman Singh, Michael J Schulte, and Nam Sung Kim

    2014

    Abs DOI

11. RCS: Runtime resource and core scaling for power-constrained multi-core processors

    Hamid Reza Ghasemi and Nam Sung Kim

    2014

    Abs DOI

12. PATENT

    Leakage power management using programmable power gating transistors and on-chip aging and temperature tracking circuit

    Nam Sung Kim

    2014

    Abs DOI

    The number of power-gating transistors on an integrated cir cuit used for power reduction in a sleep mode is controlled during a wake state to adjust the current flow and hence Voltage drop across the power-gating transistors as a function of aging of these transistors and/or a function of temperature of the integrated circuit. In this way, the Supply Voltage to the integrated circuit may be better tailored to minimize current leakage when the integrated circuit is young or operating at low temperatures.

13. Optimization of a cell counting algorithm for mobile point-of-care testing platforms

    DaeHan Ahn, Nam Sung Kim, SangJun Moon, Taejoon Park, and Sang Hyuk Son

    Sensors , 2014

    Abs DOI

    In a point-of-care (POC) setting, it is critically important to reliably count the number of specific cells in a blood sample. Software-based cell counting, which is far faster than manual counting, while much cheaper than hardware-based counting, has emerged as an attractive solution potentially applicable to mobile POC testing. However, the existing software-based algorithm based on the normalized cross-correlation (NCC) method is too time- and, thus, energy-consuming to be deployed for battery-powered mobile POC testing platforms. In this paper, we identify inefficiencies in the NCC-based algorithm and propose two synergistic optimization techniques that can considerably reduce the runtime and, thus, energy consumption of the original algorithm with negligible impact on counting accuracy. We demonstrate that an Android™ smart phone running the optimized algorithm consumes 11.5× less runtime than the original algorithm.

14. Energy-efficient reconfigurable cache architectures for accelerator-enabled embedded systems

    Amin Farmahini-Farahani, Nam Sung Kim, and Katherine Morrow

    2014 IEEE International Symposium on Performance Analysis of Systems and Software (ISPASS) , 2014

    Abs DOI

15. Maximizing throughput of power/thermal-constrained processors by balancing power consumption of cores

    Abhishek A Sinkar, Hao Wang, and Nam Sung Kim

    Fifteenth International Symposium on Quality Electronic Design , 2014

    Abs DOI

16. Multiplier supporting accuracy and energy trade‐offs for recognition applications

    Nam Sung Kim, Taejoon Park, Srinivasan Narayanamoorthy, and Hadi Asgharimoghaddam

    Electronics letters , 2014

    Abs DOI

    The need to support various recognition applications on energy‐constrained mobile computing devices has steadily grown. Exploiting common characteristics of recognition algorithms, a very energy‐efficient multiplier that can support a runtime trade‐off between computational accuracy and energy consumption is proposed. Compared to a precise multiplier, the proposed multiplier consumes 11.6×–3.2× less energy per multiplication while achieving 82–99% of the computational accuracy with negligible negative impact on recognition accuracy for three different recognition applications.

17. PATENT

    Memory cell supply voltage control based on error detection

    Muhammad Khellah, Dinesh Somasekhar, Yibin Ye, Nam Sung Kim, and Vivek De

    2014

    Abs DOI

    Described herein is an apparatus for adjusting a power Supply level for a memory cell to improve stability of a memory unit. The apparatus comprises memory circuitry including memory cells, error detection circuitry to detect error in data stored by memory cells of the memory circuitry, and Supply Voltage control circuitry to increase Supply Voltage for one or more memory cells of the memory circuitry based at least in part on detected error.

18. Sleepscale: Runtime joint speed scaling and sleep states management for power efficient data centers. In 2014 ACM/IEEE 41st International Symposium on Computer Architecture (ISCA)

    Yanpei Liu, Stark C Draper, and Nam Sung Kim

    2014

    Abs DOI

19. Energy-efficient pixel-arithmetic

    Syed Zohaib Gilani, Nam Sung Kim, and Michael Schulte

    IEEE Transactions on Computers , 2014

    Abs DOI

    With the advent of pervasive computing, the performance requirements of visual applications have increased significantly. On the other hand, the energy budget for future devices may decrease due to reduced form factors and thus smaller battery sizes. It is thus imperative to improve energy efficiency of visual applications to meet their stringent demands in energy-constrained devices. This paper presents a novel energy-efficient floating-point unit (E 2FPU) that dynamically detects multiplications in which at least one operand is an integer power of two or the sum of consecutive integer powers of two (POW2 operands) and executes them on the E2FPU. We show that POW2 operands are extremely common in visual applications. For non-POW2 operands, we propose dynamic approximation of suitable operands as the closest POW2 operand with the same exponent while ensuring a small and limited approximation error. Finally, we exploit the limited dynamic range of pixel values to enhance the E2FPU to support low-energy FP addition for a sub-set of the single-precision floating-point dynamic range. The proposed E2 FPU can reduce execution energy by 35 percent with a negligible impact on the peak signal to noise ratio. Overall, at the chip-level, our approaches yield a 12 percent dynamic energy reduction for a graphics processing unit (GPU).

20. Quantitative comparison of the power reduction techniques for samsung reconfigurable processor

    Hoyoung Kim, Soojung Ryu, Abhishek Sinkar, and Nam Sung Kim

    2014 IEEE International Symposium on Circuits and Systems (ISCAS) , 2014

    Abs DOI

21. Low-cost scratchpad memory organizations using heterogeneous cell sizes for low-voltage operations

    Syed Gilani, Taejoon Park, and Nam Sung Kim

    Microprocessors and Microsystems , 2014

    Abs DOI

22. PATENT

    Energy efficient processor having heterogeneous cache

    Nam Sung Kim and Stark C Draper

    2014

    Abs DOI

2013

1. ISCA

   GPUWattch: Enabling energy optimizations in GPGPUs

   Jingwen Leng, Tayler Hetherington, Ahmed ElTantawy, Syed Gilani, Nam Sung Kim, Tor M Aamodt, and Vijay Janapa Reddi

   International Symposium on Computer Architecture , 2013

   Abs DOI

   General-purpose GPUs (GPGPUs) are becoming prevalent in mainstream computing, and performance per watt has emerged as a more crucial evaluation metric than peak performance. As such, GPU architects require robust tools that will enable them to quickly explore new ways to optimize GPGPUs for energy efficiency. We propose a new GPGPU power model that is configurable, capable of cycle-level calculations, and carefully validated against real hardware measurements. To achieve configurability, we use a bottom-up methodology and abstract parameters from the microarchitectural components as the model’s inputs. We developed a rigorous suite of 80 microbenchmarks that we use to bound any modeling uncertainties and inaccuracies. The power model is comprehensively validated against measurements of two commercially available GPUs, and the measured error is within 9.9% and 13.4% for the two target GPUs (GTX 480 and Quadro FX5600). The model also accurately tracks the power consumption trend over time. We integrated the power model with the cycle-level simulator GPGPU-Sim and demonstrate the energy savings by utilizing dynamic voltage and frequency scaling (DVFS) and clock gating. Traditional DVFS reduces GPU energy consumption by 14.4% by leveraging within-kernel runtime variations. More finer-grained SM cluster-level DVFS improves the energy savings from 6.6% to 13.6% for those benchmarks that show clustered execution behavior. We also show that clock gating inactive lanes during divergence reduces dynamic power by 11.2%.

2. HPCA

   Energysmart: Toward energy-efficient manycores for near-threshold computing

   Ulya R Karpuzcu, Abhishek Sinkar, Nam Sung Kim, and Josep Torrellas

   2013 IEEE 19th International Symposium on High Performance Computer Architecture (HPCA) , 2013

   Abs DOI

   While Near-Threshold Voltage Computing (NTC) is a promising approach to push back the manycore power wall, it suffers from a high sensitivity to parameter variations. One possible way to cope with variations is to use multiple on-chip voltage (Vdd) domains. However, this paper finds that such an approach is energy inefficient. Consequently, for NTC, we propose a manycore organization that has a single Vdd domain and relies on multiple frequency domains to tackle variation. We call it EnergySmart. For this approach to be competitive, it has to be paired with effective core assignment strategies and also support fine-grain (i.e., short-interval) DVFS. This paper shows that, at NTC, a simple chip with a single Vdd domain can deliver a higher performance per watt than one with multiple Vdd domains.

3. Coping with parametric variation at near-threshold voltages

   Ulya R Karpuzcu, Nam Sung Kim, and Josep Torrellas

   IEEE Micro , 2013

   Abs DOI

   Near-threshold voltage computing (NTC) promises significant improvement in energy efficiency. Unfortunately, when compared to conventional, super-threshold voltage computing (STC), NTC is more sensitive to parametric variation. This results in not only slower and leakier cores, but also substantial speed and power differences between the cores in a many-core chip. NTC’s potential cannot be unlocked without addressing the higher impact of variation. To confront variation at the architecture level, the authors introduce a parametric variation model for NTC. They then use the model to show the shortcomings of adapting state-of-the-art STC techniques for variation mitigation to NTC. Finally, they discuss how to tailor variation mitigation to NTC.

4. Reevaluating the latency claims of 3D stacked memories

   Daniel W Chang, Gyungsu Byun, Hoyoung Kim, Minwook Ahn, Soojung Ryu, Nam S Kim, and Michael Schulte

   2013 18th Asia and South Pacific Design Automation Conference (ASP-DAC) , 2013

   Abs DOI

5. Exploiting GPU peak-power and performance tradeoffs through reduced effective pipeline latency

   Syed Zohaib Gilani, Nam Sung Kim, and Michael J Schulte

   2013

   Abs DOI

6. Improving throughput of power-constrained many-core processors based on unreliable devices

   Hao Wang and Nam Sung Kim

   IEEE Micro , 2013

   Abs DOI

   Using slightly less device-level redundancy than is necessary to make all processor cores defect free actually makes cores smaller, faster, and more power efficient. Under the same power and yield constraints, a carbon nanotube processor with less device-level redundancy can provide 1.75x higher throughput, while also being nearly 2x smaller than a similar processor that has more device-level redundancy and makes all cores defect free.

7. CAD

   Improving platform energy-chip area trade-off in near-threshold computing environment

   Hao Wang, Abhishek A Sinkar, and Nam Sung Kim

   2013 IEEE/ACM International Conference on Computer-Aided Design (ICCAD) , 2013

   Abs DOI

8. Resilient high-performance processors with spare ribs

   David J Palframan, Nam Sung Kim, and Mikko H Lipasti

   IEEE Micro , 2013

   Abs DOI

   Resilience to defects and parametric variations is of the utmost concern for future technology generations. Traditional redundancy to repair defects, however, can incur performance penalties owing to multiplexing. This article presents a processor design that incorporates bit-sliced redundancy along the data path. This approach makes it possible to tolerate defects without hurting performance, because the same bit offset is left unused throughout the execution core. In addition, the authors use this approach to enhance performance by avoiding excessively slow critical paths created by random delay variations. Adding a single bit slice, for instance, can reduce the delay overhead of random process variations by 10 percent while providing fault tolerance for 15 percent of the execution core.

9. Queuing theoretic analysis of power-performance tradeoff in power-efficient computing

   Yanpei Liu, Stark C Draper, and Nam Sung Kim

   2013 47th Annual Conference on Information Sciences and Systems (CISS) , 2013

   Abs DOI

   In this paper we study the power-performance relationship of power-efficient computing from a queuing theoretic perspective. We investigate the interplay of several system operations including processing speed, system on/off decisions, and server farm size. We identify that there are oftentimes ’sweet spots’ in power-efficient operations: there exist optimal combinations of processing speed and system settings that maximize power efficiency. For the single server case, a widely deployed threshold mechanism is studied. We show that there exist optimal processing speed and threshold value pairs that minimize the power consumption. This holds for the threshold mechanism with job batching. For the multi-server case, it is shown that there exist best processing speed and server farm size combinations.

10. CAD

    Dynamic bandwidth scaling for embedded DSPs with 3D-stacked DRAM and wide I/Os

    Daniel W Chang, Young Hoon Son, Jung Ho Ahn, Hoyoung Kim, Minwook Ahn, Michael J Schulte, and Nam Sung Kim

    2013 IEEE/ACM International Conference on Computer-Aided Design (ICCAD) , 2013

    Abs DOI

11. REEL: Reducing effective execution latency of floating point operations

    Vignyan Reddy, Syed Zohaib Gilani, Erika Gunadi, Nam Sung Kim, Michael J Schulte, and Mikko H Lipasti

    International Symposium on Low Power Electronics and Design (ISLPED) , 2013

    Abs DOI

    The height of the dynamic dependence graph of a program, as executed by a processor, determines the minimum bound on the execution time. This height can be decreased by reducing the effective execution latency of operations that form dependence chains in the graph. In this paper, we propose a technique called REEL to reduce overall latency of chains of dependent floating point (FP) operations by increasing the throughput of computation. REEL comprises of a high-throughput floating point unit (HFP) that allows early issue of an FP Add that is dependent on another FP Add or FP Multiply. This is complemented by instruction scheduler modifications that allow early issue of dependent FP Adds, and a novel checker logic that corrects any precision errors. Unlike conventional static operation fusion, like fused Multiply-Add (FMA), there are no changes to the instruction set to enable utilization of the new hardware, and no recompilation is necessary. Furthermore, unlike ISA-level FMA, our technique produces results that are bit compatible while boosting performance of Add-Add dependence pairs in addition to Multiply-Add pairs. Our evaluation of REEL using CFP2006 benchmarks shows an average performance gain of 7.6% and maximum performance gain of 17% while consuming 1.2% lower energy.

12. IMPROVING MEMORY RELIABILITY POWER AND PERFORMANCE USING MIXED-CELL DESIGNS.

    Alaa R Alameldeen, Nam Sung Kim, Samira M Khan, Hamid Reza Ghasemi, Chris Wilkerson, Jaydeep Kulkarni, and Daniel A Jiménez

    Intel Technology Journal , 2013

    Abs

    In this article, we show that mixed-cell memory designs could play a key role in achieving the right balance between performance, power, and reliability for single-core and multi-core systems. In such designs, part of the memory structure is built with cells that are more robust and failure-resistant, while the rest is designed using traditional cells. Robust cells ensure resiliency under low-voltage conditions to protect the most vulnerable data, while the rest of the memory structure could be used to store redundant data to improve performance. We demonstrate this concept using two specific examples:(1) A cache system that only turns on the robust portion at low-voltage, achieving good reliability and power savings while providing high-voltage performance improvements;(2) A cache system that uses the whole cache (including the non-robust portion) at low voltage, achieving good performance and reliability while not exceeding power limits. While the specific examples we explore in this article are cache-related, the same concept could be used throughout the entire memory hierarchy to improve memory resiliency without sacrificing performance or energy efficiency.

13. Optimal Power Allocation for Multiprogrammed Workloads on Single-Chip Heterogeneous Processors

    Euijin Kwon, Jae Young Jang, Jae W Lee, and Nam Sung Kim

    2013 , 2013

    Abs DOI

    Due to the growing requirements of performance and energy efficiency, processor manufacturers now integrate both CPU and GPU cores onto a single chip and support fine-grain dynamic voltage/frequency scaling (DVFS). To effectively utilize abundant hardware resources, the CPU and the GPU on a single-chip heterogeneous processor (SCHP) are required to execute multiple programs simultaneously via both space and time sharing. Since the performance is primarily limited by the available power budget, it is crucial to optimally allocate it across the CPU and the GPU by considering both workload characteristics and evaluation metrics. This paper advocates adaptive, workload-aware power allocation for multiprogrammed workloads on an SCHP with fine-grain DVFS capabilities. Using a detailed cycle-level SCHP simulator, we first demonstrate that workload-aware power allocation can improve throughput and energy efficiency by an average of 12% and 18%, respectively, over workload-oblivious uniform power allocation for 11 multiprogrammed workloads. We also propose two efficient run-time algorithms that find an optimal voltage/frequency setting for the two metrics(ie, throughput and energy efficiency), which allow the SCHP to reach the optimal or near-optimal setting within four iterations.

2012

1. HPCA

   The case for GPGPU spatial multitasking

   Jacob T Adriaens, Katherine Compton, Nam Sung Kim, and Michael J Schulte

   IEEE International Symposium on High-Performance Comp Architecture , 2012

   Abs DOI

2. Lossless and lossy memory I/O link compression for improving performance of GPGPU workloads

   Vijay Sathish, Michael J Schulte, and Nam Sung Kim

   2012

   Abs DOI

3. VARIUS-NTV: A microarchitectural model to capture the increased sensitivity of manycores to process variations at near-threshold voltages

   Ulya R Karpuzcu, Krishna B Kolluru, Nam Sung Kim, and Josep Torrellas

   IEEE/IFIP International Conference on Dependable Systems and Networks (DSN 2012) , 2012

   Abs DOI

4. Power-efficient computing for compute-intensive GPGPU applications

   Syed Zohaib Gilani, Nam Sung Kim, and Michael J Schulte

   2012

   Abs DOI

5. Workload and power budget partitioning for single-chip heterogeneous processors

   Hao Wang, Vijay Sathish, Ripudaman Singh, Michael J Schulte, and Nam Sung Kim

   2012

   Abs DOI

6. Cost-effective power delivery to support per-core voltage domains for power-constrained processors

   Hamid Reza Ghasemi, Abhishek A Sinkar, Michael J Schulte, and Nam Sung Kim

   2012

   Abs DOI

7. Workload-aware voltage regulator optimization for power efficient multi-core processors

   Abhishek A Sinkar, Hao Wang, and Nam Sung Kim

   2012 Design, Automation & Test in Europe Conference & Exhibition (DATE) , 2012

   Abs DOI

8. A linear algebra core design for efficient level-3 blas

   Ardavan Pedram, Syed Zohaib Gilani, Nam Sung Kim, Robert Van De Geijn, Michael Schulte, and Andreas Gerstlauer

   2012 IEEE 23rd International Conference on Application-Specific Systems, Architectures and Processors , 2012

   Abs DOI

9. PATENT

   Method and apparatus for optimizing clock speed and power dissipation in multicore architectures

   Nam Sung Kim

   2012

   Abs DOI

10. PATENT

    Sequential circuit with error detection

    Keith Bowman, James Tachanz, Nam Sung Kim, Janice Lee, Chris Wilkerson, Shih-Lien L Lu, Tanay Karnlk, and Vivek De

    2012

    Circuit Memory

    Abs

    BACKGROUNDMicroprocessor clock frequency (FCLK) is typically based on expected worst-case operating parameters such as Supply voltage (VCC) droop and temperature. However, since oper ating parameters are normally not actually at the worst-case extremes, the operating FCLK may be unnecessarily limited.

11. Virtual floating-point units for low-power embedded processors

    Syed Zohaib Gilani, Nam Sung Kim, and Michael Schulte

    2012 IEEE 23rd International Conference on Application-Specific Systems, Architectures and Processors , 2012

    Abs DOI

12. Mitigating random variation with spare RIBs: Redundant intermediate bitslices

    David J Palframan, Nam Sung Kim, and Mikko H Lipasti

    IEEE/IFIP International Conference on Dependable Systems and Networks (DSN 2012) , 2012

    Abs DOI

13. Parameter variation at near threshold voltage: The power efficiency versus resilience tradeoff

    Josep Torrellas, Nam Sung Kim, and Radu Teodorescu

    University of Illinois, Tech. Rep , 2012

    Abs

    The strongest lever that we have to improve the power-efficiency of CMOS devices is to reduce their supply voltage (VDD). When VDD is only a bit higher than the threshold voltage (VTH), we attain a fairly optimal operation point, where energy per operation is low and switching delay is not too high. This region, known as Near Threshold Voltage (NTV), is attractive because it can offer a 1-2 orders of magnitude reduction in power relative to the conventional Super Threshold Voltage (STV) operation [6, 10, 32]. While NTV is appealing, it has three major limitations: lower speed, higher share of static power, and higher impact of process variations. The first two are well known and likely to be worked around—especially for highly-parallel workloads. The third one, however, is less known and harder to tame. Process variations are the deviation of the values of device parameters (such as a transistor’s VTH) from their nominal specifications. They lead to circuits and chips with lower speed, higher power consumption and lower resilience. The negative impact of process variations is much higher at NTV: small changes in VTH induce large changes in transistor speed and power consumption due to an intrinsic effect coming from having VDD so close to VTH.

14. VLSI

    Clamping virtual supply voltage of power-gated circuits for active leakage reduction and gate-oxide reliability

    Abhishek Sinkar, Taejoon Park, and Nam Sung Kim

    IEEE transactions on very large scale integration (VLSI) systems , 2012

    Abs DOI

    In an integrated circuit (IC) adopting a power-gating (PG) technique, the virtual supply voltage (VVDD) is susceptible to: 1) negative-bias temperature instability (NBTI) degradation that weakens the PG device over time and 2) temporal temperature variation that affects active leakage current (thus total current) of the IC. The PG device is sized to guarantee a minimum VVDD level over the chip lifetime. Thus, the NBTI degradation and the worst-case total current at high-temperature must be considered for sizing the PG device. This leads to higher VVDD (thus active leakage power) than necessary in early chip lifetime and/or at low temperature, negatively impacting the gate-oxide reliability of transistors. To reduce active leakage power increase and improve the gate-oxide reliability due to these effects, we propose two techniques that adjust the strength of a PG device based on its usage and IC’s temperature at runtime. We demonstrate the efficacy of these techniques with an experimental setup using a 32-nm technology model in the presence of within-die spatial process and temperature variations. On an average of 100 die samples, they can reduce dynamic and active leakage power by up to 3.7% and 10% in early chip lifetime. Finally, these techniques also reduce the oxide failure rate by up to 5% across process corners over a period of 7 years.

2011

1. Improving throughput of power-constrained GPUs using dynamic voltage/frequency and core scaling

   Jungseob Lee, Vijay Sathisha, Michael Schulte, Katherine Compton, and Nam Sung Kim

   2011 International Conference on Parallel Architectures and Compilation Techniques , 2011

   Abs DOI

2. VLSI

   Analyzing the impact of joint optimization of cell size redundancy and ECC on low-voltage SRAM array total area

   Nam Sung Kim, Stark C Draper, Shi-Ting Zhou, Sumeet Katariya, Hamid Reza Ghasemi, and Taejoon Park

   IEEE Transactions on Very Large Scale Integration (VLSI) Systems , 2011

   Abs DOI

   The increasing power consumption of processors has made power reduction a first-order priority in processor design. Voltage scaling is one of the most powerful power-reduction techniques introduced to date, but is limited to some minimum voltage VDDMIN. Below VDDMIN on-chip SRAM cells cannot all operate reliably due to increased process variability with technology scaling. The use of larger SRAM cells, which are less sensitive to process variability, allows a reduction in VDDMIN. However, since the large-scale memory structures such as last-level caches (LLCs) often determine the VDDMIN of processors, these structures cannot afford to use large SRAM cells due to the resulting increase in die area. In this paper we first propose a joint optimization of LLC cell size, the number of redundant cells, and the strength of error-correction coding (ECC) to minimize total SRAM area while meeting yield and VDDMIN targets. The joint use of redundant cells and ECC enables the use of smaller cell sizes while maintaining design targets. Smaller cell sizes more than make up for the extra cells required by redundancy and ECC. In 32-nm technology our joint approach yields a 27% reduction in total SRAM area (including the extra cells) when targeting 90% yield and 600 mV VDDMIN. Second, we demonstrate that the ECC used to repair defective cells can be combined with a simple architectural technique, which can also fix particle-induced soft errors, without increasing ECC strength or processor runtime.

3. HPCA

   Low-voltage on-chip cache architecture using heterogeneous cell sizes for high-performance processors

   Hamid Reza Ghasemi, Stark C Draper, and Nam Sung Kim

   2011 IEEE 17th International Symposium on High Performance Computer Architecture , 2011

   Abs DOI

4. Analyzing throughput of GPGPUs exploiting within-die core-to-core frequency variation

   Jungseob Lee, Paritosh Pratap Ajgaonkar, and Nam Sung Kim

   (IEEE ISPASS) IEEE International Symposium on Performance Analysis of Systems and Software , 2011

   Abs DOI

5. VLSI

   Analyzing potential throughput improvement of power-and thermal-constrained multicore processors by exploiting DVFS and PCPG

   Jungseob Lee and Nam Sung Kim

   IEEE transactions on very large scale integration (VLSI) systems , 2011

   Abs DOI

6. Scratchpad memory optimizations for digital signal processing applications

   Syed Z Gilani, Nam Sung Kim, and Michael Schulte

   2011 Design, Automation & Test in Europe , 2011

   Abs DOI

7. Energy-efficient floating-point arithmetic for software-defined radio architectures

   Syed Zohaib Gilani, Nam Sung Kim, and Michael Schulte

   ASAP 2011-22nd IEEE International Conference on Application-specific Systems, Architectures and Processors , 2011

   Abs DOI

8. Time redundant parity for low-cost transient error detection

   David J Palframan, Nam Sung Kim, and Mikko H Lipasti

   2011 Design, Automation & Test in Europe , 2011

   Abs DOI

9. PATENT

   Memory cell supply voltage control based on error detection

   Khellah Muhammad, Dinesh Somasekhar, Yibin Ye, Nam Sung Kim, and Vivek De

   2011

   Abs

   For one embodiment, an apparatus comprises memory cir cuitry including memory cells, error detection circuitry to detect error in data stored by memory cells of the memory circuitry, and Supply Voltage control circuitry to increase Supply Voltage for one or more memory cells of the memory circuitry based at least in part on detected error. Other embodiments have one or more other features.

10. Energy-efficient floating-point arithmetic for digital signal processors

    Syed Zohaib Gilani, Nam Sung Kim, and Michael Schulte

    2011 Conference Record of the Forty Fifth Asilomar Conference on Signals, Systems and Computers (ASILOMAR) , 2011

    Abs DOI

11. A low cost approach to calibrate on-chip thermal sensors

    Krishna Bharath, Chunhua Yao, Nam Sung Kim, Parameswaran Ramanathan, and Kewal K Saluja

    2011 12th International Symposium on Quality Electronic Design , 2011

    Abs DOI

    Thermal management of Integration Circuit (IC) becomes more and more important with the scaling of CMOS technology and dramatic increase of power density. Performance of thermal management application is highly related to the accuracy of the temperature monitoring devices. However, due to increasing process variations and parameter drifts, temperature measurements by on-chip thermal sensors may not be accurate unless these sensors are calibrated before shipping the devices to the users. Existing calibration methods impose large time cost and cannot be used for in-field calibration. In this paper, we propose a technique to calibrate on-chip thermal sensors during manufacturing as well as in-field. We model the cyclic dependence between leakage power and temperature and perform calibrations to evaluate the calibration accuracy. Experimental results on 100 dies with different process variation parameters and three corner cases show that our technique have very high accuracy.

12. Analyzing the performance and energy impact of 3D memory integration on embedded DSPs

    Daniel W Chang, Nam S Kim, and Michael J Schulte

    2011 International Conference on Embedded Computer Systems: Architectures, Modeling and Simulation , 2011

    Abs DOI

    Recently, three-dimensional integration technology has allowed researchers and designers to explore novel architectures for computing systems. Due to the memory-intensive nature of signal processing systems, DSPs can greatly benefit from 3D memory integration technology realized by vertically stacking high-density memory below processing cores. In this paper, we analyze the energy and performance impacts of 3D memory integration in DSP systems by exploring a wide variety of memory configurations the technology enables. Our analysis demonstrates that a 3D memory hierarchy can increase performance by an average of 20.6% while decreasing energy by 6.7% when compared to our baseline 2D memory hierarchy, with a processor similar to a Texas Instrument C67× DSP. This proposed memory architecture also allows us to scale the voltage and frequency to decrease energy consumption by an average of 23% without decreasing performance.

13. VLSI

    Maximizing frequency and yield of power-constrained designs using programmable power-gating

    Nam Sung Kim, Abhishek Sinkar, Jun Seomun, and Youngsoo Shin

    IEEE transactions on very large scale integration (VLSI) systems , 2011

    Abs DOI

14. AVS-aware power-gate sizing for maximum performance and power efficiency of power-constrained processors

    Abhishek Sinkar and Nam Sung Kim

    16th Asia and South Pacific Design Automation Conference (ASP-DAC 2011) , 2011

    Abs DOI

15. A Microarchitectural Model of Process Variation for Near-threshold Computing

    Ulya R Karpuzcu, Krishna Kolluru, Nam Sung Kim, and Josep Torrellas

    2011

    Abs DOI

    Near-Threshold Voltage Computing (NTC), where the supply voltage is only slightly higher than the transistors’ threshold voltage, is a promising approach to push back the many-core power wall. A key NTC shortcoming is the higher sensitivity to parameter (process, supply voltage, temperature) variation. This report introduces a microarchitectural model to characterize increased sensitivity to process variation at near-thresold voltages.

16. Spare RIBs: Redundant Intermediate Bitslices

    David J Palframan, Nam Sung Kim, and Mikko H Lipasti

    2011

    Abs DOI

    Delay variations due to dopant fluctuations are expected to become more prominent in future technology generations. To regain performance lost due to within-die variations, many architectural techniques propose modified timing schemes such as time borrowing or variable latency execution. As an alternative, we propose introducing redundancy along the processor datapath in the form of one or more extra bitslices. This approach allows us to insert a dummy slice into the datapath to avoid a slow critical path created by delay variations. Using the ALU critical path as an example, we demonstrate that our technique can reduce the delay penalty due to random variations by 10% with an area overhead of approximately 2%.

2010

1. MICRO

   Combating aging with the colt duty cycle equalizer

   Erika Gunadi, Abhisek A Sinkar, Nam Sung Kim, and Mikko H Lipasti

   2010 43rd Annual IEEE/ACM International Symposium on Microarchitecture , 2010

   Architecture Memory

   Abs DOI

2. PATENT

   Delay fault detection using latch with error sampling

   James W Tschanz, Keith A Bowman, Nam Sung Kim, Chris Wilkerson, Shih-Lien L Lu, and Tanay Karnik

   2010

   Abs DOI

   Some embodiments provide sampling of a data signal output from a path stage using a latch, sampling of the data signal output from the path stage using an edge-triggered flip-flop, comparing a first value output by the latch with a second value output by the edge-triggered flip-flop, and generating an error signal if the first value is different from the second value.

3. Runtime temperature-based power estimation for optimizing throughput of thermal-constrained multi-core processors

   Dongkeun Oh, Nam Sung Kim, Charlie Chung Ping Chen, Azadeh Davoodi, and Yu Hen Hu

   2010 15th Asia and South Pacific Design Automation Conference (ASP-DAC) , 2010

   Abs DOI

4. CAD

   Optimal algorithm for profile-based power gating: A compiler technique for reducing leakage on execution units in microprocessors

   Danbee Park, Jungseob Lee, Nam Sung Kim, and Taewhan Kim

   2010 IEEE/ACM International Conference on Computer-Aided Design (ICCAD) , 2010

   Architecture FPGA

   Abs DOI

   This paper proposes a compiler-based solution to the problem of inserting power gating instructions into code to control activation/deactivation (i.e., ON/OFF) of functional units in microprocessor during the code execution, so that the leakage power is maximally saved. Precisely, based on an execution profile of code containing conditional branches and/or loops, we propose a polynomial time optimal algorithm, called PG-instr, of inserting ON/OFF instructions into code with the objective of minimizing the expected total leakage power while considering the power and delay overhead on power gating.

5. Analyzing and minimizing effects of temperature variation and NBTI on active leakage power of power-gated circuits

   Abhishek Sinkar and Nam Sung Kim

   2010 11th International Symposium on Quality Electronic Design (ISQED) , 2010

   Abs DOI

   Power-gating (PG) techniques have been widely used in modern digital ICs to reduce their standby leakage power during idle periods. Meanwhile, virtual supply voltage (VVDD) of a power-gated IC is a function of strength of a PG device and total current flowing through it. Thus, the VVDD level becomes susceptible to 1) negative bias temperature instability (NBTI) degradation that weakens the PG device over time and 2) temporal temperature variation that affects active leakage current (thus total current) of the IC. To account for the NBTI degradation, the PG device must be upsized such that it guarantees a minimum VVDD level that prevents any timing failure over chip lifetime. Moreover, the PG device is also sized for the worst-case voltage drop partly resulted by a large amount of active leakage current at high temperature. However, increasing the size of the PG device to consider both effects leads to higher VVDD (thus active leakage power) than necessary at low temperature and/or in early chip lifetime. To minimize active leakage power increase due to these effects, we propose two techniques that adjust strength of a PG device based on its usage and IC’s temperature at runtime. Both techniques are applied to an experimental setup modeling total current consumption of an IC in 32nm technology and their efficacy is demonstrated in the presence of within-die spatial process and temperature variations. On average of 100 die samples, they can reduce active leakage power by up to 10% in early chip lifetime.

6. PATENT

   Memory cell bit valve loss detection and restoration

   Nam Sung Kim, Muhammad Kheliah, Yibin Ye, Dinesh Somasekhar, and Vivek De

   2010

   Abs DOI

7. Workload-adaptive process tuning strategy for power-efficient multi-core processors

   Jungseob Lee, Chi-Chao Wang, Hamid Ghasemil, Lloyd Bircher, Yu Cao, and Nam Sung Kim

   2010

   Abs DOI

8. PATENT

   Sleep transistor array apparatus and method with leakage control circuitry

   Nam Sung Kim and Vivek De

   2010

   Abs DOI

   FIG. 1 is a schematic diagram of a chip with a virtual Supply in accordance with some embodiments. FIG. 2 is a routine for identifying a suitable combination of enabled sleep transistors in a programmable sleep transistor array in accordance with some embodiments. FIG. 3 is a schematic diagram of a chip with a virtual low Supply reference in accordance with Some embodiments. FIG. 4 is a schematic diagram of a processor with a virtual Supply reference in accordance with Some other embodi mentS.

9. The compatibility analysis of thread migration and DVFS in multi-core processor

   Dongkeun Oh, Charlie Chung Ping Chen, NamSung Kim, and Yu Hen Hu

   2010 11th International Symposium on Quality Electronic Design (ISQED) , 2010

   Abs DOI

10. Analyzing impact of multiple ABB and AVS domains on throughput of power and thermal-constrained multi-core processors

    Jungseob Lee, Shi-Ting Zhou, and Nam Sung Kim

    2010 15th Asia and South Pacific Design Automation Conference (ASP-DAC) , 2010

    Abs DOI

    Recently, semiconductor industries have integrated more cores in a single die, which substantially improves the throughput of the processors running highly-parallel applications. However, many existing applications do not have high enough parallelism to exploit multiple cores in a die, slowing the transition to many-core processors with smaller and more cores that benefit future applications with high parallelism. In this paper, we analyze the impact of multiple adaptive voltage scaling (AVS) and adaptive body biasing (ABB) domains on the throughput of power and thermal-constrained multi-core processors when they are combined with per-core power-gating (PCPG). Both AVS and ABB can be effectively used to either increase frequency (thus throughput) or decrease power consumption of the processors. Meanwhile, PCPG can provide extra power and thermal headroom when application’s parallelism is limited. First, we analyze the throughput impact of applying AVS, ABB, and PCPG for power and thermal constrained multi-core processors. Second, we investigate the impact of multiple AVS and ABB domains on the throughput, and recommend the most cost-effective number of domains for AVS and ABB in 16 and 8-core processors. Our analysis using the 32nm predictive technology model considering within-die variations suggests that the most cost-effective number of domains for AVS and/or ABB should be one for each when they are combined with PCPG in both 16 and 8-core processors. Since within-die core-to-core variations provide many choices in terms of core frequency and power consumption for limited-parallelism applications, one AVS or ABB domain can leads to the throughput improvement by 1.77∼2.49x; more than one AVS and/or ABB domains only improve the throughput marginally.

2009

1. Optimizing throughput of power-and thermal-constrained multicore processors using DVFS and per-core power-gating

   Jungseob Lee and Nam Sung Kim

   Design Automation Conference, 2009. DAC’09. 46th ACM/IEEE , 2009

   Abs DOI

2. Process Temperature and Supply-Noise Tolerant 45< formula formulatype=

   Muhammad Khellah, Nam Sung Kim, Yibin Ye, Dinesh Somasekhar, Tanay Karnik, Nitin Borkar, Gunjan Pandya, Fatih Hamzaoglu, Tom Coan, Yih Wang, Kevin Zhang, Clair Webb, and Vivek De

   Solid-State Circuits, IEEE Journal of , 2009

   Abs DOI

3. Optimizing total power of many-core processors considering voltage scaling limit and process variations

   Jungseob Lee and Nam Sung Kim

   2009

   Abs DOI

4. PATENT

   Memory having bit line with resistor (s) between memory cells

   Muhammad M Khellah, Dinesh Somasekhar, Yibin Ye, Nam Sung Kim, and Vivek De

   2009

   Abs DOI

5. Frequency and yield optimization using power gates in power-constrained designs

   Nam Sung Kim, Jun Seomun, Abhishek Sinkar, Jungseob Lee, Tae Hee Han, Ken Choi, and Youngsoo Shin

   2009

   Abs DOI

   Manufactured dies exhibit a large spread of maximum frequency and leakage power due to process variations, which have been increasing with technology scaling. Reducing the spread is very important for maximizing the frequency and the yield of power-constrained designs, because otherwise many dies that do not satisfy frequency or power constraints would be discarded. In this paper, we propose two optimization methods to improve the maximum operating frequency and the yield using power gates that already exist in many power-constrained designs. In the first method, we consider the designs of multiple cores, where each of them can be independently power-gated. When each core shows different frequencies due to within-die variations, the strength of a power gate in each core is adjusted to make their maximum operating frequencies even. This allows faster cores to consume less active leakage power, reducing the total power consumption well below a power constraint in a globally-clocked design. We subsequently increase global supply voltage for higher overall frequency until the power constraint is satisfied. In our experiments assuming multicore processors with 2–16 cores, the maximum operating frequency was improved by 4-23%. In the second method, we take leaky-but-fast dies (which otherwise would be discarded) and adjust the strength of the power gates such that they can operate in an acceptable power and frequency region. The problem is extended to designs employing a frequency binning strategy, where we have an additional objective of maximizing the number of dies for higher frequency bins. In our experiments with ISCAS benchmark circuits, most discarded fast-but leaky dies were recovered using the second method.

6. SRAM dynamic stability estimation using MPFP and its applications

   DiaaEldin Khalil, Muhammad Khellah, Nam-Sung Kim, Yehea Ismail, Tanay Karnik, and Vivek De

   Microelectronics journal , 2009

   Abs DOI

7. Analyzing potential power reduction with adaptive voltage positioning optimized for multicore processors

   Abhishek Sinkar and Nam Sung Kim

   2009

   Abs DOI

8. PATENT

   Method and apparatus improving performance of a digital memory array device

   Min Huang, Chris Wilkerson, Nam Sung Kim, and Moinuddin K Qureshi

   2009
9. Statistical static timing analysis considering leakage variability in power gated designs

   Michael J Anderson, Azadeh Davoodi, Jungseob Lee, Abhishek Sinkar, and Nam Sung Kim

   2009

   Abs DOI

10. PATENT

    Sense amplifier method and arrangement

    Dinesh Somasekhar, Muhammad M Khellah, Yibin Ye, Nam Sung Kim, and K De Vivek

    2009

    Abs DOI

2008

1. Energy-efficient and metastability-immune resilient circuits for dynamic variation tolerance

   Keith A Bowman, James W Tschanz, Nam Sung Kim, Janice C Lee, Chris B Wilkerson, Shih-Lien L Lu, Tanay Karnik, and Vivek K De

   IEEE Journal of Solid-State Circuits , 2008

   Abs DOI

   A 65 nm resilient circuit test-chip is implemented with timing-error detection and recovery circuits to eliminate the clock frequency guardband from dynamic supply voltage (VCC) and temperature variations as well as to exploit path-activation probabilities for maximizing throughput. Two error-detection sequential (EDS) circuits are introduced to preserve the timing-error detection capability of previous EDS designs while lowering clock energy and removing datapath metastability. One EDS circuit is a dynamic transition detector with a time-borrowing datapath latch (TDTB). The other EDS circuit is a double-sampling static design with a time-borrowing datapath latch (DSTB). In comparison to previous EDS designs, TDTB and DSTB redirect the highly complex metastability problem from both the datapath and error path to only the error path, enabling a drastic simplification in managing metastability. From a survey of various EDS circuit options, TDTB represents the lowest clock energy EDS circuit known; DSTB represents the lowest clock energy static-EDS circuit with SER protection known. Error-recovery circuits are introduced to replay failing instructions at lower clock frequency to guarantee correct functionality. Relative to conventional circuits, test-chip measurements demonstrate that resilient circuits enable either 25%-32% throughput gain at equal VCC or at least 17% VCC reduction at equal throughput, corresponding to 31%-37% total power reduction.

2. VLSI

   Accurate estimation of SRAM dynamic stability

   DiaaEldin Khalil, Muhammad Khellah, Nam-Sung Kim, Yehea Ismail, Tanay Karnik, and Vivek K De

   IEEE transactions on very large scale integration (VLSI) systems , 2008

   Abs DOI

3. Energy-efficient and metastability-immune timing-error detection and instruction-replay-based recovery circuits for dynamic-variation tolerance

   Keith A Bowman, James W Tschanz, Nam Sung Kim, Janice C Lee, Chris B Wilkerson, Shih-Lien L Lu, Tanay Karnik, and Vivek K De

   2008 IEEE International Solid-State Circuits Conference-Digest of Technical Papers , 2008

   Abs DOI

   Microprocessor clock frequency (FCLK) is traditionally determined based on maximum supply voltage (Vcc) droop and temperature specifications. Since typical usage patterns usually run at nominal Vcc and temperature, these infrequent dynamic variations severely limit FCLK. The concept of timing-error detection and correction in previous work [1,2] is extended and implemented in a test-chip in 65nm CMOS [3] to explore the effectiveness of resilient circuits in eliminating Vcc and temperature FCLK guardbands as well as exploiting path-activation probabilities to maximize throughput (TP).

4. On-chip cache device scaling limits and effective fault repair techniques in future nanoscale technology

   David Roberts, Nam Sung Kim, and Trevor Mudge

   Microprocessors and Microsystems , 2008

   Abs DOI

5. PATENT

   Memory with dynamically adjustable supply

   Fatih Hamzaoglu, Kevin Zhang, Nam Sung Kim, Muhammad M Khellah, Dinesh Somasekhar, Yibin Ye, Vivek K De, and Bo Zheng

   2008

   Abs DOI

6. Energy-efficient and metastability-immune timing-error detection and recovery circuits for dynamic variation tolerance

   Keith A Bowman, James W Tschanz, Nam Sung Kim, Janice C Lee, Chris B Wilkerson, Shih-Lien L Lu, Tanay Karnik, and Vivek K De

   2008 IEEE international conference on integrated circuit design and technology and tutorial , 2008

   Abs DOI

   Timing-error detection and recovery circuits are implemented in a 65 nm resilient circuit test-chip to eliminate the clock frequency guardband from dynamic supply voltage (VCC) and temperature variations as well as to exploit path-activation probabilities for maximizing throughput. Two error-detection sequential (EDS) circuits are introduced to preserve the timing-error detection capability of previous EDS designs while lowering clock energy and removing datapath metastability. Error-recovery circuits replay failing instructions at lower clock frequency to guarantee correct functionality. Relative to conventional circuits, silicon measurements indicate that resilient circuits enable either 25 to 32% throughput gain at equal VCC or at least 17% VCC reduction at equal throughput, resulting in 31 to 37% total power reduction.

7. PATENT

   Address hashing to help distribute accesses across portions of destructive read cache memory

   Nam Sung Kim, Muhammad M Khellah, and Vivek De

   2008

   Abs DOI

8. PATENT

   Memory driver circuits with embedded level shifters

   Muhammad M Khellah, Dinesh Somasekhar, Yibin Ye, Nam Sung Kim, and Vivek K De

   2008

   Abs DOI

9. PATENT

   Analog phase control circuit and method

   Nam Sung Kim and Vivek De

   2008

   Abs DOI

   Correspondence Address: 57 ABSTRACT INTEL/BLAKELY(57) 1279 OAKMEAD PARKWAY In some embodiments, an array of sleep transistors is pro SUNNYVALE, CA 94085-4040 vided, wherein a combination of said transistors may be enabled during an active mode to reduce leakage depending (21) Appl. No.: 11/609,823 on the leakage characteristics of a chip or associated chip.

2007

1. Adaptive frequency and biasing techniques for tolerance to dynamic temperature-voltage variations and aging

   James Tschanz, Nam Sung Kim, Saurabh Dighe, Jason Howard, Gregory Ruhl, Sriram Vangal, Siva Narendra, Yatin Hoskote, Howard Wilson, Carol Lam, Matthew Shuman, Carlos Tokunaga, Dinesh Somasekhar, Stephen Tang, David Finan, Tanay Karnik, Nitin Borkar, Nasser Kurd, and Vivek De

   2007 IEEE International Solid-State Circuits Conference. Digest of Technical Papers , 2007

   Abs DOI

   Temperature, voltage, and current sensors monitor the operation of a TCP/IP offload accelerator engine fabricated in 90nm CMOS, and a control unit dynamically changes frequency, voltage, and body bias for optimum performance and energy efficiency. Fast response to droops and temperature changes is enabled by a multi-PLL clocking unit and on-chip body bias. Adaptive techniques are also used to compensate performance degradation due to device aging, reducing the aging guardband.

2. CAD

   Yield-driven near-threshold SRAM design

   Gregory Chen, Dennis Sylvester, David Blaauw, Trevor Mudge, and Nam Sung Kim

   Proceedings of the 2007 IEEE/ACM international conference on Computer-aided design , 2007

   Abs DOI

3. PATENT

   Memory cell having p-type pass device

   Muhammad M Khellah, Dinesh Somasekhar, Nam Sung Kim, Yibin Ye, Vivek K De, Kevin Zhang, and Bo Zheng

   2007

   Abs DOI

   For one disclosed embodiment, an apparatus comprises a first p-type device coupled between a cell Voltage node and a storage node, an n-type device coupled between the storage node and a reference Voltage node, and a second p-type device to couple the storage node to a bit line in response to a signal on a select line. At least one side of diffusion regions in a substrate to form both the first p-type device and the second p-type device are substantially aligned. Other embodiments are also disclosed.

4. PATENT

   Reducing aging effect on memory

   Nam Sung Kim, Shih-Lien L Lu, Chris Wilkerson, and Edward Grochowski

   2007

   Abs DOI

   Methods and apparatus to reduce aging effect on memory are described. In one embodiment, a modified version of data is stored in a portion of a storage unit during a first time period.

5. SRAM dynamic stability estimation using MPFP

   DiaaEldin Khalil, Muhammad Khellah, Nam-Sung Kim, Yehea Ismail, Tanay Karnik, and Vivek De

   2007 Internatonal Conference on Microelectronics , 2007

   Abs DOI

2006

1. Wordline & bitline pulsing schemes for improving SRAM cell stability in low-Vcc 65nm CMOS designs

   Muhammad Khellah, Yibin Ye, N Kim, Dinesh Somasekhar, Gunjan Pandya, Ali Farhang, Kevin Zhang, Clair Webb, and Vivek De

   2006 Symposium on VLSI Circuits, 2006. Digest of Technical Papers. , 2006

   Abs DOI

   Pulsed wordline (PWL) & pulsed bitline (PBL) techniques to improve SRAM cell stabilities in single-Vcc microprocessor designs are evaluated in 65nm CMOS. At 0.7V Vcc, PWL improves cell failure rate by 15times while incurring <1% area overhead. Both PBL & PWL with read-modify-write (PWL-RMW) provide the best improvements (26times) in cell stability, with significant area overheads (4-8%)

2. A 256-Kb Dual- SRAM Building Block in 65-nm CMOS Process With Actively Clamped Sleep Transistor

   Muhammad Khellah, Dinesh Somasekhar, Yibin Ye, Nam Sung Kim, Jason Howard, Greg Ruhl, Murad Sunna, James Tschanz, Nitin Borkar, Fatih Hamzaoglu, Gunjan Pandya, Ali Farhang, Kevin Zhang, and Vivek De

   IEEE Journal of Solid-State Circuits , 2006

   Abs DOI

3. PATENT

   Data processor memory circuit

   Krisztian Flautner, David T Blaauw, Trevor N Mudge, Nam S Kim, and Steven M Martin

   2006

   Abs DOI

   2003-07-17 Assigned to MICHIGAN, UNIVERSITY OF, ARM LIMITED reassignment MICHIGAN, UNIVERSITY OF ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: BLAAUW, DAVID T., MARTIN, STEVEN M., KIM, NAM S., MUDGE, TREVOR N., FLAUTNER, KRISZTIAN

4. A 4.2 GHz 0.3 mm2 256kb Dual-V/sub cc/SRAM Building Block in 65nm CMOS

   M. Khellah, Nam Sung Kim, J. Howard, G. Ruhl, Yibin Ye, J. Tschanz, D. Somasekhar, N. Borkar, F. Hamzaoglu, G. Pandya, A. Farhang, K. Zhang, and V. De

   2006 IEEE International Solid State Circuits Conference-Digest of Technical Papers , 2006

   Abs DOI

   An SRAM macro, implemented in a 65nm CMOS process, uses a dual supply to maximize density while enabling the use of low voltage for the processor core. Measurements of a 256kb block show 4.2GHz operation using 29mW from 1.2V at 85degC, with core logic operating down to 0.7V and a sleep biasing scheme that autonomously compensates for PVT and aging

5. Session 8: leakage power analysis and optimization

   Nam Sung Kim, Naehyuck Chang, and Sanu Mathew

   Annual ACM IEEE Design Automation Conference: Proceedings of the 43 rd annual conference on Design automation , 2006

   Abs DOI

2005

1. VLSI

   Quantitative analysis and optimization techniques for on-chip cache leakage power

   Nam Sung Kim, David Blaauw, and Trevor Mudge

   Very Large Scale Integration (VLSI) Systems, IEEE Transactions on , 2005

   Abs DOI

2. CAD

   Total power-optimal pipelining and parallel processing under process variations in nanometer technology

   Nam Sung Kim, Taeho Kgil, Keith Bowman, Vivek De, and Trevor Mudge

   ICCAD-2005. IEEE/ACM International Conference on Computer-Aided Design, 2005. , 2005

   Abs DOI

3. Total leakage optimization strategies for multi-level caches

   Robert Bai, Nam-Sung Kim, Dennis Sylvester, and Trevor Mudge

   2005

   Abs DOI

4. Power-performance trade-offs in nanometer-scale multi-level caches considering total leakage

   Robert Bai, Nam-Sung Kim, Tae Ho Kgil, Dennis Sylvester, and Trevor Mudge

   Design, Automation and Test in Europe , 2005

   Abs DOI

5. VLSI

   Erratum: Circuit and microarchitectural techniques for reducing cache leakage power (IEEE Transactions on Very Large Scale Integration (VLSI) Systems (Feb. 2004) 12: 2 (167-184))

   Nam Sung Kim, K Flautner, D Blaauw, and T Mudge

   IEEE Transactions on Very Large Scale Integration (VLSI) Systems , 2005

   Abs DOI

2004

1. Razor: circuit-level correction of timing errors for low-power operation

   Dan Ernst, Shidhartha Das, Seokwoo Lee, David Blaauw, Todd Austin, Trevor Mudge, Nam Sung Kim, and Krisztián Flautner

   IEEE Micro , 2004

   Abs DOI

   Dynamic voltage scaling is one of the more effective and widely used methods for power-aware computing. We present a DVS approach that uses dynamic detection and correction of circuit timing errors to tune processor supply voltage and eliminate the need for voltage margins.

2. VLSI

   Circuit and microarchitectural techniques for reducing cache leakage power

   Nam Sung Kim, Krisztian Flautner, David Blaauw, and Trevor Mudge

   IEEE Transactions on Very Large Scale Integration (VLSI) Systems , 2004

   Abs DOI

   On-chip caches represent a sizable fraction of the total power consumption of microprocessors. As feature sizes shrink, the dominant component of this power consumption will be leakage. However, during a fixed period of time, the activity in a data cache is only centered on a small subset of the lines. This behavior can be exploited to cut the leakage power of large data caches by putting the cold cache lines into a state preserving, low-power drowsey mode. In this paper, we investigate policies and circuit techniques for implementing drowsy data caches. We show that with simple microarchitectural techniques, about 80%-90% of the data cache lines can be maintained in a drowsy state without affecting performance by more than 0.6%, even though moving lines into and out of a drowsy state incurs a slight performance loss. According to our projections, in a 70-nm complementary metal-oxide-semiconductor process, drowsy data caches will be able to reduce the total leakage energy consumed in the caches by 60%-75%. In addition, we extend the drowsy cache concept to reduce leakage power of instruction caches without significant impact on execution time. Our results show that data and instruction caches require different control strategies for efficient execution. In order to enable drowsy instruction caches, we propose a technique called cache subbank prediction, which is used to selectively wake up only the necessary parts of the instruction cache, while allowing most of the cache to stay in a low-leakage drowsy mode. This prediction technique reduces the negative performance impact by 78% compared with the no-prediction policy. Our technique works well even with small predictor sizes and enables a 75% reduction of leakage energy in a 32-kB instruction cache.

3. Single-vDD and single-vT super-drowsy techniques for low-leakage high-performance instruction caches

   Nam Sung Kim, Krisztián Flautner, David Blaauw, and Trevor Mudge

   2004

   Abs DOI

4. Microarchitectural power modeling techniques for deep sub-micron microprocessors

   Nam Sung Kim, Taeho Kgil, Valeria Bertacco, Todd Austin, and Trevor Mudge

   2004

   Abs DOI

5. Shidhartha Das

   Dan Ernst and Nam Sung Kim

   Sanjay Pant, Rajeev Rao, Toan Pham , 2004

   Abs DOI

6. Circuit and microarchitectural techniques for processor on-chip cache leakage power reduction

   Nam Sung Kim

   2004

   Abs DOI

   On-chip L1 and L2 caches dissipate a sizeable fraction of the total power of processors. As feature sizes shrink and threshold voltages decrease, sub-threshold leakage is becoming the dominant component of this power consumption. In this study we examine various circuit and microarchitectural techniques for low-leakage on-chip caches, designed with nano-scale complementary-metal-oxide semiconductor (CMOS) technologies.

2003

1. MICRO

   Razor: A low-power pipeline based on circuit-level timing speculation

   Dan Ernst, Nam Sung Kim, Shidhartha Das, Sanjay Pant, Rajeev Rao, Toan Pham, Conrad Ziesler, David Blaauw, Todd Austin, Krisztian Flautner, and Trevor Mudge

   Proceedings. 36th Annual IEEE/ACM International Symposium on Microarchitecture, 2003. MICRO-36. , 2003

   Abs DOI

   With increasing clock frequencies and silicon integration, power aware computing has become a critical concern in the design of embedded processors and systems-on-chip. One of the more effective and widely used methods for power-aware computing is dynamic voltage scaling (DVS). In order to obtain the maximum power savings from DVS, it is essential to scale the supply voltage as low as possible while ensuring correct operation of the processor. The critical voltage is chosen such that under a worst-case scenario of process and environmental variations, the processor always operates correctly. However, this approach leads to a very conservative supply voltage since such a worst-case combination of different variabilities is very rare. In this paper, we propose a new approach to DVS, called Razor, based on dynamic detection and correction of circuit timing errors. The key idea of Razor is to tune the supply voltage by monitoring the error rate during circuit operation, thereby eliminating the need for voltage margins and exploiting the data dependence of circuit delay. A Razor flip-flop is introduced that double-samples pipeline stage values, once with a fast clock and again with a time-borrowing delayed clock. A metastability-tolerant comparator then validates latch values sampled with the fast clock. In the event of timing error, a modified pipeline mispeculation recovery mechanism restores correct program state. A prototype Razor pipeline was designed in a 0.18 /spl mu/m technology and was analyzed. Razor energy overhead during normal operation is limited to 3.1%. Analyses of a full-custom multiplier and a SPICE-level Kogge-Stone adder model reveal that substantial energy savings are possible for these devices (up to 64.2%) with little impact on performance due to error recovery (less than 3%).

2. Leakage current: Moore’s law meets static power

   Nam Sung Kim, Todd Austin, David Blaauw, Trevor Mudge, Jie S Hu, Mary Jane Irwin, Mahmut Kandemir, and Vijaykrishnan Narayanan

   computer , 2003

   Abs DOI

   Off-state leakage is static power, current that leaks through transistors even when they are turned off. The other source of power dissipation in today’s microprocessors, dynamic power, arises from the repeated capacitance charge and discharge on the output of the hundreds of millions of gates in today’s chips. Until recently, only dynamic power has been a significant source of power consumption, and Moore’s law helped control it. However, power consumption has now become a primary microprocessor design constraint; one that researchers in both industry and academia will struggle to overcome in the next few years. Microprocessor design has traditionally focused on dynamic power consumption as a limiting factor in system integration. As feature sizes shrink below 0.1 micron, static power is posing new low-power design challenges.

3. Reducing register ports using delayed write-back queues and operand pre-fetch

   Nam Sung Kim and Trevor Mudge

   2003

   Abs DOI

4. A 2.3 Gb/s fully integrated and synthesizable AES Rijndael core

   Nam Sung Kim, Trevor Mudge, and Richard Brown

   Proceedings of the IEEE 2003 Custom Integrated Circuits Conference, 2003. , 2003

   Abs DOI

   The growth of the Internet as a vehicle for secure communication and electronic commerce has brought cryptographic processing performance to the forefront of high throughput system design. This trend will gain further momentum with the widespread adoption of secure protocols such as secure IP (IPSEC) and virtual private networks (VPNs). In this paper, we present a fully integrated and synthesizable cipher core supporting the advanced encryption standard - Rijndael. We designed and fabricated the fully integrated core key scheduler, encipher, and decipher using TSMC 0.18 /spl mu/m technology. The core operating frequency is 465 MHz and throughput is 2.3 Gb/s.

5. The microarchitecture of a low power register file

   Nam Sung Kim and Trevor Mudge

   2003

   Abs DOI

6. Shidhartha Das Sanjay Pant Toan Pham Rajeev Rao Conrad Ziesler David Blaauw Todd Austin Trevor Mudge and Krisztián Flautner. Razor: A Low-Power Pipeline Based on Circuit-Level Timing Speculation

   Dan Ernst and Nam Sung Kim

   Proc. 36th Intl. Symp. Microarch , 2003

   Abs DOI

7. Power analyzer for pocket computing (papc)

   T Mudge, NS Kim, J Ringenberg, and T Kgil

   University of Michigan, Tech. Rep. , 2003
8. Reducing Register Ports Using Delayed Write-Back

   Nam Sung Kim and Trevor Mudge

   Conference Proceedings , 2003

   Abs DOI

   In high-performance wide-issue microprocessors the access time, energy and area of the register file are often critical to overall performance. This is because these parameters grow superlinearly as read and write ports are added to support wide-issue. This paper presents techniques to reduce the number of ports of a register file intended for a wide-issue microprocessor without noticeably impacting its IPC. Our results show that it is possible to replace the16 read/8 write port file of an eight-issue processor with an 8 read/8 write port file so that the impact on IPC is insignificant.

9. Leakage current: Moore’s law meets static power. computer 36 12 (2003) 68–75

   Nam Sung Kim, Todd Austin, David Baauw, Trevor Mudge, Krisztián Flautner, Jie S Hu, Mary Jane Irwin, Mahmut Kandemir, and Vijaykrishnan Narayanan

   Google Scholar Google Scholar Digital Library Digital Library , 2003

   Abs DOI

2002

1. Drowsy caches: simple techniques for reducing leakage power

   Krisztián Flautner, Nam Sung Kim, Steve Martin, David Blaauw, and Trevor Mudge

   ACM SIGARCH Computer architecture news , 2002

   Abs DOI

   On-chip caches represent a sizable fraction of the total power consumption of microprocessors. Although large caches can significantly improve performance, they have the potential to increase power consumption. As feature sizes shrink the dominant component of this power loss will be leakage. However, during a fixed period of time the activity in a cache is only centered on a small subset of the lines. This behavior can be exploited to cut the leakage power of large caches by putting the cold cache lines into a state preserving, low-power drowsy mode. Moving lines into and out of drowsy state incurs a slight performance loss. In this paper we investigate policies and circuit techniques for implementing drowsy caches. We show that with simple architectural techniques, about 80%-90% of the cache lines can be maintained in a drowsy state without affecting performance by more than 1%. According to our projections, in a 0.07 um CMOS process, drowsy caches will be able to reduce the total energy (static and dynamic) consumed in the caches by 50%-75%. We also argue that the use of drowsy caches can simplify the design and control of low-leakage caches, and avoid the need to completely turn off selected cache lines and lose their state.

2. MICRO

   Drowsy instruction caches. leakage power reduction using dynamic voltage scaling and cache sub-bank prediction

   Nam Sung Kim, Krisztian Flautner, David Blaauw, and Trevor Mudge

   35th Annual IEEE/ACM International Symposium on Microarchitecture, 2002.(MICRO-35). Proceedings. , 2002

   Abs DOI

3. Challenges for architectural level power modeling

   Nam Sung Kim, Todd Austin, Trevor Mudge, and Dirk Grunwald

   Power aware computing , 2002

   Abs DOI

   The power aware design of microprocessors is becoming increasingly important. Power aware design can best be achieved by considering the impact of architectural choices on power early in the design process. A natural solution is to build a power estimator into the cycle simulators that are used to gauge the effect of architectural choices on performance. Cycle simulators intentionally omit considerable implementation detail in order to be efficient. The challenge is to select the details that must be put back in if the simulator is required to also produce meaningful power figures. In this paper we propose how to augment a cycle simulator to produce these power figures.

4. Low-energy data cache using sign compression and cache line bisection

   Nam Sung Kim, Todd Austin, and Trevor Mudge

   Proceedings of the 2nd Annual Workshop on Memory Performance Issues (WMPI’02) , 2002

   Abs DOI

   Cache accesses consume a significant portion of total energy dissipation in modern microprocessors. In this paper, we introduce a new technique for data cache energy reduction, which exploits the prevalence of small values and the inefficiency of using a full word for their storage. Simulation results show that we can reduce total data cache energy by around 23% for the SPEC benchmarks without significant performance degradation. We also propose a modified data compression cache architecture utilizing empty cache space that is not used after data compression. This proposed architecture improves the cache miss rate, which results in improved performance and reduced energy dissipation when accessing the next level of the memory hierarchy. We show that this scheme has an energy saving of about 19%.

5. MICRO

   Leakage Power Reduction using Dynamic Voltage Scaling and Cahe Sub-bank Prediction

   Nam Sung Kim, Krisztián Flautner, David Blaauw, and Trevor Mudge

   35th Annual IEEE/ACM International Symposium on Microarchitecture, 2002.(MICRO-35). Proceedings. , 2002

   Abs DOI

2001

1. Power Aware Computing chapter Challenges for Architectural Level Power Modeling

   Nam Sung Kim, Todd Austin, Trevor Mudge, and Dirk Grunwald

   2001

   Abs DOI

2. VLSI Implementation of the Symmetric Key Block Cipher with the Advanced Encryption Standard-Rijndael

   Nam Sung Kim, Richard B Brown, and Trevor Mudge

   University of Michigan , 2001

   Abs DOI

   The National Institute of Standard and Technology (NIST) has initiated a process to develop a Federal Information Processing Standard (FIPS) for the Advanced Encryption Standard (AES), specifying an Advanced Encryption Algorithm to replace the Data Encryption Standard (DES) that expired in 1998. NIST has solicited candidate algorithms for inclusion in AES, resulting in fifteen official candidate algorithms of which Rijndael was chosen as the Advanced Encryption Standard. In this project, we will propose an efficient VLSI architecture for Advanced Encryption Standard-Rijndael algorithm and implement it with an ASIC design methodology in order to provide a high-speed and cost-effective cryptographic hardware and simulation results show that the throughput of the implementation is 1.024 Gigabits/second.

3. VLSI Implementation of Binaural Spatializer using FIR Head-Related Transfer Function (HRTF)

   Nam Sung Kim, Jiyoun Kim, Min-Gyu Cho, and Tae-Young Choi

   2001

   Abs

   Many 3-D sound generation algorithms have been developed for multimedia applications. In this paper, we propose a VLSI implementation of a Binaural Spatializer, using the Head-Related Transfer Function (HRTF), one of the widely used methods for performing sound localization. The main feature of the proposed implementation is the use of timedomain convolution to perform HRTF to increase the computational efficiency of the HRTF.

1998

1. Virtual chip: making functional models work on real target systems

   Namseung Kim, Hoon Choi, Seungjong Lee, Seungwang Lee, In-Cheolo Park, and Chong-Min Kyung

   Proceedings of the 35th annual Design Automation Conference , 1998

   Abs DOI

   As design complexity increases, functional verification becomes a crucial issue to ensure design correctness at an early design stage. Traditional methods for verifying functional designs are based on the HDL simulation, which is becoming the bottleneck of the design cycle because of the increasing design complexity. The accurate verification ability at the architectural level through a large set of the test programs and real world applications is a foundation for the next design step. In this paper, we describe how to verify a functional model on a real target system. The proposed methodology called virtual chip makes it possible not only to check the functional correctness on real systems, but also to explore design space by measuring the performance effectiveness of various architecture parameters under real applications. Experimental results show that functional models can be verified on real systems using complicated application programs. The proposed functional verification method is faster than HDL simulation and even comparable to emulation.