In a significant stride for the future of artificial intelligence (AI) hardware, Samsung Electronics has presented pioneering memory semiconductor technologies at this year’s Hot Chips conference, a prestigious gathering for the semiconductor sector held annually at Stanford University.
The symposium, which took place from the 27th to the 29th of August, drew influential industry players such as SK Hynix, Intel, AMD, and Nvidia, and served as a platform for cutting-edge technological unveilings. Samsung Electronics distinguished itself by debuting its research findings on two revolutionary memory types—High-Bandwidth Memory Processing-In-Memory (HBM-PIM) and Low Power Double Data Rate Processing-In-Memory (LPDDR-PIM).
The South Korean technology behemoth showcased compelling data indicating that implementing HBM-PIM in generative AI algorithms could lead to a twofold increase in accelerator performance and power efficiency. The study utilised AMD’s MI-100 GPU for its benchmarking exercises.
For further empirical validation, Samsung constructed a cluster featuring 96 MI-100 GPUs equipped with HBM-PIM to test the Mixture of Experts (MOE) model. Results were promising; the HBM-PIM technology demonstrated that accelerator performance was twice as effective and three times as energy-efficient as its conventional HBM counterpart.
HBM-PIM has emerged at a crucial juncture in the semiconductor landscape, addressing the current memory bottleneck that is hampering advances in the burgeoning field of AI. The technology integrates Processing-In-Memory (PIM) capabilities with HBM, thus minimising data movement by localising operations previously performed by the CPU within the memory.
On the same occasion, Samsung also unveiled its LPDDR-PIM, a technology designed with edge devices in mind. This innovative memory type marries PIM with Dynamic Random Access Memory (DRAM) and performs calculations directly within edge devices. With a comparatively modest bandwidth of 102.4GB/s, the company underscored that the LPDDR-PIM technology could reduce power consumption by an impressive 72% in relation to traditional DRAM.
While these advancements are significant, their transition from the research stage to commercial availability remains a complex challenge. “The development of memory technology is lagging compared to the rapid evolution in AI accelerator performance,” observed an industry analyst, also noting that the HBM-PIM’s efficacy in large-scale language models is high due to reduced data movement.
However, the path to commercialisation faces headwinds, chiefly due to the intricate technicalities involved in bringing non-memory semiconductor characteristics into a memory process. Moreover, the projected cost of these next-generation memory chips is expected to be considerably higher than existing solutions.
Samsung’s unveilings at the Hot Chips 2023 conference signal a promising direction for memory technologies in AI applications, although commercialisation hurdles and cost considerations loom large on the horizon.
