Samsung recently demonstrated its latest advances in processing-in-memory (PIM) technology at Hot Chips 33, including the first successful integration of its PIM-enabled High Bandwidth Memory (HBM-PIM) into a commercialised accelerator system.

The company also broadened PIM applications to include DRAM modules and mobile memory, accelerating the move toward memory and logic convergence.

Samsung Brings In-memory Processing Power to Wider Range of Applications

Samsung released the industry’s first HBM-PIM (Aquabolt-XL) in February, which integrates the AI processing capability into Samsung’s HBM2 Aquabolt to improve high-speed data processing in supercomputers and AI applications.

Since then, the HBM-PIM has been put to the test in the Xilinx Virtex Ultrascale+ (Alveo) AI accelerator, delivering a nearly 2.5X performance boost and a 60% reduction in energy usage.

The Acceleration DIMM (AXDIMM) moves computation to the DRAM module itself, reducing huge data transfers between the CPU and DRAM and boosting AI accelerator systems’ energy efficiency.

The AXDIMM can do parallel processing of many memory ranks (sets of DRAM chips) instead of accessing only one rank at a time thanks to an AI engine incorporated into the buffer chip. This dramatically improves system performance and efficiency.

The AXDIMM supports drop-in replacement without requiring system modifications because the module can retain its typical DIMM form factor.

The AXDIMM, which is now being tested on client servers, can provide around twice the performance in AI-based recommendation applications and a 40% reduction in system-wide energy utilisation.

Without data centre connectivity, Samsung’s LPDDR5-PIM mobile memory technology can enable standalone AI capabilities.

When utilised in applications like voice recognition, translation, and chatbots, simulation testing has shown that the LPDDR5-PIM may more than double performance while decreasing energy usage by over 60%.

In the first half of 2022, Samsung hopes to expand its AI memory portfolio by collaborating with other industry leaders to complete the standardisation of the PIM platform.

In order to ensure broad application across the memory market, the business will continue to create a healthy PIM ecosystem.