Tech news in 3 minutes
Micron Evolving Memory Architectures for AI at Hot Chips 2026
Samsung High Bandwidth Memory (HBM) at Hot Chips 2026: Samsung is presenting "Evolving HBM Base Die," detailing plans to transform the base die from a passive interposer into a more capable, SoC-like component for next-generation AI chips and data center memory. The talk covers how HBM splits work between stacked DRAM core dies (C-die) and the base die (B-die), which carries PHY and through-silicon vias (TSVs) for communication with the compute die. Samsung traces HBM bandwidth and capacity growth from ~1 GB/1 TB/s in original HBM to over 60 GB/6 TB/s in HBM5, noting that rising bandwidth forces fundamental base die changes. Key scaling limits include TSV pitch, I/O count, and PHY speed. Samsung shifts the B-die from older logic to 4nm starting with HBM4, enabling advanced logic integration for power reduction and active area minimization. The company introduces custom HBM (cHBM) using advanced logic to offload XPU functions onto the B-die. Phase 1 reclaims XPU area by replacing traditional HBM PHY with a die-to-die interface, shrinking footprint and improving energy efficiency, while a Heat Path Block reduces peak temperature by over 35% as power density climbs to 2.0 W/mm². Memory controller offloading and SRAM-based cell repair further enhance reliability. Phase 2 adds SoC-level RAS with thermal, voltage, and aging sensors, plus on-chip self-test. It also enables external memory expansion via the B-die shoreline and offloads processing elements to cut die-to-die bandwidth. Phase 3 introduces zHBM, a true 3D vertical integration of XPU and C-die stack, eliminating the 2.5D interposer for ultra-low power consumption. Samsung targets these innovations to address AI model capacity demands, with context windows expanding ~30x per year, making high-capacity, high-bandwidth memory critical for next-gen AI inference.