At Hot Chips 2026, Nvidia presented the Groq 3 LPX architecture built on the LP30 chip obtained through its $20 billion Groq deal in December 2025. Publishing the hardware’s first third-party inference benchmark, Artificial Analysis measured the system at 3,431 output tokens per second on a 100K-context Gemma 4 31B reasoning workload, establishing a production-ready footprint for agentic AI workflows.
The silicon unveiled on stage represents a distinct structural pivot away from traditional high-bandwidth memory setups. Igor Arsovski, Nvidia’s VP of hardware and former chief architect at Groq, detailed how the LP30 chip carries roughly 500MB of on-die SRAM and completely eschews HBM. This layout enables a full LPX rack of 256 chips to pack 128GB of memory, delivering 40 PB/s of aggregate bandwidth against 315 PFLOPS of FP8 compute with 350 ns of chip-to-chip latency.
Inside the SRAM-Only Design and Deterministic Execution
By keeping model weights resident in SRAM rather than streaming them from HBM, the architecture eliminates the memory-access latency that typically dominates single-token decode operations. The design drops traditional caches, branch prediction, and out-of-order execution. Instead, it relies on a fully deterministic pipeline that a compiler schedules precisely at clock-cycle granularity.
This deterministic approach stems directly from the Tensor Streaming Processor concept originally described by Groq founder Jonathan Ross in a 2020 ISCA paper titled Think Fast. At Hot Chips, Arsovski and Raghavan reused that exact framing to explain how cycle-level scheduling alters power delivery.
Because the compiler can predict power draw cycle by cycle, Nvidia uses the data to pre-order current from the rack’s regulators ahead of demand. According to the company’s presentation metrics, this drops voltage droop by more than 60% and overshoot by more than 70% compared to an uncompensated load. Furthermore, per-block scheduling lets the hardware equalize heat distribution rather than throttling to the hottest individual tile.
Arsovski noted that this thermal management yields roughly 10% to 11% additional performance under a fixed thermal limit. Across the rack, chips synchronize to a single virtual clock through a plesiosynchronous network. Each chip functions simultaneously as a processor and router, removing the need for adaptive routing or congestion sensing while compensating for clock drift directly at the links.
Splitting Inference Workloads Across Rubin and LPU Racks
Because a single LP30 carries 500MB of SRAM—roughly 576 times less memory than a Rubin GPU’s 288GB of HBM4—holding a 31-billion-parameter model at FP8 requires approximately 62 LPUs. Larger mixture-of-experts models scale into four figures of chips across multiple racks. Consequently, Nvidia is positioning the LPU as a specialized decode co-processor rather than a general-purpose replacement for its GPUs.
Nvidia outlines three primary ways to divide inference workloads between the Vera Rubin NVL72 platform and the new LPU hardware:
- Disaggregated Prefill and Decode: Rubin GPUs handle the compute-heavy prefill phase and build the KV cache, while LPUs generate output tokens.
- Attention-FFN Disaggregation: Attention mechanisms and their cache remain on GPU HBM, while the LPU executes the feed-forward layers.
- External-Draft Speculative Decoding: A small model running on the LPU proposes tokens that the GPU verifies in parallel, ensuring only draft tokens cross the interconnect link.
An FPGA bridges the synchronous LPU domain with the asynchronous world of host I/O and GPU hand-offs. Nvidia orchestrates this division using its Dynamo runtime alongside an LPU extension to CUDA. Nvidia-measured internal figures estimate three-to-five-times performance gains over Rubin alone when running a two-trillion-parameter workload with a 400K-token cached context.
Deployment has already begun. Nebius announced plans to bring Nvidia Groq 3 LPX to Nebius Token Factory, its production inference platform. Danila Shtan, chief technology officer of Nebius, noted that generation determines real-world system responsiveness, stating that the hardware is built specifically to accelerate that phase of inference.
Wafer-Scale Competition and Regulatory Backdrop
The Hot Chips session also featured competing wafer-scale architectures. Jean-Philippe Fricker, chief system architect at Cerebras, presented the CS4 system. Fricker stated that the CS4 runs up to 30 times faster than GPUs, doubles the token rate of the CS3, and carries 10 times the token capacity. Each CS4 rack houses three wafer-scale engines in a modular platform called Nexus, which separates power, compute, and I/O via pluggable compute backpacks. Cerebras placed its memory bandwidth at 43 PB/s, which Fricker described to the audience as 2,000 times higher than Nvidia’s next-generation Rubin chip. For prefill duties, Cerebras agreed in July to pair AMD Helios GPUs with its wafer-scale engines for decode—mirroring the exact division of labor Nvidia engineered internally with Groq.
Nvidia’s path to the LP30 involved pulling the Rubin CPX, its own GDDR7-based long-context accelerator, to prioritize shipping the LPU this year, as VP Ian Buck detailed at GTC 2026. The $20 billion transaction was structured as a non-exclusive IP license paired with the hiring of Ross, president Sunny Madra, and the majority of Groq’s engineering team, avoiding a formal merger review. In early 2026, Senators Elizabeth Warren and Richard Blumenthal wrote to the FTC and Nvidia arguing that the arrangement acquired Groq in all but name.
