OpenAI's upcoming Jalapeño chip looks like it'll be an inference beast

OpenAI is developing the Jalapeño chip, designed for high-scale inference. Featuring 128 chips, 1.7 exaFLOPS, and 27 TB of HBM, it aims to outperform Blackwell and Rubin. According to Semianalysis’s InferenceX benchmark, Jalapeño delivers higher throughput per kilowatt and more tokens per user than current state-of-the-art technology. These specifications position the hardware as an inference beast, providing Sam Altman and his team a significant competitive advantage in processing power and energy efficiency for AI workloads.