Tech Desk — August 26, 2026 — OpenAI says its first custom inference chip has posted benchmark numbers that put it ahead of Nvidia's flagship systems on the metrics that matter most for serving AI models. The chip, called Jalapeño, delivered up to 1.9 times more AI work per watt at peak throughput and cut end-to-end latency by as much as 3.6 times compared with Nvidia hardware in tests run across three large language models. The results appeared Monday on OpenAI's research blog, timed to the Hot Chips conference where the company is presenting technical details this week.
The announcement lands in the middle of a fight over who pays for AI's compute bill. Nvidia still owns the accelerator market, but the customers writing the biggest checks are tired of waiting. OpenAI built the chip alongside Broadcom, which manufactured it, and says deployment inside its own compute infrastructure starts before year-end. Data Center Dynamics reported that Jalapeño runs at a 700-watt package power rating — well under the 1,400 watts of the Nvidia GB300 systems it was tested against, and under the 1,200 watts of last generation's GB200. In practice, the chip drew no more than 550 watts sustained across the tested workloads.
The stakes reach past one product review. Serving costs now dominate what users pay for AI, and every major lab is hunting for ways to deliver more tokens from the same power envelope. If OpenAI's numbers hold up in production, its cost to serve drops while rivals keep paying Nvidia's margin.
OpenAI is hardly alone in wanting out from under Nvidia's pricing. Google has designed its own TPU accelerators for a decade. Amazon pushes Trainium chips into its cloud. Microsoft built Maia. Meta ships its own MTIA parts. What separates OpenAI now is posture: it is simultaneously the biggest buyer on the market and, as of this week, a credible builder of its own answer.
What the Numbers Show
The company ran its comparisons on InferenceX, a public benchmark from analyst firm SemiAnalysis that measures how fast a system serves an AI request from start to finish. Rather than quoting per-chip figures, OpenAI normalized results using each accelerator's published power rating, arguing that useful work completed per unit of power is the number customers actually feel.
Across GPT-OSS 120B, DeepSeek R1 670B, and Kimi K2.5 1T, Jalapeño delivered 1.5 to 1.9 times more peak throughput per watt than the comparison systems. Latency gaps were wider still. Against the GB300, end-to-end latency on DeepSeek R1 dropped from 5.99 seconds to 1.65 seconds — a 3.6-times improvement. On interactive workloads the spread reached 4.1 times. On Kimi K2.5, the largest public model in the test, the chip held a 3.4-times latency lead while delivering roughly 1.5 times higher performance per watt.
OpenAI says it evaluates performance at a matched user experience, counting only work completed while meeting the latency that interactive agents require. The most eye-catching figure sits in a footnote-style table: at a fixed tokens-per-second-per-user target matching the comparison system's minimum time-between-tokens, Jalapeño delivered more than 50 times the throughput per kilowatt on GPT-OSS and over 100 times on DeepSeek R1. Those are matched-latency comparisons rather than peak-speed ones, which is exactly how inference buyers shop — nobody wants fast batches that leave individual users waiting.
The GPT-OSS results illustrate the pattern. At peak, Jalapeño posted 85,448 mixed tokens per second per kilowatt against 44,960 for the GB200 system it faced, while returning the first full response in 1.03 seconds rather than 1.80 seconds. Time between tokens, the number that decides whether a chatbot feels laggy, came in at 0.69 milliseconds versus 1.87 — meaning roughly twice the tokens stream to each user per second.
The gains come from an unusual place: the network. Jalapeño keeps an entire workload inside one connected system rather than shuttling data between separate machines. Model state, including the KV cache built while a model generates tokens, stays local to the chips working on it. The architecture treats prefill — the compute-heavy phase where a prompt gets processed — and decode, the memory-bandwidth-bound phase where tokens stream out one at a time, as first-class design targets instead of tradeoffs.
That balance matters more than it sounds. Agents chain many model calls together, and delays compound across each step of a task, as OpenAI's post notes. A system tuned only for raw throughput can leave an interactive agent crawling; one tuned only for snappy replies wastes the hardware between requests.
OpenAI describes the result as a balanced and fungible accelerator — hardware that can shift the mix of compute, memory, and networking it applies as workloads change, instead of being optimized for one phase and idling through the next.
Built With Broadcom, Programmed by AI
Broadcom took the design from initial work to tapeout in nine months. OpenAI credits its own models with speeding up that timeline — earlier generations explored implementations and compressed verification loops, and AI even helped optimize the chip's arithmetic circuits. The collaboration deepened a partnership Reuters reported when engineering samples were announced in June, part of a broader push that also includes Broadcom-built networking gear across OpenAI's data centers.
The software story may matter just as much as the silicon. Using Codex with GPT-Astra, OpenAI engineers brought three open-weight models outside the original production plan to high performance within two months. For selected attention and mixture-of-experts blocks in GPT-OSS, AI-written kernel implementations ran 1.5 to 1.8 times faster than the human-expert versions they replaced. Engineers describe work through local tensors, explicit communication, and predictable synchronization, then let models optimize how it gets mapped onto the machine — a workflow aimed squarely at parallel programming, historically the hardest part of exploiting exotic hardware.
Why OpenAI Built Its Own Silicon
Money explains most of it. Nvidia's data-center margins have made accelerators the priciest line item in any frontier lab's budget, and buying custom silicon lets a big customer capture some of that margin itself. Control matters too: co-designing chip, memory, network, and serving software around real workloads is the same playbook Google ran with its TPU program, and Broadcom has spent years building exactly this kind of silicon for hyperscale customers.
OpenAI isn't abandoning Nvidia, to be clear. The company says it will keep deploying accelerators from Nvidia and other partners for both training and inference, and Jalapeño covers only part of its fleet. But the message lands during Nvidia's earnings week: the largest buyer of AI compute can now build what it cannot buy off the shelf.
Deployment begins within OpenAI's own infrastructure by the end of the year. Gen 2 is deep in development and Gen 3 is taking shape, according to the company. Data Center Dynamics reports that racks will hold 128 chips, with full pods reaching 2,048 ASICs, citing Richard Ho, OpenAI's VP of hardware, speaking at Hot Chips.
Independent verification is the obvious next question. Every number published Monday comes from OpenAI's own testing, normalized by its own methodology. SemiAnalysis maintains the InferenceX benchmark, but the configurations and tuning choices remain in the chipmaker's hands until third parties get rack time. The company has invited scrutiny by publishing on a public benchmark rather than a private one — a choice some rivals haven't made. Nvidia, for its part, will get a chance to respond with its own benchmark disclosures as Rubin-class parts reach customers.
Sources: Data Center Dynamics — OpenAI details Jalapeño AI chip, with 700W TDP, OpenAI — Jalapeño's first results show industry-leading speed and efficiency in AI inference, Reuters — OpenAI unveils custom chip it designed with Broadcom
Internal links: AI, Semiconductors
Keywords: artificial intelligence, OpenAI, Broadcom, Jalapeno, custom silicon, AI chips, Nvidia, AI inference, Hot Chips