Microsoft just made its biggest bet yet on its own silicon. At the OCP Global Summit in San Jose on Monday, the company unveiled Azure Maia 300, the third generation of its in-house AI accelerator — and announced plans to deploy potentially more than a million of them next year, making Maia the primary inference system across Microsoft’s products and services, Reuters reports.
Satya Nadella framed the scale bluntly in a post on X: starting next calendar year, Microsoft will scale the Maia family to multi-gigawatt deployments. Maia 200, the current generation, is already running production workloads including Microsoft 365 Copilot across the US, Europe, and Australia.
The Headline Numbers
Microsoft claims Maia 300 delivers 40 percent more tokens per megawatt than leading GPU-based systems. In a full rack configuration of 96 accelerators, it promises up to seven times the rack-scale token throughput of Maia 200 — aimed squarely at long-context reasoning, latency-sensitive conversations, and multi-step AI agent tasks, according to CNBC.
The specs behind those claims are unusually detailed. Eight stacks of HBM4 memory deliver 288GB at 18 TB/s of peak bandwidth within a 950-watt TDP, and the die splits its duties across chiplets built on different process nodes: the compute chiplets use TSMC’s advanced 2-nanometer-class technology, while data-movement functions sit on cheaper 3-nanometer silicon — a design choice that trades Nvidia’s one-big-die philosophy for a cheaper, more modular build.
The Number Microsoft Won’t Name
Here’s the part to read carefully: Microsoft says “leading GPU-based systems” without naming them. The comparison that Reuters could pin down is power, not performance: a Maia 300 rack draws about 167 kilowatts, versus roughly 142 kilowatts for Nvidia’s latest GB300 racks with 72 GPUs. Microsoft is claiming 40 percent more useful work per megawatt while feeding the rack more power — which only pencils out if the throughput gains are as large as advertised. Independent benchmarks don’t exist yet, and the IEEE ComSoc technology blog notes that validating the claim will require comparable workloads, defined power boundaries, and latency measurements.
Meanwhile in Tech News: the AI chip build-out is being financed by a wave of debt nobody wants to talk about.
Why This Matters
The real fight isn’t about training giant models — Nvidia still holds a near-lock there. It’s about inference: the relentless, power-hungry business of answering questions, summarizing meetings, and running agents once models are trained. Inference is the bulk of what Copilot and GitHub Copilot do all day, and power translates directly into operating cost. “More intelligence for every watt, every dollar, and every byte of memory,” as Rani Borkar, president of Azure hardware systems and infrastructure, put it to Reuters.
Microsoft is also riding a broader wave: Amazon, Google, and Meta all have custom silicon in production or in the works, and OpenAI is developing its own chips. Maia 200’s early numbers — up to 1.7 times more tokens per watt on real Teams and Copilot workloads — suggest Microsoft already has the software stack to make custom hardware pay.
What Happens Next
Production deployments of Maia 300 begin in 2027. The honest verdict on this chip won’t come from a spec sheet — it will come from cost per million tokens on real workloads, under real concurrency, measured by people who don’t work at Microsoft. Until then, treat the 40 percent as an engineering target with a marketing tailwind.
Meanwhile in Tech News: Anthropic’s $11.6 billion Akamai cloud deal shows how fast AI infrastructure money is moving.
