Microsoft unveils Maia 300

Microsoft used the OCP Global Summit in San Jose to reveal Maia 300, the next generation of its in‑house inference accelerator and the cornerstone of an expanded Azure Maia program.

The company said Maia 300 will begin production deployments in 2027 and that it plans to scale the Maia family to multi‑gigawatt levels starting next year, making Maia the primary inference system for Microsoft services.

At the architectural level, Microsoft describes Maia 300 as an "inference‑first" system that prioritizes reducing data movement and maximizing locality across compute, memory and networking. That approach is meant to lower the real‑world energy cost of running large language and agentic workloads.

Microsoft published a set of chip and rack figures that underline that system focus: a Maia 300 chip built with a multi‑chiplet design on TSMC’s N2P process, eight HBM4 stacks delivering a peak 18.0 TB/s of bandwidth inside a 950 W TDP, and software‑managed on‑die SRAM to cut pressure on HBM.

On the rack level, Microsoft said a full Maia 300 system packs 96 accelerators into a single node drawing roughly 167 kilowatts and offering 27.6 TB of HBM capacity with 1.7 PB/s of aggregate HBM bandwidth. The company also claimed Maia 300 will deliver about 40% more "tokens per megawatt" than leading GPU‑based systems.

The blog and company briefings framed those numbers as practical gains: Maia 300 aims to support long‑context reasoning, latency‑sensitive conversational experiences and sustained multi‑step agent tasks while improving throughput per watt at datacenter scale. Microsoft said Maia 300 can yield up to seven times the rack‑scale token throughput of Maia 200 when configured with 96 accelerators.

Microsoft has already deployed Maia 200 in production for internal workloads, and it expects Maia 300 to take over as the primary inference engine for many of its services, including Copilot and internal Microsoft AI models. The company said this is part of a broader move to reduce dependence on third‑party accelerators for inference.

Outside analysts and reporters framed the announcement as another sign that hyperscalers are racing to own more of their AI stack. Observers note similar programs at Google, Amazon, Meta and other large AI users, and said Microsoft’s push underscores how inference — not just training — is a high‑value target for custom silicon.

Press reports and Reuters‑based coverage have also said Microsoft may deploy hundreds of thousands — potentially more than a million — Maia 300 chips as it ramps to multi‑gigawatt scale next year, though Microsoft did not publish a definitive chip count in its technical overview. Those figures are reported as targets and estimates tied to the ramp, not final purchase orders.

The announcement also highlights practical engineering tradeoffs. Maia 300’s multi‑chiplet approach and custom AI transport — built on Ethernet‑based fabrics — aim to make networking part of the dataflow rather than a separate layer, which Microsoft says helps scale from a rack to the datacenter without moving unnecessary data between tiers. Competitors and partners will watch whether that software‑defined dataflow delivers gains at scale.

For Microsoft, the business case is clear: controlling more of the inference stack can lower unit costs, improve latency for interactive services and hedge against supply constraints in the GPU market. But the shift also deepens the industry’s hyperscaler silicon race, raising new questions about supply chains, manufacturing capacity at foundries, and how heterogeneous fleets of custom and third‑party accelerators will interoperate.