Advertisement
Business

Why Nvidia’s Biggest Customers Are Building Their Own Chips

When one supplier controls the most sought-after component in your business, and knows it, what do you do? For Amazon, Google, Microsoft and Meta, the answer has been to start designing the component themselves. Each of these companies spends enormous sums on Nvidia’s AI accelerators, and each has also built, or is building, its own silicon to run a growing share of its workloads.

The effort is no secret. Google has run its Tensor Processing Units for years, Amazon rents out its Trainium and Inferentia chips through AWS, Microsoft introduced its Maia accelerator and Cobalt processor in late 2023, and Meta has developed its MTIA line. None of these is an attempt to dethrone Nvidia overnight. They are a hedge, and the logic behind that hedge shows where the money in artificial intelligence actually goes.

Rows of servers inside a large data center
Photo by panumas nikhomkhai / Pexels

Paying the Toll on Every Training Run

Nvidia’s accelerators are expensive because demand has outrun supply and because, for many tasks, there is no equal alternative. For a company buying them by the hundreds of thousands, that price stops being a line item and becomes a strategic exposure. Every dollar that flows to Nvidia as margin is a dollar unavailable for data centers, power contracts or research staff. When a product sits underneath your entire future revenue stream, the obvious question is what it would cost to make it yourself.

The math favors the giants. Designing a chip is a vast engineering undertaking, but a company running millions of servers can spread that cost across an enormous fleet, while a smaller firm cannot. Supply matters as much as price. Waiting in line for allocation of the industry’s scarcest hardware is a poor position for a business that has promised customers more AI features every quarter, and an in-house chip gives a company at least some control over its own delivery schedule.

What a Custom Chip Does Differently

A Nvidia GPU is a general-purpose engine. It is built to serve thousands of customers with different needs, so it carries capabilities that any single buyer may never use. A custom chip is the opposite proposition. It is built for one company’s models and one company’s data centers, which means designers can strip out what is unnecessary and tune memory, networking and power draw to the work actually being done.

The clearest payoff comes from the difference between training and inference. Training is the process of building a model, an enormous but bounded expense. Inference is what happens every time a person asks a chatbot a question, generates an image or gets a recommendation, and it repeats billions of times, scaling with the popularity of the product. A tiny saving per query turns into a very large saving across that volume. This is why many custom chips are aimed first at inference. The workload is predictable, the volume is huge, and the buyer knows exactly what it needs the chip to do. A chip that is merely good enough at a narrow job, at a much lower cost per answer, can beat a more powerful general-purpose part on the metric that matters.

Close-up of a computer processor chip on a circuit board
Photo by Jakub Pabis / Pexels

Google’s TPUs are the most mature example, having been refined across multiple generations and used for its own search, ads and model work. Amazon has pitched Trainium as a cheaper route for customers who want to train and run models on AWS. Anthropic, one of the leading model developers, has publicly tied its compute plans to both Google’s TPUs and Amazon’s Trainium, which suggests that even the most demanding AI labs are willing to spread their bets across hardware.

The phrase “in-house” deserves some skepticism. Design partners such as Broadcom and Marvell help cloud companies turn architectural ideas into manufacturable chips, and OpenAI has reportedly been working with Broadcom on its own accelerator. The factories that physically produce these chips, led by TSMC, are the same ones that produce Nvidia’s. The cloud giant owns the roadmap and the specifications, while a partner handles much of the physical engineering. That arrangement lowers the barrier to entry, but it also means the hyperscalers have not escaped dependence on the supply chain. They have only moved it.

The Software Moat Nvidia Still Holds

Hardware is the easy part to compare. What keeps engineers on Nvidia is CUDA, the software platform the company has built up for nearly two decades, along with its libraries, debugging tools and a generation of developers who learned to write AI code on it. A customer can order a cheaper chip, but if the models run slowly, crash in strange ways or need months of rewriting, the savings evaporate. Many of the engineers inside these very companies are fluent in CUDA, so the in-house chip often starts at a disadvantage in tooling and talent.

The counter-effort is aimed at the software layer itself. Frameworks such as PyTorch and JAX, along with compiler projects built to target multiple kinds of hardware, make it easier to move a model from one chip to another. Every step toward portability weakens the lock-in that CUDA provides.

Customers and Competitors at Once

Nvidia is still the default. The same companies building their own chips continue to buy Nvidia hardware in huge quantities, partly for their own use and partly because cloud customers ask for Nvidia-powered instances by name. Custom silicon is better understood as a second supply line than as a replacement. The most demanding frontier training runs still call for the fastest chips with the most mature software, and that remains Nvidia’s territory.

Nvidia’s response has been to move faster.

Illuminated server racks in a modern computing facility
Photo by panumas nikhomkhai / Pexels

The danger for the company is not that a rival chip will beat its best product in a head-to-head test. It is that its largest customers each need a little less of it every year, as more of their steady, predictable inference work migrates to silicon they designed. If inference becomes the bulk of AI spending, the winners will be whoever delivers the cheapest answer per query, and Nvidia’s biggest buyers are quietly building exactly that.

Related Articles