Demand Will Fill Your Data Center. Performance Will Decide What It’s Worth.
Right now, it feels like everything in AI is working: another coding assistant, another wrapper, another neocloud running the same GPUs as everyone else. In infrastructure, power, permits, memory and accelerators are so scarce that any data center that can generate tokens finds an off-taker quickly.
However, every tech cycle teaches the same lesson: when it turns, the market re-prices everything undifferentiated. It happened to hosting and bandwidth after the dot-com boom. It will happen to tokens.
Demand is not a moat. Neither is capacity. The neoclouds that win will generate the most revenue and profit from every gigawatt and every dollar invested, over the entire life of the cluster.
The money is made after deployment day
Scarcity pushes neoclouds toward the conservative bet: the standard reference design, one vendor for everything, the fastest path to power-on. That optimizes for deployment day, but leaves you with the same service as every other neocloud that made the same bet.
In contrast, the most ambitious neoclouds and inference providers I talk to are optimizing for what comes next. A large share of a cluster’s profit comes after the first lease ends, and by then, supply will likely have caught up.
Once the crunch eases, revenue per gigawatt comes down to uptime, sustained performance, utilization and the ability to charge for performance. The choices you make before deployment day decide all four. Get them right, and the cluster stays profitable throughout its lifetime, making the next one easier to finance. Get them wrong, and the market will punish you.
In that market, intelligence is the new oil, and undifferentiated tokens trade on price. A refinery whose cost per barrel sits above the market price shuts down, no matter how strong demand was last year. An AI factory that spends more to produce a useful token than the market will pay for it faces the same fate.
Performance is the product
Off-takers pay for GPU-hours, but what they’re buying is performance: tokens per second, time to first token, and a cluster that behaves in production the way it did in acceptance testing. To make matters worse, when it doesn’t, they often can’t tell why, because they run on infrastructure they don’t own and can’t see into.
We’ve heard many of these stories. Here are a few examples:
- An opaque system. One cluster delivered more than an order of magnitude less all-gather bandwidth than another on the same workload. The neocloud didn’t share its switch counters, and weeks later the gap disappeared, unexplained.
- The wrong NIC. KV cache transfers ran several times slower than they should have because traffic quietly went over a storage NIC. It took months to find.
- The silent fallback. With RDMA misconfigured, traffic fell back to TCP and throughput collapsed. No alarm fired.
- The problem that only appeared at scale. Fine within one leaf switch, erratic across the full cluster, and without visibility into the fabric, no one could say why.
Each one cost both sides: underused accelerators, weeks of engineering time, and traffic moved to neoclouds that performed. As performance moves into contracts, SLAs turn problems like these into expensive penalties. On the flip side, a neocloud that can see, explain and prove its performance can guarantee it and charge for it.
Built for workloads that don’t exist yet
There’s another reason to look past deployment day: inference changes every few months. Prefill and decode increasingly run on separate pools. KV caches spill into tiers of memory and storage. Mixture-of-experts models spread work across many accelerators, and some AI factories mix accelerators to match each workload to the right silicon. Every shift makes the system more disaggregated. The cluster you build today will spend most of its life running inference stacks that don’t exist yet, and whatever you build has to adapt with them.
The highest-leverage line item
Of all the choices you make before deployment day, the network has the most leverage. It’s only 10-15% of the cost of an AI cluster, but it touches every accelerator, every storage node and every request. And every one of the four stories above played out in the network, the part of the AI factory most often run as a black box.
That’s why we built Aria Deep Networking. Our AI-class switches run AriaOS, our hardened, SONiC-based operating system, on Broadcom silicon. Aria correlates its microsecond-level telemetry end to end with hosts and NICs, whatever the accelerator vendor, and its agents find the root cause and remediate.
Take disaggregated inference. Every request hands its KV cache from prefill to decode across the fabric, sometimes between different vendors’ silicon, and even brief congestion leaves decode waiting. Aria sees that congestion, or KV traffic riding the wrong NIC, as it happens rather than weeks later. Or take mixture-of-experts. When NVIDIA Dynamo or its equivalent over-schedules a few workers, or a few experts run hot, traffic piles onto a handful of links. A blind network sees random slowdowns. Aria sees which links, hosts and jobs are involved, and rebalances traffic across paths as appropriate.
Because we’ve specialized AI for the networking use cases, it continuously learns and adapts as models and inference stacks change. The result is higher uptime, more useful work from every accelerator, and performance you can prove and charge for: revenue per gigawatt, compounded over the life of the cluster.
Bringing both sides together
Lately, off-takers have started asking us to introduce them to neoclouds that run Aria. They want capacity they can see into, from operators who can prove what they deliver.
So let’s build the market for performance:
- Off-takers: if you’ve had performance problems nobody could explain, whether you’re an inference provider, a lab or an enterprise renting capacity, come tell us about them. We’ll connect you with neoclouds that run Aria.
- Neoclouds and AI factory builders: if you want to compete on performance, we’ll supply the network, put our forward-deployed engineers on your cluster, and introduce you to off-takers that want visibility into their infrastructure.
At OCP Global Summit in San Jose, October 12-15, we’ll be talking to our customers, partners, and stakeholders - and we’d love to make introductions. If you’re there, come find us. Our team will be at the San Jose Marriott, next to the convention center, with Tomahawk 6 hardware and Deep Networking demos. We’d love to compare notes. Reach out at sales@arianetworks.com and we’ll make time.
Demand will fill your data center. Performance will decide what it’s worth.
