Networking Is Core to AI Factory Full-Stack Optimization

Getting your Trinity Audio player ready...

The event reflected that shift. Held July 8-9 at Le Carrousel du Louvre, RAISE Summit 2026 drew 9,000 attendees, 2,000 companies, and 350 speakers, weighted toward C-level leaders and founders.

That scale matters because the conversation is changing. The questions are no longer only about which model performs best, which application will win, or which company will release the next major capability. At RAISE, the discussion was increasingly about the practical systems required to make AI work at industrial scale: compute, storage, networking, power, cooling, cybersecurity, governance, sovereign infrastructure, financing, latency, and production operations.

Why AI is moving from models to AI factories

An AI factory is the complete stack of infrastructure an organization runs to produce AI outputs at industrial scale. A factory is not defined by one machine. It is defined by the way many systems work together to produce an output reliably, repeatedly, and economically. In AI, that output may be recommendations, code, images or autonomous workflows. But the principle is the same: production quality depends on the whole system. That same logic is reshaping how the factory is judged, a shift from a GPU-as-a-service mindset to a token-driven one: what matters is the tokens produced and the performance delivered, not which GPU is under the hood. That reframing expands the field beyond a single dominant accelerator vendor to a genuinely multi-vendor ecosystem.

The answer will not come from optimizing one component in isolation. It will come from optimizing the full stack. Every layer matters: compute, storage, power and cooling, software orchestration, security and governance. The network fabric is what binds them together.

The network is not a separate infrastructure afterthought. It is one of the core systems that determines whether an AI factory behaves like a collection of expensive components or like a coordinated production system.

Time to First Token (TTFT) makes full-stack performance visible

One of the clearest examples from RAISE was Time to First Token, or TTFT, the subject of the session “Time to First Token, the metric that will define our era”.

TTFT has become an important metric because it connects infrastructure directly to user experience. When a user sends a prompt, how quickly does the system begin producing a useful response? The answer depends on many layers at once: the application, the model, the scheduler, the accelerator, memory, storage, inference architecture, and network behavior.

TTFT is not only a model metric or a GPU metric. It is a system metric.

If an AI service is slow, the bottleneck may not be obvious from a single component benchmark. The model may be ready, but the accelerator may be waiting. The accelerator may be available, but the data may not arrive fast enough. The system may have enough aggregate bandwidth, but congestion or synchronization delays may hurt tail latency. The infrastructure may look powerful on paper, but under real workloads, the user experience depends on how well the full system behaves.

For AI factories, this is the difference between capacity and productivity.

A cluster can have massive compute capacity and still underperform if the system cannot keep accelerators fed, synchronized, and efficiently utilized. As AI workloads become more distributed, more latency-sensitive, and more heterogeneous, the network fabric becomes a central part of that productivity equation.

The network fabric is the operating layer of the AI factory

In conventional enterprise infrastructure, networking is often discussed in terms of connectivity, bandwidth, ports, and availability. Those attributes still matter, but AI infrastructure adds a different set of demands. AI workloads require extremely high levels of coordination between accelerators, storage systems, memory, data pipelines, schedulers, and applications.

That is why AI networking is better described as an AI fabric than as a traditional network.

The fabric is what turns many distributed components into one functioning AI system. It determines how efficiently accelerators communicate. It affects how quickly data moves across the environment. It shapes congestion behavior, tail latency, synchronization efficiency, and operational visibility.

RAISE’s own agenda reflected this systems view. The session “Inside the AI Inference Cluster: Measuring What Matters” described the network as a central layer connecting every accelerator, linking prefill to decode, and connecting storage to compute. It also highlighted that this is the kind of layer that per-chip benchmarks can miss, even though real-world performance increasingly depends on it.

The strongest evidence that this is not a vendor talking point is that the industry’s largest chip and cloud companies have standardized around it. In June 2025 the Ultra Ethernet Consortium published Specification 1.0, a full Ethernet-based communication stack purpose-built for AI and HPC, including a new Ultra Ethernet Transport. Its founding members include AMD, Broadcom, Intel, Meta, and Microsoft, and the consortium has since grown past 100 members, including NVIDIA, alongside AI-networking vendors such as DriveNets. Companies that compete on almost everything else agreed that the transport layer is a limiter worth rebuilding. When network architecture is treated as a late-stage deployment detail, the entire factory inherits constraints that are expensive to fix later.

Three priorities for teams building AI factories

1. Latency is becoming a strategic metric, not an engineering footnote. As distributed inference grows into the majority of AI compute and enterprise usage, responsiveness becomes part of how users and buyers judge quality. TTFT, time between tokens, and tail latency will sit alongside throughput and cost in procurement criteria.
2. Optimization is becoming system-level. Accelerator performance, storage throughput, model architecture, orchestration, and fabric behavior are tightly coupled. Improving one layer without understanding the others no longer moves the number that matters, which is delivered output per dollar and per watt.
3. Environments are becoming heterogeneous. The next generation of AI infrastructure will not be built around a single accelerator generation, a single cloud, or a single deployment model. Enterprises will run across different chips, clouds, clusters, models, and software stacks, and increasingly across sites. Heterogeneity makes the fabric more important, not less, because it is the common operational and performance layer across a more complex environment.

Why AI’s next phase will reward systems, not components

RAISE 2026 showed that AI has entered its factory phase. Compute is the engine. Storage holds the fuel. Power and cooling make the factory possible. The fabric is what lets the whole system operate as one.
The next phase of AI infrastructure will reward the organizations that optimize systems rather than components. As AI factories scale, the network fabric will be one of the places where that optimization becomes real, and one of the clearest dividing lines between cluster capacity that sits idle and capacity that pays for itself.

Key takeaways

• The Shift to Infrastructure: The AI industry has officially transitioned from model-level excitement to industrial-scale infrastructure execution.
• A Token-Driven Focus: Infrastructure value is now being judged by token production and system-wide performance rather than raw hardware or GPU counts.
• System-Wide Latency Metrics: Time to First Token (TTFT) is recognized as a holistic system metric that exposes synchronization delays across compute, storage, memory, and the network.
• Ethernet Fabric Standard: The network has emerged as a strategic core, highlighted by the rapid adoption of the Ultra Ethernet Consortium Specification 1.0 for multi-vendor optimization.

 

Frequently Asked Questions

What is an AI factory and how is its performance measured?

An AI factory is the complete stack of infrastructure an organization runs to produce AI outputs at scale. Rather than evaluating isolated GPU capacity, its performance is measured by total token output and system efficiency. This full-stack approach optimizes compute, storage, power, and networking to deliver reliable, economic, and high-performance AI results.

Why is Time to First Token (TTFT) considered a critical system metric?

Time to First Token is a system-level metric because it directly measures how quickly a user receives a response. This latency depends on the synchronized behavior of applications, models, schedulers, accelerators, storage, and networking. TTFT reveals overall system productivity, highlighting whether network congestion or data delays are bottlenecking powerful compute clusters.

How does the network fabric enable multi-vendor AI factory environments?

The network fabric serves as the common operational layer that unifies heterogeneous compute, storage, and software systems. By standardizing on open protocols like the Ultra Ethernet Consortium Specification 1.0, organizations avoid vendor lock-in. This high-performance communication stack coordinates diverse accelerators and chips to optimize delivered output per dollar and watt.