Read the full transcript
How to make scale-across work
Hi, I’m Dudy Cohen from DriveNets, and we’re going to talk about scale across today. Actually, we have successfully deployed scale across together with WEKA, the storage provider, and WhiteFiber, which operates the system. And we want to talk about a few points you need to remember when you plan and deploy scale across.
First, you need to plan ahead. You need to plan your network, for instance, with shallow buffer for the intra-site connectivity and deep buffer for the cross-site connectivity.
You need to make sure you have sufficient capacity, preferably dark fibers between the systems, and you need to make sure you have a scheduler that can spread or schedule the workload across different systems. We have done it successfully. We have great results.
Data protection and resiliency for regulated enterprises
I’m Toby, Head Design Architect at WhiteFiber.
Our scale-across architecture allows us to support workloads across multiple sites. Today we have a single setup between 2 sites. We’re soon to expand into multiple sites where we have 3 or more in a hub-and-spoke model or in a mesh configuration, which allows us to use data center capacity in smaller modular units, combine them together into one larger fabric.
In our scale across architectures, we’re Putting those in place to make sure that we can provide our customers the kind of uptime that they expect and meet the compliance regimes that they have. We ensure that we have an appropriate amount of capacity at their disposal and then extra as needed to make sure that if any failures happen, that we can recover for any kind of disaster.
Breaking the datacenter MW limit
Today we’re going to talk about how to resolve the major challenge of any AI infrastructure, power, because it’s all about power. And when you run out of power in your data center, you are locked because you cannot add more GPUs and CPUs in order to serve your workload.
Now we have a solution. You can use another data center up to 50 and even 100 miles apart from the original data center and scale across the network between the 2 data centers and add GPUs there. Then both GPU clusters are working as a single cluster, and you might as well deploy it in the same data center.
We have done this together with WEKA and with WhiteFiber that operates the system, and the results are amazing.
Elastic scale and uptime for NeoClouds
Hey, I’m Toby, Head Design Architect at WhiteFiber.
Our scale-across architecture allows us to support workloads across multiple sites. Today we have a single setup between 2 sites, We’re soon to expand into multiple sites where we have 3 or more in a hub and spoke model or in a mesh configuration, which allows us to use data center capacity in smaller modular units, combine them together into one larger fabric.
Distributing latency-sensitive inference
Hi, this is Dudy Cohen from DriveNets, and we’re going to talk about latency-sensitive applications in AI today. In order to reduce the latency associated With inference workload, you need to deploy these GPUs closer to the customer at the edge of the network.
In order to do that, you need to use scale across like we did with white fiber. You need to distribute your GPU infrastructure across multiple sites and use scale across in order to distribute the workload across the entire network.
You need to measure time to first token. pay latency, time between tokens, not just round-trip delay in order to make everything work. But now you have a solution and you can deploy latency-sensitive applications and get great performance.
From 2 sites to 20
Hey, I’m Toby, Head Design Architect at WhiteFiber. I’m talking a bit about our scale-across architecture.
Today we have a 2-site architecture in Atlanta where we have workloads that span a GPU fabric across both sites. We’re now gonna expand to the next level, which would include more than 2 sites, 3 in a mesh or 4 or 5 in a hub and spoke model. Those, these types of scale-out, scale-across architectures allow us to take advantage of a lot of unused or smaller sets of capacity at our data center partners.
Want to learn how DriveNets is reshaping AI infrastructure?
Explore DriveNets AI Fabric full stack networking solutions for AI Infrastructure
