According to Chalan Aras, Riverbed’s senior vice president and general manager of acceleration, the physical limitations of on-premises storage and computing power initially drove companies to move to the cloud, giving users access to “virtually limitless” resources.
As a result, petabytes of data are now being stored in the cloud, and organizations are eager to apply artificial intelligence (AI) to leverage this information.
However, this data may not be in an ideal location for AI processing. In a multi-cloud strategy, enterprise data is inherently distributed across multiple providers. Even if all the data needed for an AI project resides in one cloud, power can be expensive and data may be stored in regions that don’t have the graphics processing units (GPUs) needed for AI workloads. Either way, businesses are faced with the daunting task of moving large amounts of data.
Moving such data is expensive, Aras warned, with egress charges alone costing up to $80,000 per petabyte, even when transferred within a single cloud provider. Furthermore, the transfer must be tightly controlled to ensure that the right data reaches the right destination fully intact. Another big bottleneck is speed. It takes about 9 days to transfer just 1PB over a 10Gbps connection.
That’s just historical data. It is common to keep an AI model fed with fresh data, perhaps once or twice a day. Volumes are much smaller, but the model is running 24 hours a day, so it’s still important to make transfers quickly and efficiently, and governance is still important.
Riverbed is now taking its 25 years of data movement experience and applying it to customers’ cloud environments, Aras explained. This process involves extracting data from storage and optimizing it for network transfer. “We serve it on a plate,” he said.
One organization needed to transfer 1PB of data to a new location for AI training, but found that the existing process would take 12 days. This is just the first part of the data, with another 20PB still to be moved. The organization had already reserved highly sought-after GPU time, and the looming deadline was at risk. Riverbed completed the entire task in three to four weeks instead of the planned eight to nine months, ensuring that data transfer would no longer be a limiting factor for the project.
Similarly, after a merger in the financial services sector, a company needed to transfer approximately 30PB of data from one cloud to another. Riverbed completed the migration in just over a month while meeting the necessary governance standards.
More broadly, IT teams at established companies have traditionally made decisions about data storage and processing based on the moment in time. On top of that are various decisions made at the departmental or line-of-business level. As a result, today’s businesses typically operate a combination of on-premises data centers, multiple cloud environments, and numerous Software-as-a-Service (SaaS) applications.
Although it is possible to consolidate to a single cloud provider, organizations must determine whether one provider can truly meet all their needs without having to make compromises. Even the biggest hyperscalers don’t have a presence in every region, Alas points out. For these and other reasons, a second provider is often needed, even if it means sacrificing the simplicity of a single contract and a single skill set.
Spreading your systems across multiple locations is rarely a problem until you need all your data in one place. This requirement is becoming increasingly common as AI adoption expands. To get the most value from their data, businesses face a real need to move large amounts of data continuously, rather than just once.
This is especially true for agent AI, which may need to pull information from a myriad of sources to respond effectively to prompts, Aras points out. This is great for users because it gives them very quick answers, but it requires moving data around frequently, he said.
Until recently, much of Riverbed’s business revolved around supporting one-time data transfers, such as migrating systems from on-premises locations to the cloud. However, Aras noted that customers increasingly need to continually move large amounts of data to feed their AI strategies. Riverbed’s approach makes the product suitable for both situations, he said.
