Together AI raises $800 million to grow AI-optimized public cloud

Together AI Inc., which operates a cloud platform optimized for running open source artificial intelligence models, has raised $800 million from investors.

In today’s funding announcement, the company said the Series C transaction was led by Aramco Ventures. Nvidia Corp., Vista Equity Partners, General Catalyst and several other institutional investors also contributed. Together, AI is now worth $8.3 billion.

Together AI’s platform includes a serverless inference service that developers can use to run open source AI models, eliminating the need to configure graphics cards or networking equipment. The company claims its serverless environment provides approximately twice the performance of the fastest alternatives.

The company also sells three other inference services. The two use a dedicated infrastructure that offers more reliability guarantees and customization options than serverless products. The third service, batch inference, prioritizes cost efficiency over speed. We offer up to 50% price reductions on models that do not require the ability to instantly respond to user prompts.

Under the hood, Together AI’s platform is powered by an Nvidia chip and a custom software engine called ATLAS. Accelerate customer workloads using a machine learning technique called speculative decoding.

Speculative decoding allows engineers to integrate an AI model with a second lightweight neural network. When a user enters a prompt, a lightweight algorithm quickly generates a draft response. The main model then checks for errors, makes any necessary changes, and returns a quick response to the user. This process is significantly faster than having the main model generate the output on its own.

Lightweight algorithms that create draft responses typically have a fixed configuration. Fixed-configuration models often become less accurate over time. Together AI says its ATLAS technology addresses the problem by automatically adapting lightweight models to changing user requirements. The company claims its software can speed up some inference workloads by 400%.

Customers can also use Together AI’s platform to fine-tune open source models. Provides access to training clusters with up to thousands of graphics cards. Developers can manage clusters using Kubernetes, which is relatively easy to use, or a tool called Slurm, which offers more customization options.

One of the main challenges with AI training projects is that you may encounter technical issues with your graphics card. In some cases, chip failure can cause errors in the training workflow. Together AI’s training cluster includes software that automatically detects and remediates technical issues.

The company announced today that annual bookings exceeded $1.15 billion in the second quarter. Its platform is used by thousands of organizations, including LG Inc.’s AI Research Institute, Cohere Inc., and the Mozilla Foundation.

Together, AI will use the newly raised capital to purchase more infrastructure. The company hopes to expand its public cloud capacity by 50 times over the next five years. Additionally, we plan to enhance training and inference capabilities.

Together AI founder photo: Together AI