Q&A: Unlocking the Potential of Generative AI with the CEO of OctoML

Generative Artificial Intelligence activity has surged in recent months, and OctoML Inc. has been at the forefront of this revolution.

OctoML, a leading start-up in the generative AI space, has just announced its OctoAI computing service. The company aims to revolutionize the AI ​​industry by providing developers with a fully managed cloud infrastructure that simplifies the process of creating AI applications. OctoAI aims to eliminate complexity, abstract the heavy lifting involved in model deployment, and provide developers with freedom, efficiency, and ease of use.

This solution marks an important milestone in the company’s mission to empower developers and push the boundaries of generative AI. By providing a self-optimizing computing service, OctoML addresses the challenges faced by developers such as the shortage of GPUs and the confusion surrounding various AI services available in the market.

Madrona Venture Group Partner Jon Turow (pictured left) and OctoML Co-Founder and CEO Luis Ceze (right) spoke with theCUBE Industry Analyst John Furrier in the CUBE Conversation. They discussed OctoAI and the impact of generative AI on the industry. [The following conversation has been condensed for clarity.]

Lewis, we’ve talked a lot about your role in the industry. I have news to announce today.

Ceze: Release OctoAI computing service. This is the first self-optimizing computing service for generative AI. Freedom as users can choose their own model or bring their own custom model. Second, it improves efficiency by optimizing the model and choosing the right hardware to get the right trade-off between performance and efficiency. Third, it is very easy to use. We make it easy for people to get started by providing a collection of ultra-optimized models such as stable diffusion, LLaMA-based his LLM, and Whisper for speech transcription.

There is a lot of confusion about how to get into the generative AI business. I have never seen such acceleration. What problem are you solving for developers?

Ceze: First, it abstracts complexity and helps clear this confusion. We provide the ability for developers to access the platform, choose a use case such as text-to-image conversion or text-to-text conversion, and immediately start using state-of-the-art out-of-the-box models. doing. Ready to integrate into your environment. It also abstracts away all this incredible complexity that is required when deploying a model to production.

John, you’ve been in charge of this area. Madrona is apparently an early-stage investor in the company. The world spins right next to OctoML. what is your rating? What do you see on this new platform?

Turow: What’s really exciting about the world we’re in right now is that we’re having the kind of Android moment that Lewis and I wrote about earlier. We have a very exciting model. You could also call it an iPhone-like model, such as his GPT-4 or ChatGPT, an AI model from OpenAI. These models can be assembled into so-called ensembles (some people call them cocktails) of many models working together. That’s a good thing, and we’ve seen RunwayML, Midjourney, and other great companies build on open source AI, but until now, it’s been a long time since we’ve started using AI, managed it, and managed everything. It was difficult to implement. What’s interesting about Octo, Luis, and the team’s work is that, for the first time, open-source AI offers usability similar to what you’d get in a closed model. And it will create many new innovations.

Why OctoML? What does this mean for me? I want to participate and nail the model. I would like to understand how that applies. I’m fiddling with tires and kicking them. How does it work?

Ceze: This topic is obviously close and dear to me because of the OctoAI computing service, but also because I am a computer architect by training. I love seeing how chips are so important in this new phase of the world. What we offer developers is the ability to not worry about infrastructure. What does this mean in practice? It gives you choice because you can radically optimize your model and how it will run on real hardware. You don’t need an A100, you can use the A10G for example, or you can use the readily available T4. Combining the optimization itself, which uses less compute, with the ability to move the work and abstract it from the end-user’s perspective, gives you access to more silicon. That translates directly into increased cost efficiency. Abstracting hardware choices is a key part of our mission so that users can focus on building applications that really matter.

Here is the full video interview, one of many CUBE conversations with SiliconANGLE and theCUBE:

Photo: Silicon ANGLE