Gcore announced the launch of Gcore Inference at the Edge, a solution that delivers an ultra-low latency experience for AI applications. The solution enables distributed deployment of pre-trained machine learning (ML) models to edge inference nodes, ensuring seamless, real-time inference.
Gcore Inference at the Edge provides cost-effective, scalable and secure deployment of AI models for enterprises across industries including automotive, manufacturing, retail and technology. Use cases such as generative AI, object recognition, real-time behavioral analysis, virtual assistants and production monitoring can now be rapidly realized at global scale.Gcore Inference at the Edge runs on Gcore's extensive global network of more than 180 edge nodes, all interconnected by Gcore's low-latency smart routing technology. Each high-performance node is located at the edge of the Gcore network, strategically placing servers close to end users. Inference at the Edge runs on NVIDIA L40S GPUs, the market-leading chips designed specifically for AI inference. When a user submits a request, the edge node determines the route to the nearest available inference area with the lowest latency, delivering typical response times of less than 30 milliseconds.
The new solution supports a wide range of foundational ML models and custom models. Open-source foundational models available on the Gcore ML model hub include LLaMA Pro 8B, Mistral 7B, and Stable-Diffusion XL. Models can be selected and trained for any use case and then distributed globally to Gcore Inference on edge nodes. This addresses the challenge faced by development teams where AI models typically run on the same servers used for training, resulting in reduced performance.
The benefits of Gcore inference at the edge include:
Andre Reitenbach, CEO of Gcore Comment: “Gcore Inference at the Edge enables our customers to focus on training their machine learning models without worrying about the cost, skills, and infrastructure required to deploy AI applications globally. At Gcore, we believe that the edge is where you get the best performance and end-user experience, so we are continuously innovating to ensure all our customers get unparalleled scale and performance. Gcore Inference at the Edge gives you all the power without the hassle, delivering a modern, effective, and efficient AI inference experience.”
