- GenAI can increase productivity by up to 66%.
- The majority of employees bring their own GenAI tools to work, whether their boss knows about it or not.
- As AI becomes more sophisticated, organizations are leveraging strategies to manage security risks and skyrocketing costs.
Workers around the world are harnessing the benefits of AI to improve workflows and drive innovation, but as the widespread use of generative AI explodes across teams and functions, organizations need to implement it securely and cost-effectively.
According to a new survey from May 2024, 75% of knowledge workers worldwide are already using GenAI applications, up 46% from six months ago. And employees no longer need to wait for direction from management to get started: more than 78% bring their own GenAI tools to the workplace.
It's easy to see why employees are eager to use GenAI: it can increase productivity by up to 66%. But with increased use of GenAI comes increased risk, especially for organizations without a formal AI strategy. A recent survey found that 71% of organizations don't provide employees with guidance on when, where, and how to use AI.
“Enabling anyone to deploy GenAI applications and use them however they like raises serious security and governance issues,” says Faz Hussain, senior manager, AI portfolio marketing, Dell Technologies. “Organizations need to remember the hard lessons learned when cloud computing was introduced and poor management gave rise to shadow IT. If we're not careful now, we'll fall victim to shadow AI.”
Like shadow IT, shadow AI leads to spiraling costs, data silos, and security risks.
Fortunately, organizations can avoid the mistakes of the past by building an AI strategy for the future. Making the right strategic decisions about development and implementation can maximize control while optimizing cost efficiency.
Solutions for GenAI implementation
Enterprises have a variety of options for building and deploying GenAI and need to carefully consider security and cost factors. Here are some options:
- Building a Large Language Model (LLM) from Scratch
Building an LLM from scratch requires significant resources and expertise, making it too costly for most organizations.
- Accessing the LLM via an Application Programming Interface (API)
API-based LLM often requires third-party data processing, creating security risks and making PPI and other information subject to privacy or compliance regulations unavailable to users.
Pricing is based on the number of queries your users submit, which can make it difficult to evaluate and scale your application.
“There are also charges for the hardware, software and maintenance of the solution. These costs are also included in the price,” Hussain said.
- Building Applications on Public Cloud Infrastructure
Building in the public cloud allows developers to fine-tune their models more than they can with APIs, and it also gives them control over costs, but there are limits.
“Many companies think they are taking the 'easy' route by contracting for only the computing power they think they will need, but costs can add up quickly,” Hussain said.
Estimating computing resources can be difficult, especially for new capabilities like GenAI, and companies must also consider data storage and data movement charges.
“There are costs associated with fine-tuning models and moving data around, especially to different regions,” Hussain added. “You'll also pay for the provider's hardware and software, as well as API services.”
- Download and customize the open source LLM
After downloading an open-source model like Mistral or Meta Llama 3, you can tweak it to suit your needs or use a process called search augmentation generation (RAG) to feed the model with your company's data to improve its answers.
The RAG directs the LLM to retrieve reliable information relevant to your use case. You can input your own data in-house and do not need to send it outside your company.
For organizations considering adopting GenAI, RAG is a great starting point, says Hussain.
“It eliminates the need to train models from scratch and allows you to leverage your data in-house and protect it wherever it goes,” he said.
Once a solution is developed, employees can use it from anywhere, running inferences on-premise, in the cloud, or through a data centre on their own computers – data is always protected by the company's own security and governance protocols, and because applications are built and deployed in-house, there are no service fees.
Proven cost-effectiveness
A customized open source LLM offers clear security benefits by keeping corporate data in-house, and it also provides better cost control, but how much of a difference does it make?
Tech Target's Enterprise Strategy Group conducted an economic analysis in collaboration with Dell to find out: Researchers found that running GenAI inference on a company's infrastructure can be up to 75% more cost-effective than using the public cloud and up to 88% more cost-effective than using API services.
A similar economic analysis conducted by Principled Technologies in collaboration with Dell found that running GenAI fine-tuning and inference solutions using Dell Technologies on-premise can be up to 74% more cost-effective than the public cloud.
For organizations with thousands of users and large LLMs, the benefits are even greater as they can leverage economies of scale.
Moving forward with GenAI
Hussain said businesses, regardless of size, shouldn't wait to adopt GenAI.
“Your employees are already using it, so if you don't plan for it, you're essentially succumbing to shadow AI,” he said. “There are a variety of architectures you can use to securely and cost-effectively integrate GenAI into your ecosystem and unlock its power for your business.”
To learn more about how Dell solutions can help you deploy GenAI securely and cost-effectively, click here.
This post Insider Studio With Dell.
