Artificial intelligence is the most revolutionary paradigm shift since the internet became popular in 1994. And not surprisingly, many companies are eager to introduce AI into the way they do business.
One of the most important ways to achieve this is through generative AI and Large Language Models (LLMs), asking ChatGPT to write posts on specific topics for corporate blogs or to write code. It goes far beyond getting help. In fact, LLM is fast becoming an integral part of the application stack.
Building generative AI interfaces (“agents”) like ChatGPT on top of an LLM database that contains all the data it needs and can “speak the language” is the future (and increasingly present day) of mobile apps. ) is. Dynamic levels of interaction, access to vast amounts of public and proprietary data, and the ability to adapt to specific situations make applications built on LLM powerful and compelling in a way not available until recently. has become
And the technology has evolved so quickly that virtually anyone with the right database and the right APIs can build these experiences. Let’s see what is involved.
Generative AI revolutionizes how applications work
When you hear “agent” and “AI” in the same sentence, some people think of a simple chatbot that appears as a pop-up window asking how they can help you when you visit an e-commerce site. But LLM can do more than just respond with simple conversational prompts or answers from FAQs. When users have access to the right data, applications built on LLM are able to interact with us in a more sophisticated way, providing more useful, specific, and richer, expert-curated data. , and often provide surprisingly prescient information.
An example is shown below.
You want to build a deck in your backyard, so open your home improvement mobile application and ask them to create a shopping list. The application is connected to LLMs such as his GPT-4 and many data sources (the company’s own product catalog, store inventory, customer information, order history, and many other data sources) so that what the user does You can easily find out if Need to complete a DIY project. But you can do more than that.
You describe the dimensions and features you want your deck to include, and the application provides visualization tools and design aids. Since we know your zip code, we can tell which stores near you have the items you need in stock. Based on your purchase history data, we can also suggest that you may need a contractor to help with the work, and provide contact information for nearby professionals.
The application can also tell you how long it will take for dirt on your deck to dry (including seasonal climate trends where you live) and how long until you can actually host a birthday party on your deck. can. I have planned. The application can also provide information or help with many other related areas, such as project permit requirements and details of how construction affects property values. Still have questions? This application will help you every step of the way as a handy assistant in reaching your destination.
Is it difficult to use LLM in your application?
This is not sci-fi. Many organizations, including DataStax’s largest customers, are working on many projects that incorporate generative AI.
But these projects aren’t just the domain of established big companies. No extensive knowledge of machine learning, data science, or training ML models is required. In fact, building an LLM-based application requires little more than a developer who can make database and API calls. Anyone with the right database, a few lines of code, and an LLM like GPT-4 can build an application that can provide a level of personalized context unprecedented until recently.
Using LLM is very easy. They take context (often called “prompts”) and generate responses. So building an agent starts with thinking about how to give the LLM the right context to get the desired response.
Broadly speaking, this context comes from three places. These can be user questions, predefined prompts created by the agent developer, or data retrieved from a database or other source (see figure below).
A simple diagram showing how LLM collects context and generates a response.
User-provided context is typically just a question that the user has entered into the application. The second part is provided by the product manager who worked with the developer to describe the role the agent should play (e.g., “You are a friendly salesperson trying to help a customer plan a project. I’m an agent, please include content) “List of relevant products to include in answer”).
Finally, the third bucket in the provided context contains external data pulled from databases and other data sources that LLM uses to build responses. Some agent applications may call her LLM multiple times before printing the response to the user in order to create a more detailed response. This is facilitated by technologies such as the ChatGPT plugin and LangChain (more on these below).
Give memory to LLM
AI agents need a source of knowledge, but that knowledge must be comprehensible by an LLM. Let’s take a step back and think about how LLM works. ChatGPT has a very limited memory or “context window” when you ask a question. If you have a long conversation with ChatGPT, ChatGPT will pack previous queries and corresponding responses and send them back to the model, but it will start to “forget” the context.
This is why connecting agents to databases is so important for companies wanting to build agent-based applications on top of LLM. However, the database must store the information in a way that LLM can understand, i.e. as vectors.
Simply put, vectors allow us to reduce a sentence, concept, or image to a set of dimensions. You can take a concept or context, such as a product description, and transform it into some dimensional, or vector, representation. Recording these dimensions enables vector searches. This means you can now search based on multidimensional concepts rather than keywords.
This helps LLM to generate more accurate and contextually appropriate responses, while providing the model with a form of long-term memory. In essence, vector search is an important bridge between LLMs and the vast knowledge bases on which they are trained. Vectors are the “language” of LLM. Vector search is an essential feature of any database that provides context.
Therefore, a key component in being able to deliver the right data to LLM is a vector database with the throughput, scalability, and reliability to handle the massive datasets required to drive the agent experience. .
…with a suitable database
Scalability and performance are two key factors to consider when choosing a database for AI/ML applications. Agents need access to large amounts of real-time data and need fast processing, especially when deploying agents that every customer who visits a website or uses a mobile application might use. . When it comes to storing the data that feeds agent applications, being able to scale quickly when needed is the most important factor for success.
Apache Cassandra is the database that leaders like Netflix, Uber, and FedEx rely on to power their systems of engagement, making AI essential to powering every interaction a company offers. As engagements leverage agents, Cassandra becomes essential by providing horizontal scalability, speed, and rock-solid stability, and for storing the data needed to power agent-based applications. is the natural choice for
For this reason, the Cassandra community has developed a significant vector search feature that simplifies the task of building AI applications on huge datasets. DataStax has made these capabilities easily available via the cloud with Astra DB, the first AI-enabled petascale NoSQL database. It has vector capabilities (read more about this news here).
how it is done
As mentioned earlier, there are several ways organizations can create agent application experiences. Developers will hear him talking about frameworks like LangChain. LangChain, as its name suggests, facilitates the development of LLM-powered agents by chaining the inputs and outputs of multiple LLM calls and automatically fetching the right data from the right data sources as needed. Make it possible. .
But the most important way to move forward in building these experiences is through ChatGPT, which is currently the most popular agent in the world.
The ChatGPT plugin allows third-party organizations to connect to ChatGPT with add-ons that provide information about those companies. Think about Facebook.have become of It features a huge ecosystem of organizations building games, content and news feeds that can be connected to social networking platforms. ChatGPT has become that kind of platform, a ‘super agent’.
Developers may be working on building their own agent-based application experiences using frameworks like LangChain, but there is a huge opportunity cost to focus solely on that. By not working on a ChatGPT plugin, organizations are missing out on a large-scale distribution opportunity that integrates business-specific context into the range of information ChatGPT can provide and the actions it can recommend to users.
Various companies such as Instacart, Expedia, OpenTable, and Slack have already built ChatGPT plugins. Consider the competitive edge that an integration with ChatGPT might create.
Accessible Agents of Change
Building a ChatGPT plugin will be an important part of any AI agent project a company will be working on. With the right data architecture, especially a vector database, it’s much easier to build very performant agent experiences. Get the right information quickly to enhance your response.
All applications will be AI applications. With the rise of features like LLM and ChatGPT plugins, this future has become more accessible.
Want to learn more about vector searches in Cassandra? Register for the June 15 webinar.
