Rockset launches native support for hybrid vector and text search to power AI apps

Rockset Co., Ltd.real-time analysis database platform Built for the cloud, we announced native support today. Hybrid search that combines vector and text search and metadata filtering into a single query.

As artificial intelligence models continue to grow, Popularity of the need for powerful The data search and retrieval systems to power them have also grown with them. AI model, include meta platform Co., Ltd. Rama-3, OpenAI GPT-4, Google LLC Gemini, Applications that require access to both keyword and vector searches and the ability to index and rank vast amounts of data are proliferating.

“aThe model is being improved at an amazing pace;” Venkat Venkataramani CEO lock set, He said this in an interview with SiliconANGLE. “But there's one real problem with every model Among them they please do not they have some kind of memory or knowledge Can not remember anything and They are Can not memorize anything”

To bring that knowledge into AI models, engineers pull together knowledge from multiple sources, including vector databases, to index semantic data based on context. “intimacy” You can link to other information and search for information by similarity to other data. Additionally, traditional databases are used for keyword searches across structured data to derive additional information.rear it is Once ingested, narrow it down by re-filtering using metadata or geospatial data.

“There it is Many problems arise when you have multiple disparate systems, each performing a different part of the search.” Venkataramani said. “and you have to now combine themthe quality of the application will be degraded and the performance will be degraded and your application is not smooth”

Venkataramani should know something about data, having worked at Facebook before it became Meta Platforms Inc., managing its online data infrastructure as director of engineering. During his tenure, he helped Facebook grow its user-facing applications from about 30 million users to 40 million users to more than 1.5 billion users until he retired in 2015.

this Where lock set Hybrid search update is here. It offers hybrid search capabilities that do everything in one query, allowing developers to invoke vector searches, keyword searches, metadata filtering, and geospatial data. all at the same time. there is no “duct taping” of multiple systems, as Everything happens because Venkataramani calls it all at once.

Rockset can also apply an indexed ranking system at the same timeprovides powerful Responses for AI applications to understand data it is Recovering.

“Bcause was built for cloud, and a cloud-native database. There is There's nothing to download, but There's nothing to install either configuration, settings,” Venkataramani added.

Many applications could benefit from this type of hybrid search. For example, a real estate customer service company's AI chatbot app needs quick responses to customers searching for homes on the market. Your app must respond with the following information: House Where your customers are located — quickly Using geospatial metadata filtering similarly Dynamic rankings that are relevant to your context, based on your preferences for what types of homes you want to see and keywords related to those types of homes.

“circleWithout search expansion, you would Can not Do whatever it takes to unlock the true power of these models and you Discovering that all searches become hybrid searches,” Venkataramani said. “All large language model apps and all AI apps become search applications. There is no such thing as a pure vector search. pure keyword search or pure geographic search”

Image: Shutterstock/Nepur