The use of Large Language Models (LLMs) is causing a paradigm shift across the software development and computing industry. AI is happening and a new stack is forming before your eyes. It’s like the Internet has reappeared, bringing new infrastructure components to the service built for new ways of doing things..
LLM actually new type of computer, in a way. Can run “programs” written in natural language (such as prompts), perform arbitrary computing tasks (such as writing Python code or searching Google), and return the results to the user in a human-readable format . This is a big problem for two reasons.
- A new class of applications for summarization and generated content It can lead to changes in consumer behavior regarding software consumption.
- A new class of developers can now create software. Computer programming requires only learning English (or other human languages), no training in traditional programming languages such as Python or JavaScript.
One of Andreessen Horowitz’s top priorities is identifying the companies building key components of this new AI stack. We are pleased to announce that we are leading a $100 million Series B round. pineconeto support their vision of becoming the memory layer for AI applications.
Problem: LLM is hallucinating and stateless
A major challenge for LLMs today is hallucinations. They give very confident answers that are factually and sometimes logically wrong. For example, if you ask an LLM about Apple’s gross margin for the last quarter, confidently he might get an answer of $63 billion. The model could even support that answer by explaining that from $95 billion in revenue, he deducts $25 billion in commodity costs, yielding $63 billion in gross margin. Of course it is wrong in some dimensions.
- First, LLM doesn’t have real-time data, so the revenue numbers are wrong. You are dealing with old training data that is months or possibly years old.
- Next, we randomly sampled revenue and cost of goods from the financial statements of another fruit company.
- Third, the gross margin calculation is mathematically incorrect.
Imagine giving that answer to your company’s CEO. luck 500 companies.
After all, LLMs are prediction machines trained on vast amounts of third-party internet data. In many cases, the information users need is not included in the training set. So the model provides the most likely and linguistically well-formed answer based on the old training data. Potential solutions to the above problems are already beginning to appear. That means feeding his LLMs contextually and relevant private company data in real time.
The general form of this problem is that from a system perspective, LLM and most other AI models are stateless during the inference stage. Every time I call the GPT-4 API, the output is different that’s all About the data and parameters to send in the payload. The model has no built-in way to incorporate contextual data or remember what you have asked before. Fine-tuning the model is possible, but expensive and relatively inflexible (that is, the model cannot respond to new data in real time). Models don’t manage state or memory on their own, so it’s up to the developer to fill in the gaps.
Solution: The vector database is the storage layer of LLM.
This is where the pinecone comes into play.
Pinecone is an external database that allows developers to store relevant contextual data for LLM apps. Rather than sending and receiving large document collections with each API call, developers can store them in the Pinecone database and then select only the few that are most relevant for a given query. This is an approach called in-context learning. It’s a must-have for enterprise use cases to truly flourish.
Especially pine cones vector database. This means that the data is stored in a semantically meaningful format. embeddedA technical discussion of embeddings is beyond the scope of this post, but the important part to understand is that LLM also works with vector embeddings. So by storing data in Pinecone in this format, some of the AI work is effectively preprocessed and offloaded to the database.
Unlike existing databases designed for atomic transactional or exhaustive analytical workloads, the (Pinecone) vector database is a suitable database paradigm for high-dimensional vectors, an eventually consistent approximate neighbor search. Designed for It also provides APIs for developers to integrate with other key components of AI applications such as OpenAI, Cohere, and LangChain. Such a well-thought-out design makes the developer’s life much easier. Even simple AI tasks such as semantic search, product recommendations, and feed ranking can be directly modeled as vector search problems and run on vector databases without a final model inference step — What existing databases can’t do.
Pinecone is the emerging standard for managing state and contextual enterprise data in LLM applications. We believe this is a critical infrastructure component, providing a storage or “memory” layer for new AI application stacks.
Pinecone’s amazing progress to date
Pinecone is not the only vector database, but we believe it is the leading vector database in the industry. Pinecone has seen an eight-fold increase in paying customers (around 1,600) in just three months, including leading tech companies such as Shopify, Gong and Zapier. It is used across a wide range of industries including enterprise software, consumer apps, e-commerce, fintech, insurance, media and AI/ML.
This success has been attributed not only to the team’s deep understanding of users, markets, and technology, but importantly, to taking a cloud-native product approach from the start. One of the hardest parts of building this service is providing a highly reliable and available cloud his backend that meets a wide range of customer performance goals and his SLAs. Through multiple iterations of the product architecture and managing a large number of large paying customers in production, this team demonstrated the operational excellence expected of a production database.
pinecone was founded by Ed Liberty, a longtime advocate of the importance of vector databases in machine learning, including how any enterprise can build use cases on top of LLM. As an applied mathematician, he spent his career researching and implementing state-of-the-art vector search algorithms. At the same time, he’s also a pragmatist, building core ML tools like Sagemaker on AWS and translating applied ML research into practical products that customers can use. Deep research and practical product thinking are rarely combined like this.
Ed joins Bob Wiederhold, an experienced CEO and operator (formerly of Couchbase), as president and COO as an operational partner. Pinecone also has an amazing team of executives and engineers with deep expertise in cloud systems such as AWS, Google and Databricks. We have been impressed with the team’s deep engineering expertise, focus on developer experience, and efficient execution of his GTM, and partnered with them to build his layers of memory for AI applications. I am honored to be able to.
* * *
The views expressed herein are those of the individual AH Capital Management, LLC (“a16z”) cited and not those of a16z or its affiliates. Certain information contained herein has been obtained from third-party sources, including portfolio companies of funds managed by a16z. a16z has been obtained from sources believed to be reliable but has not independently verified such information and makes no representations as to the enduring accuracy of the information or its suitability for any particular situation. is not. Additionally, this content may include third party advertising. a16z has not reviewed such ads and does not endorse the advertising content contained therein.
This content is provided for informational purposes only and should not be relied upon as legal, business, investment or tax advice. Please consult your own advisors on these issues. Any reference to securities or digital assets is for illustrative purposes only and does not constitute an investment recommendation or the provision of investment advisory services. Further, this content is not intended for use by investors or prospective investors and should not be relied upon in making investment decisions in any fund managed by a16z under any circumstances. (An offer to invest in an a16z Fund shall be made solely by means of such Fund’s Private Placement Memorandum, Subscription Agreement, and other relevant documents, which should be read in its entirety.) does not represent all investments in vehicles managed by a16z and there is no assurance that the investment will be profitable or that other investments made in the future will have similar characteristics or results. A list of investments by funds managed by Andreessen Horowitz (excluding investments not authorized by the issuer to be published by a16z, and unannounced investments in listed digital assets) is available at https://a16z.com/investments. Available. /.
The charts and graphs provided inside are for informational purposes only and should not be relied upon in making investment decisions. Past performance is not indicative of future results. Content speaks only as of the indicated date. The projections, estimates, forecasts, goals, outlook and/or opinions expressed in these materials are subject to change without notice and may differ from or contradict opinions expressed by others. There is a possibility. See https://a16z.com/disclosures for other important information.
