An introduction to the AI ​​stack for JavaScript

It’s hard to understand without using new software infrastructure technologies. At least that’s what the a16z infrastructure team discovered. And since many of us started our careers as programmers, we often learn by doing. This is especially true for waves of generative AI. The wave of generative AI is arriving very fast and spectacularly, and good documentation is often months behind code. Therefore, we have also built projects around large scale language models (LLMs), large scale image models, vector databases, etc. in order to gain a deeper understanding of this area.

In doing so, I realized that all of this is so new and changing so quickly that there really isn’t a good framework to start with. All projects require a large amount of boilerplate code and integration. Frankly, it’s a pain. So I decided to create something very simple. “Get started with AI” Remplate For those who want to try out the core technology but don’t have to think too much about the long tail of side concerns like authentication, hosting, and tool selection.

You can fork the template and deploy here. We would love to hear your thoughts and feedback so that we can further improve our templates.

component

Many of us are JavaScript/TypeScript enthusiasts, so we chose JavaS.Use the cript stack as a starting point. However, the framework can easily be modified to support other languages, and we plan to do so in the near future.

Here’s an overview of the introductory stack we put together with our longtime collaborators and open source enthusiasts. Tim Chian. The goal is to highlight the simplest path from pulling code on GitHub to a running generative AI app (both images and text). It is designed to be easily extensible for more sophisticated architectures and projects.

For a detailed overview of the new LLM stack, see the post titled .New architecture for LLM applications”.

model and inference

Hosting a model is a hassle and largely an issue not directly related to building AI apps. So we built it using OpenAI for text and Replicate for image inference. Replicate also offers text-based models (see how easy it is to do) Vicuna) so it can be used instead of OpenAI if desired.

certification

Starter frameworks usually don’t bother to include authentication. But in this case, the model is so powerful and versatile that it has been the subject of a large and coordinated effort to make it available for free. Developers often find this out the hard way when they suddenly receive a $10,000 bill from a model provider. That’s why we chose to include Clerk, which does the heavy lifting around bot detection. Of course, when it comes to building more sophisticated apps, we offer full authentication support.

vector database

LLM needs long-term hardened memory to save state and avoid context windows. This is handled by the vector database. Currently, Pinecone is the most mature and popular vector store among the generative AI crowd. That said, we want to provide support for all use cases and environment setups, so we’ve also included support for: page vector From the repository Supabase.

introduction

We use Fly.io for deployment. This is because it is multi-regional, easy to manage, and provides a very general computing environment (anything that runs inside a container). Over time, many AI projects will end up using multiple languages ​​or adding significant functionality to their backend. As such, Fly.io is a great compromise between JavaScript-native hosting environments such as Vercel and Netlify and the traditional cloud. That being said, the code easily supports other hosting environments if needed. Fly.io coming soon Providing GPUs If you want to host your own model.

Roadmap

We think this first iteration is a good starting point, but we’re in the process of adding more options and fleshing out the stack. Here’s an overview of our roadmap:

• Interactive CLI for create-ai-stack. Developers can choose their own project scaffold and dependencies.
• Transactional database for advanced use cases (e.g. persisting questions in Q&A, user preferences, etc.)
• Add vector database and deployment platform options
• Lightweight fine-tuning steps for open source models

For bug fixes, feature requests, and feedback, please open a PR. We are excited to contribute to the open source community and believe the ecosystem always wins.

We would like to thank the following open source projects that contributed to the creation of the AI ​​Getting Started stack: These essential tools have made complex tasks simple and manageable. Thanks to their tireless efforts and valuable contributions, the JavaScript ecosystem continues to thrive and thrive.