LangChain is a programming framework for using the Large Language Model (LLM) in your applications. As with all things in generative AI, things are moving incredibly fast on the project. It launched as a Python tool in October 2022 and added support for TypeScript in February. By April, it supported multiple JavaScript environments including Node.js, browsers, Cloudflare Workers, Vercel/Next.js, Deno, and Supabase Edge Functions.
So what does a JavaScript developer (among other things) need to know about LangChain, and indeed the operation of LLMs in general? We aim to answer that question by analyzing the presentation.
LangChain started as an open source project, but quickly turned into a startup when GitHub stars began to gather. It’s been a meteoric rise for Harrison Chase, who studied at Harvard until 2017, and is now the CEO of one of Silicon Valley’s hottest startups. Earlier this month, Microsoft’s chief technology officer, Kevin Scott, gave his Chase a personal shoutout during the Build keynote.
Chat apps are all the rage
Not surprisingly, LangChain’s primary use case today is building chat-based applications on top of LLM (especially ChatGPT). Tyler McGinnis of the popular bytes.dev newsletter quipped that LangChain “can’t have enough chat interfaces.”
In an interview with Charles Frye earlier this year, Chase said the best use case at the moment is “chat while looking at documents.” LangChain offers other features that enhance the app’s chat experience, such as streaming. In the context of LLM, this means returning the output of the LLM tokens token by token, rather than all at once.
However, Chase suggested that other interfaces will evolve rapidly.
“In the long run, there will probably be better UX than chat,” he said. “But at this point, I think you’ll quickly find that you can get up and running very easily without any extra effort. Can you expect chat to be the best UX in 6 months? Probably not, but I think that what we can build to provide value at the moment, that’s probably it.? [i.e. chat]”
Since application development with LLMs is so new, startups like LangChain are scrambling to develop tools to help solve some of the problems with LLMs. For his engineering prompts, for example, Chase said that it still often depends on the developer’s intuition as to which prompt works better. However, LangChain introduced features like “trace” this year to help with that.
Agent
One of the recent features of LangChain is “Custom Agent”. Chase spoke about this at his full stack LLM bootcamp in San Francisco in April. He defined an agent as a way of “using a language model as an inference engine” that determines how to interact with the outside world based on user input.
Harrison Chase at LLM Bootcamp.
He gave an example of interacting with a SQL database and explained that typically there is a natural language query and a language model translates it into a SQL query. By running that query and passing the results to a language model and asking it to synthesize a language model based on the original question, you’ve created what Chase calls a “natural language wrapper for a SQL database.”
This is where agents come into play, handling what Chase called “edge cases.” This could (for example) be her LLM hallucination portion of the output at any point in the example above.
“We also use the LLM, the agent who chooses which tool to use, and the input to that tool,” he explained. “Then you […] Taking that action yields an observation that feeds back into the language model. And so on until a stopping condition is met. ”
Agent implementation.
One popular approach to agents is called “ReAct.” It has nothing to do with the popular JavaScript framework of the same name. This version of “ReAct” stands for Reason + Act. Chase said the process produces “higher quality, more reliable results” than other forms of rapid engineering.
react (not react)
Chase acknowledged that there are “a lot of challenges” with agents and that “most agents are surprisingly production-ready at this point.”
memory problems
Some of the problems he enumerates seem like basic computer concepts, but are more difficult in the context of LLM. For example, LLMs typically do not have long-term memory. As Pinecone’s tutorial states, “By default, LLM is stateless, meaning each incoming query is processed independently of other interactions.”
This is one of the areas where LangChain aims to help developers by adding components like memory to the process dealing with LLM. In fact, in JavaScript and TypeScript, LangChain has two methods related to memory. loadMemoryVariables and saveContext. According to the documentation, the first method is “used to retrieve data from memory (optionally using the current input value), while the second method is used to store data in memory.”
Another form of agent Chase spoke about is Auto-GPT. This is a software program that allows you to configure and deploy autonomous AI agents.
“One of the things Auto-GPT introduced is the idea of long-term memory between agent-tool interactions, and using the Retriever vector store for that,” he said of the vector database.
New LAMP stack?
Clearly, there is still a lot to be worked out when building applications with LLM. During its Build keynote, Microsoft categorized LangChain as part of the “orchestration” layer of its “Copilot technology stack” for developers. In Microsoft’s system, orchestration involves prompt engineering and what we call “metaprompts.”
Microsoft has its own tool Semantic Kernel that does something similar to LangChain. We also announced a new tool called Prompt Flow. Microsoft CTO Kevin Scott says this is “another orchestration mechanism for him that actually integrates LangChain with the Semantic Kernel.”
It’s also worth noting the word “chain” in LangChain’s name. This shows that LangChain can interoperate with other tools (not only various LLMs, but also other development frameworks). Cloudflare announced LangChain support for its Workers framework in May.
Associated with LangChain, a new acronym has also been coined, OPL, which stands for OpenAI, Pinecone, and LangChain. Its inspiration was probably his LAMP stack (Linux, Apache, MySQL, PHP/Perl/Python) which was a significant part of the 1990s and led to the emergence of Web 2.0. Who knows if OPL will stick as a term. Of course, not all of its components are open source, but in any case it’s a good sign that LangChain is already a vital part of many developers’ personal stacks.
