The field of artificial intelligence (AI) is advancing rapidly, making it difficult for experts and novices alike to keep up with the latest developments. To help with this, we have compiled a curated list of resources that have had a major impact on the field in recent years. We call it “AI Canon”. It includes papers, blog posts, courses, and guides to deepen your understanding of modern AI.
We begin with an introduction to the Transformers and potential diffusion models that are driving the current wave of AI. These resources will gently introduce you to these concepts and help you understand the basic ideas behind them. To understand the latest advances in AI, it is essential to understand transformers and latent diffusion models.
Next, we dive deep into technical learning resources and practical guides for building with Large Language Models (LLMs). Andrej Karpathy, a respected figure in AI, explains how AI represents a powerful new way to program computers. His insights in 2017 have proven incredibly insightful and continue to shape our understanding of the AI market.
The State of the GPT, also written by Karpathy, provides insight into how ChatGPT and the GPT model work, how to use them effectively, and what future research and development will require. Easily explained. Additionally, computer scientist and entrepreneur Stephen Wolfram dives into his modern AI models, outlining the evolution from early neural networks to his current LLM and ChatGPT.
For a better understanding of transformers, Dale Markowitz’s post provides a concise answer to the question of what an LLM is and how it works. This post will focus primarily on his GPT-3, but the information is applicable to newer models as well. Similarly, Chris McCormick’s explanation of how stable diffusion works provides valuable intuition on text-to-image models, especially in the field of computer vision.
To understand the latest AI key terms and technologies, we recommend reading the AI Glossary by a16z. This resource provides definitions of terms commonly used in the field and keeps you informed of the latest advances.
College-level courses are available for those who want a deeper understanding of the fundamentals of machine learning and AI. His CS229: Introduction to Machine Learning with Andrew Ng from Stanford University covers the fundamentals of machine learning, and CS224N: NLP with Deep Learning with Chris Manning focuses on natural language processing (NLP) and early generation of his LLM. I’m here.
Additionally, we’ve curated a selection of resources that explain how LLM works for a diverse audience. CS25: Transformers United, a Stanford University webinar, provides an in-depth look at Transformers, and CS324: Large Language Models provides insight into both technical and non-technical aspects of LLM.
For reference and commentary, I recommend considering Yann LeCun’s talk on predictive learning at NIPS 2016. It highlights the importance of unsupervised learning as a key aspect of large-scale AI models. Andrey Karpathy’s talk on Tesla’s AI for fully autonomous driving provides valuable insight into the challenges associated with the long-tail problem in the AI space. Additionally, Gwern’s post on the scaling hypothesis explains the concept that increasing data and computing increases the accuracy of his LLM.
For a comprehensive overview of current LLMs, including development schedules, scale, training strategies, and more, I recommend reading Large Scale Language Models Exploration. Sparks of Artificial General Intelligence: GPT-4 Early Experiments provides Microsoft Research’s preliminary analysis of the capabilities of the state-of-the-art LLM, GPT-4. It’s also important to familiarize yourself with AI agents such as Auto-GPT, as they represent a new era of automation and creativity.
As LLM becomes central to AI applications, we’ve gathered resources to help you understand the application stack. Formal education on this topic is still limited, but we encourage you to consider resources such as Building a GitHub Support Bot Using GPT3, LangChain, and Python. This early description of the modern LLM app stack kicked off widespread adoption and experimentation with new AI applications. Additionally, Chip Huyen’s discussion of building his LLM application for production addresses key challenges and recommends good use cases.
To enhance your prompt engineering skills when working with LLM, we recommend reviewing the prompt engineering guide, which provides comprehensive guidance and specific examples of common models. Brex’s Prompt Engineering Guide offers a lighter, more conversational approach to this topic.
In conclusion, AI Canon is a curated collection of resources designed to expand your knowledge of the latest AI. By exploring these resources, you can stay informed of the latest advances and gain a better understanding of this rapidly evolving field.
