The speed of innovation in applied AI is disrupting timelines for new fields, forcing practitioners to move from prototype to production almost overnight. For context engineering, the process of reliably feeding large language models (LLMs) with the necessary external information, 2025 felt like “six months compressed into a year.” This rapid evolution is fundamentally shifting the focus from optimizing individual components to establishing robust end-to-end system architectures that can operate at enterprise scale.
This was the core insight provided by Nina Lopatina, lead developer advocate for Contextual AI, who spoke live with Latent Space editor Swyx at NeurIPS 2025. Lopatina, whose background spans neuroscience and rewarded learning, highlighted the industry's struggle to transform contextual engineering from a collection of design patterns to a full-stack discipline with benchmarks and tools designed for real-world complexity.
The most direct change observed in this area is the elimination of basic search augmentation generation (RAG). Simple searches are no longer sufficient for complex enterprise queries. Lopatina acknowledged that “Agent RAGs are now the baseline. Restructuring queries into subqueries has significantly improved performance and is now the new standard (regular RAGs have been deprecated).” This change reflects the need for LLM to dynamically decompose a user's complex query into multiple targeted subqueries, retrieve diverse documents, and synthesize answers. This process requires a sophisticated control flow and a robust infrastructure.
But introducing agencies requires strict guardrails. The industry is rapidly recognizing that autonomous agents require explicit constraints to maintain reliability and performance at scale. During a recent Retail Universe hackathon, Lopatina's team worked with a dataset consisting of nearly 100,000 documents, including PDFs, CSVs, and log files. This is a real-world data environment far removed from the academic toy example. They found that subagents require defined turn limits and validation loops because “unrestricted agents degrade performance and cause hallucinations.” An agent's inherent urge to exhaustively search every possible avenue or continuously check its own work quickly becomes an anti-pattern when working with large production-scale datasets.
Scaling these systems presents challenges that researchers are only beginning to quantify. Although the problem of context corruption (where models ignore relevant information buried deep within long context windows) is widely recognized, concrete and actionable data are still lacking. Lopatina pointed out that “contextual corruption is cited on every blog, but industry benchmarks at real scale (100,000+ documents, billions of tokens) are still rare.” Recent research from Anthropic puts some hard numbers on this problem, showing that search rates drop to 30% when relevant context is placed in 700,000 tokens within a 1M window, finally making the problem quantifiable and forcing developers to be intentional about context placement and compression.
The advent of multi-component prompting (MCP) servers, which allow developers to register and discover tools via large JSON schemas, has become a double-edged sword. Although MCP servers accelerate rapid prototyping by abstracting tool management, they also contribute significantly to context bloat, as they consume valuable tokens just to describe available functionality. The long-term trend is optimization. Once your system design is validated, move away from redundant schemas and toward leaner, more direct API calls.
Rerankers that follow instructions are becoming an important component of high-performance pipelines. These small, specialized models are placed between the initial dense search phase and the final context window fed to the LLM, favoring high recall in the initial search while maintaining high precision in the final context window. This is especially important for dynamic agents reasoning over large databases.
An important optimization technique for multiturn agents is the strategic management of key-value (KV) caches. The decision-making framework is simple. Things that don't change (such as system prompts and early conversation turns) are placed at the top of the cache, while dynamically generated context (such as recent turns and tool output) is placed at the bottom. This approach stabilizes the agent's central identity and direction across turns while maintaining efficiency. After all, the model is not yet sophisticated enough to automatically compress, so intentional context compression is required, even if it means actively limiting conversational turns, as Lopatina does in her own development workflow.
Looking to the future, the discussion has moved beyond breakthroughs in individual components. According to Lopatina, the next frontier in 2026 is full system design, where “the discussion moves from 'How do we optimize the reranker?' to 'What does the end-to-end architecture look like for inferring billions of tokens in production?'” This systemic view, which includes multimodal ingestion, hybrid search, constrained agents, and strategic context management, marks the maturation of context engineering into a true field.
