As machine learning systems continue to expand, the gap between ability and understanding is widening. Large-scale language models can now perform tasks that once seemed out of reach, but the internal logic that guides their output often remains obscure. For researchers who value safety, accuracy, and long-term reliability, this opacity is not a philosophical inconvenience. It’s a structural risk. Neil Somani approaches this problem from a discipline that predates modern machine learning itself: formal methods.
Formal methods include mathematically grounded methods such as program verification, proof generation, and symbolic reasoning. They have long served as the backbone of security, privacy, and compiler correctness. Somani’s research focuses on bringing these tools to machine learning, but safety and interpretability still lack rigorous standards and are primarily driven by empirical testing and informal explanations.
First principles and the limits of empiricism
Somani’s research philosophy begins with first principles. Although it is possible to check the correctness of a large number of inputs to a function, no amount of testing is sufficient to establish continuous domain guarantees. Modern large-scale language models (transformers) operate on continuous input, so in an ideal world there would be a powerful alternative to testing.
During her undergraduate studies at the University of California, Berkeley, Somani pursued a triple major in computer science, mathematics, and business administration. His academic training taught him about type systems, differential privacy, and formal verification. While working in Berkeley’s research environment, he contributed to a project to formally prove whether a particular algorithm satisfies privacy guarantees based on precise mathematical definitions.
These experiences revealed a consistent pattern. Although many properties of interest in machine learning can be formally defined, it is not yet possible to verify them at scale. Rather than abandoning formalism, Somani has focused his research on applying these techniques where they are currently viable, while laying the groundwork for future expansion.
Security as a precedent for AI safety
There are reasons why formal methods have gained attention in security. Empirical success alone is not enough when the system deals with sensitive data or critical infrastructure. The system must prove that it works correctly within the defined constraints. Somani argues that machine learning systems are increasingly taking on similar roles.
Currently, research into AI safety and interpretability remains fragmented. Researchers propose explanations for the model’s behavior, but these explanations are often impossible to falsify. Claims about robustness, consistency, or internal reasoning are often supported by anecdotes rather than evidence. Somani describes the field as preparatory and paradigmatic, lacking common ground on what constitutes a verified claim.
His work is simple but asks tough questions. If we expect strong guarantees from our cryptographic systems, why should we accept weaker standards for AI systems that impact financial markets, medical decision-making, and automated infrastructure?
Quantitative research and high-stakes reasoning
After graduating from Berkeley, Mr. Somani joined Citadel’s products group as a quantitative researcher. There he worked on optimization systems that directly impact real-world markets. Many of these systems involve difficult NP problems solved by mixed-integer linear programming, where small modeling errors can produce large consequences.
This environment reinforced important lessons. In high-stakes systems, precision is more important than elegance. The model must work reliably not only under average conditions, but also under special circumstances. This perspective will continue to influence Somani’s research in machine learning. As AI systems are placed with greater responsibility, expectations for verification and accountability should rise accordingly.
Apply formal methods when it matters
Rather than trying to examine the entire neural network, Somani focuses on specific components where formal methods can provide immediate value. One example is his project Cuq, which applies formal verification techniques to GPU kernels written in Rust.
GPU code is notoriously difficult to infer. Unlike many high-level programming environments, GPUs offer limited protection against memory errors and undefined behavior. Small mistakes in indexing or synchronization can lead to subtle failures that evade standard tests. Cuq uses formal verification to prove the correctness properties of GPU kernels, demonstrating that they can reduce hidden risks in performance-critical systems.
This study challenges the assumption that formal methods are too theoretical to be useful in modern machine learning pipelines. Rather, it shows that targeted applications can significantly improve reliability today.
provable interpretability
Interpretability remains one of the most debated areas in AI research. Large language models are made up of layered transformations that preclude simple explanation. Researchers often rely on visualizations and conceptual metaphors to explain what a model is doing under the hood.
Somani’s project Symbolic Circuit Distillation takes a different approach. Based on research on mechanistic interpretability, this project extracts a simplified circuit from a model and generates a human-readable program that is proven to be equivalent to that circuit on a defined input space.
This distinction is important. Rather than providing a plausible story about the model’s behavior, this method allows researchers to formally prove whether an explanation is correct or not. Although this technique currently applies only in limited cases, it establishes a criterion for interpretability that is based on equivalence rather than intuition.
From explanation to decompilation
Somani describes his long-term research vision as an attempt to decompile transformer-based models. The goal is to transform a trained model whose internal workings are opaque into a human-readable program that directly captures its functionality.
This is an ambitious direction, and Somani is careful not to overestimate its feasibility. He declares that his research questions may evolve as technical constraints and community interests become more clear. This warning reflects a broader commitment to intellectual integrity. His research focuses on establishing what seems achievable, rather than prematurely promising breakthroughs.
Successful decompilation would fundamentally change the way we understand interpretability. Rather than asking whether humans can intuitively grasp a model, researchers will ask whether its behavior can be expressed in a formal language with defined semantics.
Infrastructure, inference, and real-world impact
Somani’s research is not limited to theory. He also contributed to practical optimization problems in machine learning infrastructure. His KV Marketplace project looks at improving inference efficiency by optimizing GPU caches and reducing redundant computations.
Although inference frameworks such as vLLM and SGLang have standardized many optimization techniques, opportunities for additional benefits remain. By directly modifying the inference engine, Somani demonstrates that by working carefully at the system level, you can achieve measurable improvements without changing the model’s output.
This balance between theory and application is a hallmark of his research. Formal methods are not treated as abstract ideals, but as tools that ultimately need to be integrated with real systems.
Rethinking architecture for verification
Looking to the future, Somani suggests that formal methods could influence the design of future machine learning architectures. Certain components, such as attention mechanisms and normalization layers, are difficult to reason about within existing verification frameworks.
Future models may evolve to be more verifiable, rather than forcing formal tools to accommodate every architectural choice. This reflects the historical development in software engineering that once a system is stable based on well-defined abstractions, formal verification becomes possible.
Somani is similar to efforts like CompCert, which formally verified the correctness of a compiler after its compilation pipeline had matured. He believes a similar trajectory may emerge in machine learning as the fundamental components become standardized.
A cautious approach to responsible AI
Throughout her work, Somani emphasizes correctness over rhetoric. He avoids unsubstantiated claims and prioritizes technical accuracy even when limiting scope. In a field driven by rapid iteration and public attention, this constraint is increasingly rare.
His work provides a defensible framework for thinking about AI safety for hiring managers, research directors, and policy makers. Formal methods may not yet provide comprehensive guarantees for large-scale models, but they do provide principled direction based on evidence rather than speculation.
As AI systems continue to shape critical decisions, the demand for verifiable assurance will only increase. Neel Somani’s research shows that while full validation remains a long-term challenge, meaningful progress is already possible by applying rigorous tools to the most critical parts of the system.
