Tee rollup enables 0.07 second blockchain inference for large language models

The growing demand for scalable and reliable artificial intelligence within decentralized networks faces a fundamental challenge, often referred to as the verifiability trilemma. This limits the ability to simultaneously ensure computational integrity, low latency, and low cost. DGrid AI's Aaron Chan, Alex Ding, and Frank Chen, along with Alan Wu, Bruce Zhang, and Arthur Tian, ​​introduce a new solution in the form of Optimistic TEE-Rollups, a hybrid verification protocol designed to overcome these limitations. In this work, we introduce a system that combines the speed of an optimistic approach with the security of cryptographic proofs, leveraging a trusted execution environment to provide near-instantaneous results while maintaining a high degree of reliability. By formally defining a new consensus mechanism, Proof of Efficient Attribution, and employing probabilistic zero-knowledge spot checks, the team demonstrated a system that achieves performance comparable to centralized systems and robust protection against malicious actors and hardware vulnerabilities, with minimal cost increase.

Optimistic TEE Rollup for Verifiable AI

Researchers have developed Optimistic TEE-Rollups (OTR), a new protocol that addresses the verifiability trilemma that hinders decentralized AI inference networks. The trilemma is that systems struggle to simultaneously achieve high computational consistency, low latency, and cost efficiency, and OTR can successfully reconcile these constraints. OTR combines the strengths of existing approaches by leveraging TEE for efficient off-chain computation, optimistic rollup for immediate interim finality, and crypto-fraud prevention and probabilistic zero-knowledge checks for enhanced security.

TEE performs AI inference and results are posted on-chain with the assumption that the calculations are correct unless challenged. A key innovation, Proof of Efficient Attribution (PoEA), cryptographically binds execution traces to hardware attestation, preventing reward hacking attacks that claim rewards for complex models while actually running cheap models. Experiments demonstrate that OTR can achieve 99% of the throughput of a centralized system with a marginal cost overhead of only $0.07 per query. Compared to zero-knowledge machine learning (ZKML), OTR achieves 1400x speedup and 99% reduction in latency as opposed to optimistic machine learning (opML).

The system maintains Byzantine fault tolerance even in the presence of temporary hardware vulnerabilities, ensuring robust and reliable operation against rational adversaries. This research establishes the foundational infrastructure to move decentralized AI from a theoretical concept to a practical, production-ready environment, enabling verifiable and censorship-resistant inference at scale. Future work will focus on improving the multi-prover consensus mechanism to further reduce dependence on single manufacturer enclaves.

Binding with reliable inference and TEE rollups

The research team addressed the limitations of existing distributed inference systems by developing Optimistic TEE-Rollups (OTR), a new protocol that provides high throughput, low cost, and strong security guarantees. The protocol works through a three-step process starting with reliable inference and binding. The sequencer performs inference within the TEE, and the Proof of Efficient Attribution (PoEA) cryptographically binds the execution to a specific model through enclave measurements.

To ensure privacy, users encrypt their input queries before sending them to the sequencer, which decrypts it and computes the response within a secure enclave. To protect against compromised hardware or side-channel attacks, OTR incorporates a probabilistic verification layer during the optimistic window. System-defined security parameters determine the probability of triggering a ZK spot check and require the sequencer to generate a concise proof of the computation. By carefully tuning this parameter, the scientists demonstrated that the system achieved 99% of the throughput of native execution while maintaining a credible threat to compromised hardware.

Solving the verifiability trilemma with optimistic rollups

Researchers have introduced Optimistic TEE-Rollups (OTR), a new architecture designed to address the verifiability trilemma that hinders distributed AI inference networks. The trilemma is that systems struggle to simultaneously achieve high computational consistency, low latency, and cost efficiency, and OTR can successfully reconcile these constraints. OTR combines the strengths of existing approaches by leveraging TEE for efficient off-chain computation, optimistic rollup for immediate interim finality, and crypto-fraud prevention and probabilistic zero-knowledge checks for enhanced security. Proof of Efficient Attribution (PoEA) cryptographically links execution traces to the hardware, preventing attacks that could degrade model performance. Furthermore, the system achieves 99% of the throughput of a centralized system with sub-second finality and a cost per query of $0.07, making it economically competitive with existing web-based services.