Offline reinforcement learning uses free fermionic subroutines to compile Hamiltonian simulation circuits and stabilize value learning

Quantum simulation holds promise for breakthroughs in materials science and drug discovery, but creating the necessary quantum circuits remains a major hurdle, often limited by the number of operations and the time it takes to perform them. Ethan Decker, Christopher Watson, Runyu Chou and colleagues at the University of Pennsylvania and Pacific Northwest National Laboratory have introduced F2, a new framework that dramatically improves the compilation of these circuits using a technique called offline reinforcement learning. F2 focuses on exploiting the underlying mathematical structure of quantum systems, particularly the properties of free fermions, to design more efficient circuits to simulate how these systems evolve over time. For a variety of difficult problems, including modeling complex materials and protein fragments, F2 reduces the number of quantum operations by 47% and circuit execution time by 38% while maintaining high accuracy compared to existing methods, demonstrating a powerful new approach to scalable quantum computing.

Quantum circuit optimization for NISQ devices

Quantum computing research is increasingly focused on optimizing quantum circuits. This is a key step toward running the algorithm on short-term quantum hardware known as NISQ devices, which have limited resources. The main theme is to reduce circuit complexity, especially gate count and circuit depth, while preserving functionality. This optimization is essential for tackling complex problems in fields such as quantum chemistry and materials science. Scientists are also actively simulating quantum systems to understand material properties and molecular behavior. There is a growing trend to leverage reinforcement learning (RL) to automate and improve quantum circuit design, wiring, and optimization.

At the same time, researchers will address the effects of errors on noisy quantum hardware through error mitigation and correction techniques. Current research focuses on developing techniques to optimize quantum circuits, from general optimization strategies to strategies that leverage RL to learn optimal circuit transformations. Specific methods include topological quantum compilation and circuit decomposition methods. Several frameworks and compilers, such as t|ket⟩, Pcoast, and Qiskit, have been developed to aid in the design and optimization of quantum circuits. This research is driven by the need to commercialize quantum algorithms on NISQ devices, focusing on techniques that minimize resource requirements and improve performance. Quantum chemistry and materials science provide concrete problems that motivate the development of new algorithms and techniques.

Offline reinforcement learning for quantum compilation

Scientists have developed F2, an offline reinforcement learning framework that efficiently compiles quantum circuits for Hamiltonian simulations. This overcomes the limitations of existing methods that rely on manually designed classical heuristics. The researchers built a reinforcement learning environment focused on classically simulable free fermion subroutines. This is an important innovation that reduces the exponential complexity typically associated with quantum system representation and circuit optimization. To stabilize learning in this complex environment, the team developed a compositional action encoder combined with a learned inductive bias within the critique goal. This allows for more reliable value estimation, tackling the challenge of large hybrid discrete-continuous action spaces.

Additionally, scientists exploited the time reversibility of quantum circuits to generate a rich set of synthetic trajectories, creating a dataset of transitions with guaranteed success for offline reinforcement learning. The experiments used benchmarks spanning lattice models, protein fragments, and crystalline materials, including systems from 12 to 222 qubits. This study demonstrates that compared to powerful baseline compilers such as Qiskit and Cirq/OpenFermion, F2 achieves a 47% reduction in gate count and 38% reduction in circuit depth while maintaining an average error of only 10 -7. Coordinating deep reinforcement learning with the algebraic structure of quantum mechanics greatly enhances quantum circuit synthesis and provides a promising path toward scalable learning-based quantum compilation.

F2 framework greatly simplifies quantum simulations

Researchers have achieved significant improvements in quantum circuit synthesis using a new framework designed for Hamiltonian simulation called F2. F2 exploits the algebraic structure of free fermion systems and can significantly reduce both the number of gates and circuit depth. Experiments demonstrate that F2 reduces the total gate count by an average of 47% across benchmarks ranging from 12 to 222 qubits, including lattice models, protein fragments, and crystalline materials, while achieving a 38% reduction in circuit depth. Importantly, these optimizations are achieved while maintaining a significantly low average error of 10 -7, demonstrating the accuracy of the method.

At the core of F2 is a reinforcement learning environment specialized for classically simulable free fermion subroutines, reducing the exponential complexity typically associated with quantum system representations. Additionally, the team developed a synthetic trajectory generation mechanism that consistently generates rich data with guaranteed success for offline reinforcement learning. These advances position F2 as a promising direction for scalable learning-based quantum compilation, potentially enabling more complex simulations on future quantum hardware.

Optimized quantum circuits using reinforcement learning

This work demonstrates a significant advance in the compilation of quantum circuits for Hamiltonian simulations, achieving significant reductions in both gate count and circuit depth. By deploying an offline reinforcement learning framework, the researchers were able to optimize the circuit across a variety of benchmarks, including lattice models, protein fragments, and crystalline materials. The new method reduces gate count by 47% and circuit depth by 38% on average compared to existing approaches, while maintaining a high level of accuracy with an error of around 10^(-7). The key to this improvement lies in aligning deep reinforcement learning with the underlying algebraic structure of quantum mechanics, specifically exploiting free fermion structures in circuits. This approach enables efficient optimization through a reinforcement learning environment and a reversible synthetic trajectory generation mechanism that provides rich training data. While the authors acknowledge that the method currently focuses on classically tractable subroutines, they suggest several promising avenues for future research, including extending the paradigm to other efficiently tractable routines and exploring the incorporation of continuous parameters into the model.