JULY 14, 2023 — The Argonne Leadership Computing Facility (ALCF) is at the forefront of exascale technology, tackling the challenge of preparing for the arrival of the upcoming exascale system, Aurora. Murat Keçeli, a computational scientist at the U.S. Department of Energy’s Argonne National Laboratory, is focused on integrating Jupyter Notebook and Aurora to power high-performance computing research.
In this Q&A article, Ketchelli discusses his role, insights and experience in leading scientific computing towards new horizons of exascale performance. ALCF is a user facility of the U.S. Department of Energy, Office of Science.
ALCF: How long have you been working in HPC?
Murat Kecheri: My first job submission to the HPC cluster dates back to the summer of 2007 when I started working as a PhD student. It was his Beowulf cluster, Haku, with 32 nodes, mainly submitting independent jobs to each node. This is considered capacity computing. The first supercomputer I ran a job on was his Mira system at ALCF, dating back to 2014 when I started working as a postdoctoral researcher at Argonne University. Only then did I learn about running jobs in functional mode. This roughly means running a single supercomputer. Parallelize jobs on thousands of nodes using MPI. Note that I owe most of what I learned about supercomputers to ATPESC (Argonne Training Program on Extreme Scale Computing). This was truly a life changing experience.
Exascale development is mostly uncharted territory. What is it like to work on a project that enables the science of these powerful and unique systems?
It’s certainly very exciting, but there’s also a lot of pressure. Fundamentally, we have the potential to research problems that are orders of magnitude larger than we can tackle with our conventionally available resources. But turning that potential into a real capability accessible to other scientists is a challenge. Developing scientific software with the functionality, flexibility, and performance required for today’s most challenging applications takes years of effort. The Exascale Computing Project (ECP) gave us time to get our exascale machines ready when they turned on. However, we need to continue to support these projects to ensure they meet our users’ standards.
What are Jupyter Notebooks and how can they help with scientific research?
I see Jupyter Notebook as an interactive training, research and development environment. Jupyter Notebooks let you combine coding, documentation, and visualization into one document. Therefore, it is very useful for experimenting with new ideas, prototyping, performing data analysis, and creating tutorials. Initially supporting only Julia, Python, and R, Jupyter Notebook now supports over 40 programming languages. Additionally, cloud services such as Google Colab and GitHub Codespaces enable effective collaboration using Jupyter Notebooks. All these features make Jupyter Notebook the most popular platform for data scientists and there is an ongoing effort to use his Jupyter Notebook for reproducible research and publications. I would also like to highlight their role in education and training. They are now an integral part of academic courses and workshops that provide interactive tutorials for students.
How is Jupyter Notebook used with Aurora? Is using Jupyter Notebook on an exascale system different from using it on a traditional CPU-based system?
The shift from CPU-based to GPU-based systems parallels the shift from traditional simulation-based workflows to artificial intelligence/machine learning (AI/ML)-based workflows. This represents a significant change in HPC user profiles. Jupyter Notebooks are more popular among data science users, so it’s only natural to expect more usage of his Jupyter Notebooks with Aurora. We believe ALCF JupyterHub will provide a more user-friendly gateway to Aurora (compared to traditional terminal-based access), allowing users to submit and monitor jobs and analyze data on the fly. increase.
What are the challenges you face in preparing your Jupyter Notebooks for Aurora deployments? What are your strategies for making Jupyter exascale ready?
I think the main challenge in deploying state-of-the-art supercomputers is user experience. Although these systems are very functional in abstraction, efficient utilization of these resources is a major challenge, especially for first-time users. The purpose of Jupyter Notebook is to lower the barrier of entry for new users and provide a smoother experience. Leverage interactive features such as Jupyter widgets to prepare self-contained tutorials to help users get started. Various materials in the ALCF workshop are based on Jupyter Notebooks.
Who do you work with as part of your work at ALCF?
I started working at Argonne University as a Postdoctoral Fellow in the Chemical Mechanics Group of the Department of Chemical Science and Engineering (CSE). I have worked on several projects involving scientists from CSE, Materials Science, Mathematics and Computer Science, and ALCF. I will continue to work with these colleagues on related projects and proposals. In 2018, I joined his ALCF data science team and computational science department. ALCF workshops and various computer assignment programs have allowed us to connect with researchers around the world. Just in April, we published a nanoscale paper on exploiting the potential of machine learning with Belgian and Turkish scientists, building on a collaboration that began with his previous ALCF workshop. For the last three years I have mainly worked on his NWChemEx ECP project. This is a very large project involving six national laboratories (Ames, Argonne, Brookhaven, Lawrence Berkeley, Oak Ridge, Pacific Northwest National Laboratory) and Virginia Tech. I would also like to mention the Sustainable Research Pathways program at the Sustainable Horizon Institute, which I participated in last year. This talent development program has allowed us to connect with researchers in an underrepresented community, and this year we are hosting three of her summer students.
How have approaches to preparing for exascale evolved over time?
Initially, I was primarily concerned with strong scaling and performance optimization. This is a very important area in HPC, but over time my interest shifted to workflow development and user experience. This is an area that has become more important as interest in projects aimed at combining AI/ML and traditional simulation grows.
Argonne Leadership Computing Facility Offers super compHarness the power of the scientific and engineering community to advance fundamental discovery and understanding in a wide range of fields. Supported by the U.S. Department of Energy (DOE) Office of Science’s Advanced Scientific Computing Research (ASCR) Program, ALCF is one of two DOE Leadership Computing Facilities in the country dedicated to open science.
Argonne National Laboratory Seek solutions to pressing national problems in science and technology. As the nation’s first national laboratory, Argonne conducts cutting-edge basic and applied scientific research in virtually every scientific field. Argonne researchers work closely with researchers from hundreds of companies, universities, and federal, state, and local agencies to solve specific problems, advance America’s scientific leadership, and make the nation better. We help you prepare for a better future. With employees from over 60 countries, Argonne is managed by his UChicago Argonne, LLC for the U.S. Department of Energy’s Office of Science.
Source: Nils Heinonen, ALCF
