General Matrix Multiplication (GEMM) forms the basis of modern scientific computing, but achieving the best performance in multicore systems remains a major challenge. Yufan Xia, Marco De La Pierre, and Amanda S. Barnard from the Australian National University, along with Giuseppe Maria Junior Barca, tackled this problem by developing a machine learning approach for runtime optimization. In their work, they introduced a proof-of-concept Architecture and Data-Structure Aware Linear Algebra (ADSALA) library that dynamically selects the optimal number of threads for a GEMM task. This innovative technique was tested on both Intel Cascade Lake and Zen 3 high-performance computing nodes and showed significant speedups of 25-40% compared to traditional BLAS implementations of GEMM operations using up to 100 MB of memory, demonstrating the potential for transformation towards more efficient scientific simulations.
The single-threaded GEMM implementation benefited from extensive optimizations through techniques such as blocking and automatic tuning to achieve significant performance improvements. However, determining the optimal number of threads to minimize the execution time of multithreaded GEMMs on modern multicore shared memory systems is a major challenge. In this study, we investigate strategies to efficiently parallelize GEMMs to maximize the throughput of these complex architectures. The goal of this work is to identify and implement techniques that effectively utilize the available cores while minimizing the overhead associated with thread management and data contention, ultimately improving the scalability and performance of GEMM.
Automatic BLAS tuning with machine learning
This paper presents an extensive and technically detailed study on automating the tuning of linear algebra algorithms, particularly BLAS routines, using machine learning, with the aim of overcoming the limitations of traditional manual optimization. Traditional tuning is time-consuming, hardware-specific, and requires considerable expertise, making it poorly suited for modern computing environments where architectures and workloads change rapidly. The authors propose a data-driven system that learns to predict optimal algorithm configurations, such as number of threads and blocking parameters, for a given problem size and hardware platform. This can reduce manual effort while improving performance.
At the heart of this approach is performance modeling, in which execution time is predicted as a function of problem characteristics, hardware capabilities, and algorithm parameters. This research considers a wide range of machine learning techniques, including lasso regression for feature selection, Bayesian regression for uncertainty-aware prediction, random forests and XGBoost for high-precision ensemble learning, support vector regression for high-dimensional feature spaces, nearest-neighbor methods for capturing local performance patterns, and density-based outlier detection for improving data quality. Emphasis is placed on feature engineering to generate useful training data using data normalization techniques, interaction terms, and carefully designed sampling strategies.
The workflow begins with systematic data collection by running the BLAS routine in different configurations and measuring its execution time. Then, relevant features are extracted from the problem size, hardware specifications, and algorithm parameters. Machine learning models are trained on this data, evaluated for predictive accuracy, and ultimately used at runtime to select the best configuration for new problem instances. Experiments are performed across multiple hardware platforms and BLAS operations such as matrix multiplication and LU decomposition using established libraries.
One of the main strengths of this approach is that it automates performance tuning while adapting to different systems and workloads. The authors demonstrate that their model can accurately predict the optimal number of threads for GEMM operations, often choosing fewer threads than the maximum available. This counterintuitive result leads to significant performance improvements due to reduced synchronization overhead and increased cache efficiency, in some cases reducing latency by up to 70% in real-world measurements. Importantly, these improvements are achieved even when using highly optimized libraries such as Intel MKL and AMD BLIS, highlighting the value of intelligent runtime configuration.
The paper also discusses potential limitations, including the cost of data collection, challenges in generalizing to unseen hardware, complexity of feature selection, and interpretability of complex machine learning models. The runtime overhead introduced by predictions and the need for periodic retraining as the system evolves are also considered. Despite these challenges, the authors show that benchmarking during installation can effectively tune the model to a specific system and ensure accurate and efficient predictions during execution.
Overall, this study demonstrates the great potential of machine learning to automate performance optimization in high-performance computing. By dynamically selecting the optimal configuration based on the learned performance model, the proposed system improves the efficiency of linear algebra operations on modern multicore architectures. This result suggests that systems with high core counts and large matrix sizes particularly benefit. Future directions include extending the approach to other BLAS routines, considering transfer and online learning strategies, and adapting the framework to heterogeneous architectures combining CPUs and accelerators.
