A technical paper entitled “Optimizing Memory Mapping Using Deep Reinforcement Learning” has been published by Google DeepMind and Google.
overview:
“Resource scheduling and allocation are key components of many high-impact systems, from congestion control to cloud computing. It can save time, reduce wear and tear on your device, and even improve your carbon footprint. That is, we focus on the problem of mapping tensors to different memory layers to optimize execution time.
We present an approach to solving memory mapping problems using reinforcement learning. RL is a solution paradigm suitable for sequential decision-making problems suitable for planning and combinatorial search spaces with high-dimensional data inputs. We formulate the problem as a single-player game called mallocGame such that the game’s high-reward trajectory corresponds to efficient memory mapping on the target hardware. We also introduce the reinforcement learning agent mallocMuZero and show that you can play the game to discover new and improved memory mapping solutions that lead to faster execution times for real ML workloads on ML accelerators. We benchmark the performance of mallocMuZero against the default solver used by the accelerated linear algebra (XLA) compiler in a benchmark of a realistic ML workload. Furthermore, we show that mallocMuZero can improve the runtime of the recently published AlphaTensor matrix multiplication model. ”
View technical documentation here. Published May 2023 (preprint).
Wang, Penmin, Mikita Sazanovich, Birkin Ilvey, Pichaya Manpo Fotirimtana, Manish Prohit, Han Yang Tai, Gan Vu et al. “Optimizing Memory Mapping Using Deep Reinforcement Learning”. arXiv preprint arXiv:2305.07440 (2023).
