The efficiency of deep recommendation models, the core component of modern artificial intelligence systems, has a major impact on the performance of data-intensive applications. These models are responsible for personalized proposals in areas such as e-commerce and social media, and currently account for a significant proportion of AI workloads in large data centers. Along with Giuseppe Ruggeri, Renzo Andri, and colleagues at Huawei Technologies, Daniele Jahier Pagliari of Politecnico Di Torino and Lukas Cavigelli, we permeate the study presentation detailed in the article, “Deep Recommended Model Inference: Auto-Asymmetric Data Flowing.” We propose a framework for automatically mapping data flows across multi-core systems.
Deep Recommended Models (DLRMS) constitute a significant portion of Meta's artificial intelligence workloads, but performance frequently suffers from latency within the embedded layer. Researchers address this important bottleneck by proposing tailored data flows designed to accelerate embedded lookups and improving efficiency, focusing on access patterns and data organization optimization. The embedding layer converts categorical data such as user IDs and product categories into numeric vectors, allowing machine learning algorithms to process this information. Importantly, the methodology incorporates frameworks (SOCs) that automatically and asymmetrically map these tables to multiple cores within a system on a chip, enhancing data locality and reducing competition. The on-chip system integrates all the components of the computer into a single integrated circuit.
The core of the approach includes four different strategies implemented in a single core, enhancing embedded table access and providing substantial performance improvements. These strategies work in concert to minimize latency, maximize throughput, and enable faster, more responsive recommendation systems. The experiment utilizes Huawei's Ascend AI Accelerators to assess the effectiveness of the proposed method, compares its performance with both the default Ascend compiler and NVIDIA's A100 processor, and shows significant improvements across various benchmarks.
The results show a significant reduction in latency, achieving a speedup of 1.5 to 6.5 times when handling the actual workload distribution, indicating the practical benefits of optimization. In particular, this method is applied to highly unbalanced distributions and shows even greater improvement (20x) when highlighting its robustness and adaptability. An imbalanced distribution refers to a scenario in which some categories in the embedded table are much more frequent than others.
The evaluation includes a diverse and large dataset, including Huawei-25MB, Criteo-1TB, Avazu-Ctr, Kuairec-Big, Taobao, and Tenrec-QB-ART. These datasets facilitate a comprehensive assessment of the scalability and generalizability of methods across a variety of recommended system scenarios, and confirm their wide applicability.
Asymmetric shards accelerate the performance of deep recommended models' inference and address the fundamental challenges of modern AI infrastructure. This study identifies table access as a major restriction, particularly to create important bottlenecks, where the random memory access required is required to retrieve vectors from tables of different sizes. As a result, researchers propose an asymmetric sharding strategy, intelligently distributing these tables across multiple cores of systems on the chip, optimizing resource utilization and minimizing communication overhead. Shards involve splitting large tables into smaller, more manageable parts.
Co-innovation is in a hybrid approach, tuning data flows to optimize embedding lookups and combining replication and sharding techniques. Small embedded tables are replicated across all cores, minimizing communication overhead and allowing local access, while larger tables undergo shards, distribute load and reduce contention. This asymmetric allocation, informed by table size, clearly improves performance compared to a uniform distribution strategy, and provides significant benefits in efficiency.
The experiment utilized Huawei's Ascend AI Accelerators to demonstrate significant improvements across various benchmarks. This study highlights the resilience of the method to variation in query distribution, showing superior performance consistency compared to baseline, ensuring reliable performance under a variety of conditions.
Further work can explore the optimization of a variety of hardware platforms, extending the applicability of research beyond the Ascend architecture. Investigating the application of these principles to other machine learning models that rely on large embedded tables can broaden the impact of this study and its benefits may extend to a wider range of applications.
