Ask any developer and they will tell you how much it costs to run and build a GPT model. The main reason behind this is our reliance on legacy or obsolete GPUs, leading to pressing problems. “Today, machine learning hardware is being abused,” said Jack Dongarra, a respected American computer scientist and Turing Award winner.
Dongara also emphasized the need for a better understanding of how to effectively utilize these resources. He advocated identifying suitable applications and environments in which they can be used optimally. He also stressed the importance of developing software frameworks that efficiently drive computation and support hardware functions with each other.
So what would be the solution? Looking to the future, Dongara envisions a multi-dimensional approach to computing in which various technologies such as CPUs, GPUs, machine learning, neuromorphic, optics, and quantum computing are aggregated within high-performance computers. I’m here.
“Quantum is just around the corner. So each of these devices can be considered part of the spectrum to build into high-performance computers. Interesting devices that we’ll build. It will be even more difficult to understand how to use it effectively,” said the 72-year-old academic.
capital vs innovation
Drawing parallels between capital and innovation, Dongarra said: “Amazon has its own hardware resource, Graviton. Google has its TPU. Microsoft deploys its own hardware in the cloud. It has an incredible richness to invest in the problem, which is quite unlike what we experience in high performance computing where we don’t have that luxury. We don’t have a lot of resources to invest in hardware to solve the problem.”
While comparing Apple’s technological prowess with traditional computer companies such as HP and IBM, Dongara pointed out the big difference in market capitalization. While Apple’s value skyrockets into trillions of dollars, HP and IBM’s combined market capitalizations fall short of $1 trillion. “We have a profitable cloud-based company that allows us to build our own hardware that is innovative and unique. They’re replacing software with hardware because they’re doing it because it’s faster, and that’s very different from the general high-performance computing community,” he said. rice field.
Supercomputers, on the other hand, are mostly built using off-the-shelf components from companies like Intel and AMD. GPUs are often added to these processors, and the entire system is connected through technologies such as his InfiniBand and Ethernet. Dongara explained that the scientific community that relies on these supercomputers faces financial constraints that limit its ability to invest in specialized hardware development.
IBM is at the forefront of quantum, but at Think last week, CEO Arvind Krishna unveiled a vision highlighting the potential of combining quantum computing with hybrid cloud technologies and AI over the next decade. . The company might be thought of as a traditional company focused on hardware, but recent breakthroughs in generative AI suggest otherwise.
The story behind LINPACK
“I wanted to be a high school teacher,” Dongara said. Computer science was still in its infancy when the polymath studied. His Dongarra passion for numerical methods and software development flared up during his stint at Argonne National Laboratory near Chicago. He worked on developing software for numerical computation, and his experience solidified his interest in this area. This pivotal moment enabled him to complete his master’s degree while working full-time on designing his package of world-recognized portable linear algebra software.
Soon after, Dongara embarked on a journey to the University of New Mexico. During this period, he made the groundbreaking contribution of creating his LINPACK benchmark, which measures the performance of supercomputers. Joined by Hans Muir, he established his iconic Top 500 list in 1993, High Performance Tracking his computing progress, shedding light on the fastest computers in the world.
Today, at the University of Tennessee, Dongara continues to shape the future of computing through ongoing research exploring the convergence of supercomputing and AI. As AI and machine learning continue to gain momentum, his research focuses on optimizing the performance of algorithms on high-performance systems to enable faster and more accurate computations.
2001: A Space Odyssey Fans
In the 1980s, a prominent figure in the field of computing, where innovation reigns supreme, was Jack Dongarra. “Almost everywhere we see computational people looking for another way to solve a problem, and they see AI as a means to that end. But AI does not solve the problem. It will help them in solving their problems,” said the 2021 ACM Turing Award winner. He was awarded for his contribution to ensuring that high-performance computing software stays in sync with advances in hardware technology.
“AI has become really popular these days for a number of reasons. One is the sheer amount of data that exists on the internet today. Resources that can be mined to help with the training process. We have processors that can do calculations very fast, so we have computing devices that are optimized and can be used very effectively to help us train,” he said. He said while emphasizing evolving technology.
Dongarra also emphasized the important role of linear algebra in AI algorithms, emphasizing the importance of efficient matrix multiplication and steepest descent algorithms. “Many things have been put in place to make AI machine learning practical and a very useful resource. It really has a big impact on many areas of science, such as academic cosmology,” he added.
When it comes to artificial general intelligence (AGI), Dongara sees the need to develop machines that automate mundane tasks and assist in scientific simulations and modeling. However, he stresses caution, as the vast amount of unfiltered information available on the web can be misleading.
“2001: A Space Odyssey is a relatable movie in that it focuses on a computer-powered AI that probably goes a little off-kilter, taking over missions and doing damage along the way. I first saw it when it came out. I found it fascinating when I was young and still enjoy this story.It contains a lot of things that are relevant today,” he concluded.
