The tech world is flooded with generative AI, which is a good thing for companies like Nvidia. The company that invested in AI as a growth engine a decade ago has spent the intervening years putting pieces of hardware and software and partnerships together to prepare that future when it arrives.
The recent GTC 2023 show was all about what Nvidia is doing with AI, from new products to strategy and roadmaps. The message continues to spread widely that Nvidia has a lot in store to help companies introduce AI into their business. That included earlier this week when he was the keynote speaker for Nvidia’s VP of Enterprise Computing, Manuvir Das, at the MIT Future Compute event in Cambridge, Massachusetts.
Das came to the event to include that companies should board the generative AI train through widely used technologies such as GPT-4, ChatGPT, and OpenAI, a startup that is a partner of Nvidia, which owns Dall-E. , conveyed some messages. , and other products are rapidly being integrated across Microsoft’s broader portfolio to train large language models (LLMs) using their own data, either by other vendors or through in-house efforts. adopted by
Another message was about evolving computing environments and AI. Much of the world’s work is done through computing. That is, more energy is consumed. According to Das, about 1-2% of global energy consumption is only consumed by his use cases such as data centers and compute-heavy generative AI, and that will only increase. Growth rates are unsustainable and computing needs become more efficient. Not surprisingly, Nvidia believes that running as many workloads as possible, not just those based on AI and machine learning, on GPU-accelerated systems is a key step. Over the position that Moore’s Law once occupied driving innovation in computing.
“The message I wanted to get across to this audience was, since the topic was the future of computing, ‘Yes, it’s a good thing because new use cases like generative AI are built on accelerated computing. ‘That’s it,’ said Das. next platform“But in order to find space for these new workloads and prevent the world from having to go through this massive expansion of computing, the domain is doing everything we already do in the data center. We’re going to have to take each and basically move everything, because this accelerated computing will get 10x the output from the same data center that we have today. ”
Nvidia has been busy in recent months bringing products to market aimed at the burgeoning AI space, building what Das calls a full-stack approach to the market. This includes the H100 “Hopper” H100 GPU accelerator. It has found a home in places such as Microsoft Azure, Oracle Cloud Infrastructure, AWS, and Meta’s “Grand Teton” AI node. This is what the company uses internally for its own AI work.
In addition, the DGX H100 AI supercomputer with Nvidia’s enterprise AI software suite is in production, the H100 NVL essentially connects two H100 GPUs in one form factor and can be used for AI training and inference. server. His OEMs, including Dell and his HPE, are developing systems that support the H100 NVL double-width PCI-Express H100, Das said.
“The same system is optimized to be most efficient in training. [but] The most efficient use of the model requires slightly different characteristics of the system,” he says. “There are a lot of companies in the world that don’t train models. I hope.”
At GTC, the company also unveiled technologies such as the NeMo guardrail. This prevents AI chatbots from hoaxing (creating “hallucinations”) or simply misinterpreting information and going off course.
Addressing trust issues like this is critical as companies look to go all-in on LLMs, from OpenAI’s products to Meta Platforms’ LLaMa. Despite the rush to bring generative AI tools to market, there are still questions around data security, privacy, and compliance that must be addressed in an amorphous environment. Some countries question LLMs such as ChatGPT. Italy recently banned it, but this week said it reversed that decision. Samsung also said it has banned employees from using his ChatGPT and similar products due to the risks posed by generative AI. Some business owners worry that applications developed using generative AI will inadvertently expose sensitive data and trade secrets.
The federal government and people in other countries have started to clamor about the need for regulation and oversight of the rapidly evolving AI market. As we talked about this week, the AI market could continue to change as open source players start to market his LLM code. Vice President Kamala Harris met with the CEOs of Microsoft, OpenAI, Google and Anthropic on Thursday as she talked about AI.
Despite all of this, businesses are realizing they need to adopt generative AI to stay competitive, Das said. He said some companies such as Meta, Google and Microsoft were early adopters of AI. However, the “bread and butter enterprise customers” “did the R&D work. , enterprise companies worry, maybe what they’ve been waiting for was a compelling event, and what’s happening now with generative AI is that we’ve found a compelling event. I knew I had to do it, because if I didn’t do it, I would be left behind.”
Companies that thought it would take them five years to reflect are seeing that time dwindle rapidly. All these concerns need to be addressed and assured to do so because they have to work on it now.
“All these things are coming to the fore,” says Das. “But the technology is at hand. No. We have the technology, and it’s important to put it in people’s hands.”
Generative AI seems to be everywhere, but there are still a myriad of problems that need to be addressed. Certainly, there is one associated power consumption mentioned above. But another, he says, is the amount of data that is created, collected, stored, processed, and analyzed, which feeds back into power issues. For the future of computing to be sustainable, organizations today seem hesitant to throw data away.
“Data is the fuel that drives all of this,” says Das. “We’ve improved the state of the art in storage a lot over the years, but devices, sensors, the rate at which data is generated from everything, and all these use cases for data are popping up now. So no one wants to throw that data away, and the number of data centers that would be needed to keep all this data [and] The number of storage systems required to hold all this data would be staggering. In fact, we have a lot of data. ”
This is both a vendor and an enterprise issue. For example, companies developing self-driving cars collect vast amounts of data while driving with cameras and sensors, recording everything and building datasets to train AI models. . Can I remove some of the incremental information? Nvidia has its own autonomous vehicle unit that simulates traffic conditions. This allows researchers to get the information they need without collecting large amounts of data.
On the enterprise side, enterprises can better deduplicate data to make storage more efficient. CIOs and other executives also need to inventory the data their systems hold and determine what is valuable and what can be discarded.
“So where is the most efficient place to store this data?” Das says. “It’s all part of a very important job description going forward. That part is on them. And obviously the vendor has to provide the best technology to make it all efficient.”
