On June 29, Palo Alto-based Inflection AI announced the completion of a $1.3 billion funding round led by Microsoft, Reid Hoffman, Bill Gates, Eric Schmidt and Nvidia. Part of the new capital will be allocated to building a 22,000-unit Nvidia H100 Tensor GPU cluster, which the company claims is the largest in the world. GPUs are used to develop large-scale artificial intelligence models. A developer wrote:
“If our cluster were to make it onto the recent TOP500 list of supercomputers, we would expect it to be the second, near top entry, even though it is optimized for AI applications rather than science. “
Inflection AI is also developing its own personal assistant AI system called “Pi”. The company described Pi as a “teacher, coach, confidante, creative partner and mentor” who can be reached directly via social media and his WhatsApp. Since its launch in early 2022, total funding has reached $1.525 billion.
Despite increasing investment in large-scale AI models, experts warn that current technical limitations may severely limit practical training efficiency. In one example given by Singapore’s venture fund Foresight, the researchers wrote:
“Assuming we have 100 compute nodes and each node needs to update all parameters at each step, each step will need to send about 70TB (700GB * 100) of data. Assuming optimistically that it takes 1 second, we need to send 70TB of data per second, a bandwidth demand that far exceeds the capacity of most networks.”
Continuing the example above, Foresight warned that “communication delays and network congestion can lead to data transmission times well in excess of one second.” This means that compute nodes can spend most of their time waiting for data to be sent instead of performing the actual computation. Given current constraints, they conclude that the solution lies in smaller AI models that are “easier to deploy and manage.”
“In many application scenarios, users and enterprises do not need the more universal inference capabilities of large language models, and are focused only on highly sophisticated prediction targets.”
Magazine: AI Eye: AI Travel Bookings Are Hilariously Bad, 3 Weird Uses of ChatGPT and Crypto Plugins
