Jared Quincy Davis, founder and CEO of Foundry, joins Asking For A Trend to share his insights on the future of AI infrastructure and Foundry's role in it.
Davis argues that GPUs are one of the most important commodities in AI, and that the AI sector is currently experiencing an issue with them being underutilized. He adds that the demand for Foundry's services and the extent of AI growth is “really massive” and underestimated.
The Foundry CEO points to complex AI systems as evidence of this underestimation: “[It] “It's not reflected in the pricing yet, but we're seeing demand from all sectors. We're seeing demand not only from enterprises that are finding more productive use cases for AI and are increasingly adopting it in production, but also from startups that are application companies that are using AI for their end applications.”
For more expert insights and the latest market trends, click here to watch the full episode of “Ask the Trends.”
This post Nicholas Jacobino
Video Transcript
That's right, the artificial intelligence boom has ignited a computing war for NVIDIA alone, with first-quarter data center revenue up about 430% from last year.
This has led to an increased demand for larger graphics processing units.
While supply is a concern, our next guest also points out other important factors to consider.
Jared Quincy Davis, founder and CEO of Foundry, will be joining us with more information from Jared.
Nice to meet you.
Hey Josh, good to see you.
So I thought it would be interesting to describe your company.
Well, actually, I think one of your backers or investors was talking about something like GPUs are king in the AI world, is that true?
And as they explain it, what you're doing is creating technology to make very scarce, competing resources more widely available.
Is that a good idea?
Well, I think that's a great way to put it, and it's also part of our mindset to think that GPUs are one of the most important commodities in the whole of capitalism.
Now everybody wants to get it. Everybody wants to get it.
This is one of the biggest areas of spending for basically any major company. You know, Google spends more on computing than it does on people.
now.
Well, that's true for a lot of the big open companies.
I spend a very conservative $4-5 on compute for every $1 in labor, while the typical startup currently spends $2-3 on compute for every $1 in labor.
So tipping is a major part of spending, but having said that, I still think people don't take advantage of it very well.
Well, actually, there's a lot you can do to map workloads onto the chip more efficiently and get a lot out of it.
So they're basically underutilized.
right?
Ok, there has been a lot of talk about GPU undersupply, but I would argue that rather than an issue of undersupply, it's actually an issue of underutilization.
So one of the things that Foundry does is manage resources really well. We're like a schedule management company, if you like.
What you're saying.
It seems easy.
But in reality, I think this technology is quite complicated.
Well, that's a pretty interesting question.
please think about it.
It's like Tetris, there are a lot of workloads and they come in like blocks.
Basically, install them efficiently and try to minimise gaps and waste.
Well, this is kind of a variant of the bin packing problem or the scheduling problem, except that we have some hacks that allow us to do this very well.
What is the demand for the services you offer right now and where is that demand coming from?
Yeah, I think the demand is really impressive.
I think people are actually still underestimating not just how big the demand for AI chips is, but also the extent of the growth and the demand. We can talk about that.
There is an interesting trend called composite AI systems.
I think you're seeing tremendous growth that hasn't been reflected in the price yet.
But we are seeing demand from all corners: from companies that are finding more productive use cases for AI and putting it into production, from start-ups, from application companies that are using AI in their end applications, and also from R&D companies that are building better base models, foundational models that other people use in their applications.
By the way, how did you come up with this idea for Jared?
What is the origin here?
Well, a lot of the people on our team, myself included, have kind of interesting backgrounds.
We worked at the intersection of several normally distinct fields, one of which is deep learning research.
Previously, I was part of the Core Deep Learning team at DeepMind.
And DeepMind is now a division of Google, leading many of the company's cutting-edge AI efforts.
I and many others on our team have PhDs and systems that are not typically AI.
These are almost separate tribes and they don't mix. We have a PhD in systems under the guidance of a great PI called M Zaara, who is also the CTO and founder of Data Bricks.
So for a long time, we've been thinking about the difficult question of how to best map workloads to compute.
Well, a lot of us have a finance background.
So during my undergrad and PhD I worked for a private equity firm called KKR, for example, and learned a lot about data centers and near price software.
So I think we have kind of an interesting perspective at the intersection of the monetary system and NML and thinking about how to evolve economic calculation in a little bit of a new way, Jerry.
It's a great story and a great companion to Megatrends.
There's a lot to talk about here.
Thank you for joining us today.
I appreciate it
thank you.
Thank you, Josh.
