welcome to long view—here we peruse the news of the week and strip it down to what it essentially is. what really matters.
This week: New ideas bring low-power ML inference, and more Big Tech work is underway.
1. FP8 Brings Faster AI
First topic of the week: A new proposal for standardizing 8-bit floating point storage. The idea is to speed up real-time inference where speed and power efficiency matter. More important than accuracy..
Analysis: FP8 > INT8
Limiting the data to byte width obviously improves speed and power. 8 bit integer It’s been used in deep learning for a while, but it suffers from overflow problems. Cutting out the exponent gives much wider dynamic range.
Karen Heyman: Does Floating Point 8 solve AI/ML overhead?
“FP32 or FP16 is unnecessary overhead”
While the media is talking about … ChatGPT, engineers are focusing on the hardware challenges of running large-scale language models and other deep learning networks. High on the ML punch list is how to run models more efficiently with less power, especially in critical applications such as self-driving cars where latency is a matter of life and death.
…
IEEE 754, which defines FP32-bit and FP64, was designed for scientific computing where precision is the ultimate consideration. … machine learning often doesn’t need that much accuracy. … Arm, Intel and Nvidia have published white papers proposing… Two different flavors of FP8: E4M3 (4-bit exponent and 3-bit mantissa) and E5M2 (5-bit exponent and 2-bit mantissa) … overhead to save energy and performance.
…
Training requires a wide dynamic range to continuously adjust the coefficients, which is characteristic of backpropagation. …for inference, there is a quantization step that shifts the network to FP8 or Integer 8 (Int8), since continuing operations in FP32 or FP16 is unnecessary overhead. …As the quest for lower power consumption continues, there is also ongoing debate as to whether there will be an FP4 standard.
Fascinating discussion. And ardit33 gets it:
This is a great article. I recently had to deal with this, FP8/INT8 are very important especially for inference on mobile/devices.
AI will begin to be built into devices of all kinds…and in many cases, backend reasoning is undesirable or not feasible. It’s actually pretty cool if the industry moves to his INT8/FP8 inference as standard.
Is there another way? Wolf Star:
Personally, I would expect Posit to be more recruited. Not only because their behavior is a bit smarter than floats in most cases, but there are also advantages associated with using them for AI.
2. Amazon and Microsoft cut 28,000 jobs
Two Washington state giants announced significant job cuts this week.just like them adjusted their news.
Analysis: Welcome to a recession that’s not a recession
By that official definition, we are not in recession. But given the behavior of tech companies, it certainly seems so.
Matt Day and Spencer Soper: Amazon preparing for new job cuts
“among some of the big technology companies”
The company announced earlier this month that it would lay off more than 18,000 employees. …he at Amazon represents about 6% of the company’s 350,000 employees.
…
Amazon is among the big tech companies that have slipped in the rankings, including Cisco Systems, Intel, Meta Platforms, Qualcomm and Salesforce.
However, DevOps jobs are not necessarily affected. However, theodp doesn’t like optics.
The hiring party at Amazon is over. The number of job openings in the software development sector has decreased from 32,692 he in May 2022 to 299 he in January 2023.
…
[Last month] Amazon … has delayed the start date of some college graduates who were supposed to join the company in May 2023, blaming the “macroeconomic environment” and telling students they won’t be able to start until the end of 2023.
And the other shoe fell off. Dina Bass joined Matt and Spencer for Round II. Microsoft plans to cut 10,000 jobs
The software giant has started notifying some of the 10,000 employees who will lose their jobs this quarter. … Both companies said painful steps were needed to offset slowing sales and the possibility of a recession that had made customers more cautious.
Before the cuts were announced, CEO Satya Nadella said the tech industry was going through a period of slowing growth and needed to adjust. with our own technology. ”
Unusually, this time I learned about Microsoft’s layoffs from the press, not from the CEO. verst feels weird about it:
We learned about Microsoft’s job cuts in the news yesterday, but so far we don’t know who will be affected. I’m sure they’ll handle it as best they can, but it’s strange to learn about it first from the news and not from the CEO. There aren’t many in-house jobs that people who have been laid off can apply for because they haven’t.
…
the leak is [not] Unusual, but… Learn about it this way feel strange. … At Microsoft, the CEO usually does a great job of announcing important news to employees via email and making it available to investors at the same time. Not this time.
hang on. pause. Are we in recession or aren’t we? nightflameauto laments the dignity of labor:
i believe the current definition recession, as recently utilized by businesses and governments, is “a nebulous conception of a future in which profits are declining, by reducing employee bargaining power, employee career interests, and employee morale.” You have to fight until they come back.Stop thinking your work is worth it to the company.” Big companies are inherently paid fairly for their work Tired of employees having the stupidity to believe they should. The solution: The constant chatter of a recession while continuously pulling out historically great moves that destroy employee morale.
…
Ah, capitalism is great. It smells of death and decay.
what’s that? do you want to unite Mabo is not a socialist.
In my opinion union Not the right word for what I want. I want a co-op where the employee organization owns the company. Profits are all bonuses.
…
I don’t want to be shielded from the company’s goal of making as much money as possible. You want your company’s goal to be employee success.
Moral of the story:
You never really learn much by listening to yourself speak
-George Clooney
you are reading long view By Rich Jennings.you can contact him @RiCHi or tlv@richi.uk.
Image: Jonny Gios (via Unsplash, leveled and cropped)
