VentureBeat Q&A: Hailo CEO Orr Danon Says Edge AI Means ‘Streaming Insights’, Not Video

The development of AI chip technology has advanced rapidly recently, and reports of new processors from Google and Microsoft suggest that the AI ​​dominance of Nvidia GPUs in the data center may not be complete.

Even outside the data center, new AI processing alternatives are emerging. This latter battle is marked by a group of embedded AI chip makers adopting novel approaches to conserve power while processing AI inference.

Count Hailo among these chip makers. The company advocates a non-von Neumann data flow architecture suitable for deep learning at the edge. The chip will work with a combination of DSP, CPU and AI accelerator, Hailo CEO Orr Danon recently told VentureBeat.

The company’s latest offering, the Hailo-15, can be embedded in cameras and can target large-scale camera deployments, saving power while offloading the expensive work of cloud vision analytics. The idea behind it is that pushing this kind of work to the cloud is useless. Not so when IoT advances. (Editor’s Note: This interview has been edited for length and clarity.)

VentureBeat: Nvidia has certainly become a prominent player in the AI ​​world.how do you measure your effort Edge AI Does it use a dataflow IC compared to Nvidia’s GPU efforts?

Or Danone: To be clear, Nvidia’s main focus is on servers and data centers. This is not subject to optimization. Instead, focus on embedded spaces. Nvidia offers products that are largely offshoots of their data center offerings, thus targeting very high performance, correspondingly higher power consumption and higher prices, but very functional is. For example, I think their next product target will run at 2 petaflops in an embedded form factor.

VB: Of course it doesn’t look like a chip anymore. They look like full-scale printed circuit boards or modules.

Danone: And it is valid of course. We take a slightly different approach. That means optimizing for power and looking at built-in space. And I think that’s a bit of a differentiation.

Of course, one of the great benefits of working with Nvidia is being able to work with the Nvidia GPU ecosystem. But even if you don’t need it, you’ll still have some overhead. It works fine when I scale it up, but it doesn’t work very efficiently, especially when I try to scale it down. That’s our space, and I think it’s a little less of a concern for Nvidia looking at very large deployments in their data centers.

Combining Computer Vision and Edge AI

VB: Still, the new Hailo chip has a lot of work to do. can be built into the camera. It starts with the input video signal, right?

Danone: There are multiple processing domains. One of them is the physical interface to the image sensor that handles auto exposure, auto white balance. These are all conventional image processing.

On top of that is video encoding. On top of that, there is a heterogeneous computing stack based on CPUs licensed from ARM that manage data analysis and data processing. On top of that is a digital signal processor, capable of doing more specialized operations than a CPU. And the heavy lifting is done by the neural net core.

The idea here is that the neural network does not operate in a control-flow fashion, but in steps. Rather, the processing is distributed through neural network accelerators within the SOC. [System on Chip].

Different parts of the accelerator are responsible for different parts of the computational graph and have data flowing between them. This is why it is called dataflow. This makes a lot of sense in terms of efficiency. Power consumption is dramatically lower compared to the level of computing performance obtained.

Internet of things you can see

VB: internet of things It seems to have evolved into several separate markets, where the specialty seems to be this visual processing.

Danone: I call it “IoTwE”. This is the Internet of Things, the way we see the world. Looking at IoT, it doesn’t make sense to just broadcast or stream everything you need to a central location. It just pushes the problem into another area and is not scalable. it is very expensive.

As you know, the greatest sign of intelligence is being able to concisely describe everything you see without spewing it out. For example, if you ask me what a good student is, it is someone who can summarize in a few words what the class just said.

What we need are highly intelligent nodes that understand the world around them and give insight to the rest of the network. Everything is connected, but I want to stream insights instead of streaming video.

VB: Why pursue a data flow architecture? Does the structure of neural networks influence the approach to design that is inherent in the chip?

Danone: That’s an important point. The whole idea of ​​data flow architecture is to see how neural networks are structured, but to provide something that doesn’t try to mimic them as a kind of hard-coded neural network. it’s not a thought.

Understanding the concept of data flow and how processing is distributed can lead to flexible architectures that map problem descriptions at the software level to product implementations at the hardware level relatively easily and efficiently.

Hailo is a dedicated processor. It’s not meant for graphics. It is not intended for encryption. It is for running neural networks and is inspired by the way neural networks are written in software.And it’s part of a complete system that serves [the needs of the applications] end to end.