Esperanto Technologies is working to accelerate software support for new high-performance AI chips. The company ported the Large Language Model (LLM) from Meta to his RISC-V-based ET-SoC-1, which contains 1,088 cores. This allows researchers to evaluate powerful LLMs on servers not based on x86 or Arm architectures.
RISC-V is a new, freely-licensable architecture known for its ground-up design and minimalist approach. Apple and Nvidia have his RISC-V controllers on their chips, and Qualcomm talks about the potential of this architecture for his mobile chips. Intel has helped create a computing board based on his RISC-V, but recently withdrew financial support for the architecture as part of cost cutting. RISC-V cores are popular in microcontroller applications and are eroding market share for older Arm-based designs. Esperanto is one of the few companies trying to push RISC-V into the high-performance computing arena. This company was founded by Dave Ditzel, who previously founded Transmeta.
However, RISC-V faces an uphill battle moving into performance applications such as PCs and servers. Hardware is easy to get, but building software and data around it is hard, explains Roger Kay, principal analyst at Endpoint Technology Associates.
Now, Esperanto is working to fill that software gap with a self-developed AI tool.
ET-SoC-1 silicon. Image credit: Dave Ditzel/Esperanto Technologies.
The company ported Meta’s Open Pre-Trained Transformer model (believed to be roughly comparable in size and performance to OpenAI’s GPT-3) to the chip. This model is designed to run on an evaluation server with 8 or 16 ET-SoC-1 cards that can be plugged into PCIe slots, providing acceleration up to 17,000 total cores.
Esperanto’s software gives customers the tools they need to try out different AI hardware products, said Jim McGregor, principal analyst at Tirias Research. The servers will be used for inference, allowing customers to better understand performance and power consumption, McGregor added.
Companies behind RISC-V market this architecture as a low-power option for high-performance computing applications. AI is dominated by Nvidia, and some companies may be interested in checking out the power and performance his RISC-V offers, McGregor argued.
The experimental AI server contains dual Intel Xeon host processors, but details are unknown. Esperanto did not respond to requests to speak with company representatives about the hardware and software. The servers run all standard industrial AI models, and customers “can bring their own models and data,” the company said in a statement.
RISC-V is still years away from being a viable alternative for supercomputing and AI at x86 and Arm scale. Intel has repeatedly stated that it will offer the ability to build his RISC-V CPUs alongside GPUs in chiplet designs, but that’s a long way off. The European Processor Initiative (EPI) is funding the development of RISC-V chips through research centers such as the Barcelona Supercomputing Center (BSC). However, Europe’s first exascale He SiPearl, who is developing a chip for a supercomputer, says RISC-V is not yet ready for serious time.
Smaller chip companies are down like flies amid lack of funding and a difficult economic environment, McGregor says, taking extra steps to make chips attractive to hyperscalers and customers. .
AI accelerators typically have matrix multipliers, and ET-SoC-1 has vector instructors and tensor instructors for AI. “The ET-SoC-1 chip is designed to run ML-recommended workloads, computing at peak rates of 100-200 TOPS and consuming less than 20 W,” the company said in its Hot Said in a slide deck presented at Chips. 33.
Related
