NEC develops two high-speed generation AI LLMs

NEC Corporation (NEC, TSE 6701) has expanded its generative AI service “NEC cotori” and developed the latest high-speed generative AI large language models (LLM) “NEC cotori Pro” and “NEC cotori Light.” did. Training data and architecture.

In recent years, with the rapid development of generative AI, a wide range of organizations are considering and verifying business transformation using LLM. As specific application scenarios emerge, you must provide models and formats that meet your customers' needs in terms of implementation and operational response time, business data coordination, information protection, and other security aspects.

NEC's newly developed “NEC cotori Pro” and “NEC cotori Light” are models that achieve the same high performance as global LLM, but are more than 10 times faster and more efficient.

In general, improving LLM performance requires making the model larger, which slows down the operation. However, by developing advanced new training methods and architectures, NEC has been able to improve both speed and performance.

Top AI ML news: GenAI Enabler Langdock raises $3M to improve workplace productivity with LLM

“NEC cotori Pro” has two graphics processing units (GPU). In addition, the even faster “NEC cotori Light'' has performance equivalent to global models such as “GPT-3.5-Turbo,'' but can process large numbers of requests at high speed with an infrastructure of about 1 to 2 GPUs. , provides sufficient performance for many tasks.

Specifically, in an in-house document retrieval system that uses a technology called RAG, we achieved a correct answer rate of GPT-3.5 or higher without any fine-tuning, and a correct answer rate of GPT-4 or higher after fine-tuning. The time required will be reduced by approximately 93%.

Features of NEC cotori Pro and NEC cotori Light

High throughput for various tasks

Both “NEC cotori Pro” and “NEC cotori Light” are equipped with world-class processing power in terms of knowledge and ability to process a variety of tasks such as document summarization, logical reasoning, and question answering.

Specifically, we confirmed top-level performance not only in Japan but also in the world in two common benchmarks that measure the entire system. LLM performance:”ELYZA task 100” and “Japanese MT-Bench”. NEC cotori Pro provided results more than 5x faster than GPT-4

On a standard server with two GPUs. It also outperformed models such as Gemini 1.0 Pro and showed performance comparable to Claude 2 and GPT-4. At the same time, “NEC cotori Light” produced results that were more than 15 times faster than GPT-4, surpassed large models such as “LLaMA2-70B,” and demonstrated performance comparable to GPT-3.5-Turbo. read more:

Learn how AI can save your new business up to $5,000.

Cotori lineup of high-speed performance LLMs

For practical use of LLM, in addition to high performance during inference, the time (speed) from sending a request to receiving a response is also important. NEC cotori Pro and NEC cotori Light are equipped with two standard GPUs and achieve processing speeds of 87-93% compared to GPT-4. This was achieved thanks to architectural innovations that enhanced both performance and speed, as well as a large Japanese dictionary (for tokenization) in the model. These innovations not only increase the speed of inference, but also increase the number of concurrent accesses and reduce the training time required for fine-tuning.

NEC develops high-speed generation AI large language model (LLM) with world-class performance By utilizing a model that achieves high processing power through high-speed and large-volume access, it is possible to significantly shorten the response time of business applications that utilize generative AI and improve the user experience. Additionally, the high processing power can significantly improve performance after fine-tuning individual data for each company. NEC will continue to strengthen collaboration with partners and provide safe, secure, and highly reliable AI services based on the expanded AI infrastructure. NEC Kotomi

This lineup is designed to help customers solve a wide variety of complex problems.

Experimentally achieved 16-bit calculation accuracy in a GPU environment equipped with two L40sShare units NEC has positioned itself as a leader in the integration of IT and network technologies, driving its brand statement.

“Organizing a brighter world.”

[To share your insights with us as part of editorial or sponsored content, please write to sghosh@martechseries.com]