In the early days of artificial intelligence research, there was an atmosphere of community openly sharing ideas and working together to improve technology. But a lot has changed with ChatGPT taking the world by storm last year.
Tech giants such as Google, Microsoft, and Facebook are looking to capitalize on the AI gold rush by increasing access to the tools that will define the AI landscape. They limit access to tools that can answer questions, generate images, and understand spoken language.
But for some, especially hardware makers, handing over control of AI to some wealthy tech company would hurt their business. These companies are supporting the emerging open source movement, making AI technology cheaper and more accessible.
defector
OpenAI, a leading AI company, is a notable defector. The company was founded in 2015 as a non-profit organization to promote and share AI research. We’ve opened up access to large-scale language models, including GPT-3, the force behind ChatGPT.
However, OpenAI did not publish its latest multilingual model, GPT-4, which was released last month. GPT-4 is used by Microsoft in Bing search. Microsoft has invested billions in OpenAI, which became a for-profit organization in 2019. OpenAI is charging for his access to GPT-4.
Elon Muskwas an early contributor to OpenAI. Tweet OpenAI was meant to be open source, but has “became a closed source biggest profit company effectively controlled by Microsoft.” Indeed, Musk promised he would invest his $1 billion in OpenAI, but called it off after a power struggle and Microsoft stepped in to provide the much-needed cash injection.
One aspect of the open AI ecosystem revolves around opening large language models (LLMs) for close scrutiny by the community. In addition to OpenAI, technology giants Google, Facebook, and Nvidia have developed their own large-scale language models and deployed them in hardware. However, access to cutting-edge AI tools is limited to researchers or a small number of developers.
Closed LLM
There is concern that publicly distributed models will be used for malicious purposes. OpenAI cites security as a reason to keep GPT-4 private. Bloomberg is keeping the recently released Bloomberg-GPT model closed for safety and business reasons. The model was trained on decades of data that form the foundation of the financial services Bloomberg provides to its clients.
“As is well known, LLM is susceptible to data leakage attacks and can extract important segments of text given the model weights. Permission does not guarantee that the model will not be leaked,” the Bloomberg researchers said in a paper detailing the model.
Limited access to large, closed language models can be provided via APIs, but “giving researchers selective access does not guarantee that the models will not be leaked,” says the study. said the person.
Development tools such as TensorFlow and PyTorch are already open source, but require high-performance hardware such as GPUs to run programs.
However, more and more companies are joining the grassroots movement to open up Large Language Models so that proprietary models do not dominate the market.
Hardware manufacturers take the lead
Hardware makers are leading early efforts to advance open source AI. Cerebras Systems, which makes what is believed to be the world’s largest AI chip the size of a wafer, last month released the Cerebras-GPT model with up to 13 billion parameters.
“We open source Weights, open source Checkpoints, and display and provide full recipes for copying. We do this under the most permissive open source license possible.” Cerebras Systems CEO Andrew Feldman told The New Stack.
These models are a fork of OpenAI’s GPT-3 with 175 billion parameters. Feldman took a hard look at the increasingly proprietary approaches of OpenAI, Google and Facebook, saying Cerebras’ goal is to offer his AI model in open source as a low-cost alternative.
“OpenAI, Meta, etc. are closing these models to others because the value of their results is getting bigger. I think it’s an effort to keep it in a very large company,” Feldman said.
But he also sympathized with the tech giants who openly shared their AI research and are looking to recoup the millions or billions of dollars spent training their models. We are spending dollars upgrading our computing infrastructure to include GPUs and other accelerator chips that can run AI workloads.
“Several businessmen have said, ‘Why on earth are you sharing these results? And they can now make a lot of money.”
Cerebras’ seven AI models run on any hardware. But the company also wants to use the software to showcase the performance of what is believed to be the fastest AI chip in the world.
Evolve like Linux
Analysts say the open source movement is evolving much like Linux development was born out of a need to compete with proprietary operating systems. Linux is now the backbone of the Internet, providing the building blocks for cloud-native computing.
Intel, one of the largest contributors to the Linux kernel, also provides open source AI development plumbing. Some of its tools include OneAPI, an open source framework for developing and deploying applications.
In a roadmap presentation last month, Intel Chief Technology Officer Greg Lavender said:
Chip makers are developing a wide range of chips to run AI applications, including GPUs and accelerators like the Gaudi accelerator. An open-source approach to AI could make that chip more attractive to clients. Intel has taken a similar approach with Linux, providing hardware drivers in the Linux kernel to ensure chip compatibility with each new OS release.
But Intel’s AI presence doesn’t compare to Nvidia, which made AI computing possible through GPUs. Today, Nvidia GPUs are used to run AI applications deployed by Microsoft and Facebook. Google and Amazon also host Nvidia’s latest Hopper GPUs for clients to run training and inference for AI applications.
Nvidia wants to monetize its dominance in AI and believes a closed-source approach is the way to go. The company uses proprietary hardware and software tools to lock developers into its ecosystem.
Nvidia’s software development stack, called CUDA, is already popular among AI developers. Applications written in CUDA will only run on the company’s GPUs. Other machine learning frameworks such as OpenCL and ROCm are available, but moving out of CUDA can be expensive.
Graphics chip makers say they have open source libraries for developing vertical AI applications, but they require their GPUs to run. Intel is trying to cut off that proprietary approach with his SYCL tool, which decouples CUDA-specific code and allows applications to run on any CPU, GPU, FPGA, or other accelerator. .
Nvidia has also built a services business around AI. Companies can send their AI needs to his Nvidia. Nvidia builds and deploys applications across GPUs. Nvidia wants to become an AI software giant and capitalize on what the company believes could be a trillion dollar market opportunity.
