Explaining Google’s New Large Language Model

Google was one of the first to adopt generative AI, but is now blinded by the explosive growth of rivals such as ChatGPT and Bing Chat. So Google responded by launching the Bard AI chatbot, to mixed reactions. Recently, the company also began experimenting with displaying AI responses alongside search results. But even more surprising is that Google developed the machine learning model in-house. You may have heard of Google LaMDA around 2020, but the company is now switching AI products to his more advanced PaLM 2.

So in this article, let’s take a closer look at the differences between PaLM 2 and previous Google models. Later, I will also explain how it compares to GPT-4, which is currently regarded as the best large-scale language model available.

PaLM stands for Pathways Language Model and refers to the use of Google’s Pathways AI architecture to help train machine learning models to perform a variety of tasks. Reaching general artificial intelligence (AGI) is a big part of Google’s vision.

Google believes the Pathways architecture paves the way for multimodal AI beyond text. In a blog post, the company said:

Pathways may enable multimodal models that simultaneously encompass visual, auditory, and linguistic comprehension. So, regardless of whether the model is processing the word “leopard”, or the sound of someone saying “leopard”, or a video of a leopard running, under the hood it will react the same: concept is activated. The result is a model that is more insightful and less error-prone and biased.

Google first mentioned PaLM in 2022, but the company never announced a product using it. However, under increasing competitive pressure, Google has now upgraded the Bard chatbot to use PaLM 2. Unlike the LaMDA model he replaces, PaLM 2 has been trained in over 100 languages ​​and has even better domain-relevant knowledge in areas such as coding. They are also much better at logical reasoning and mathematical ability.

From a technical perspective, Google’s first generation LaMDA machine learning model boasted 137 billion parameters and a training size of 1.56 trillion words. This time, Google says PaLM 2 uses a new technique known as compute-optimized scaling to produce better results with fewer parameters and smaller training datasets.

Google has yet to commit to bringing multimodal capabilities like Bard into its AI products. However, future Gemini language models will support things like images and sounds.

PaLM 2’s features have improved Google Bard’s capabilities quite significantly. When first released in early 2023, the chatbot used his LaMDA language model under the hood. And now, with his move to PaLM 2, Google has leapfrogged his first-generation PaLM model.

Google has published a 92-page technical report detailing how PaLM 2 improves on its predecessor, but let’s get down to business. Here’s a quick rundown of the improvements over the company’s previous models:

1. PaLM 2 was trained in different languages ​​and its training data included many non-English text examples. He also performs better in foreign language tests than LaMDA and his PaLM in the first generation.
2. It has better logical reasoning capabilities than previous versions and was even able to match GPT-4 in tests conducted by Google.
3. Google’s Bard chatbot can now generate and debug code. That was not possible with the company’s previous language model.
4. PaLM 2 can translate between languages ​​better than previous language models. In fact, it may even surpass Google Translate in certain languages ​​such as Portuguese and Chinese.
5. The new Pathways architecture makes PaLM 2 more efficient to train and use. This makes it more economical and feasible to embed in Google services such as Gmail.
6. Google can fine-tune the PaLM 2 model for specific use cases. For example, Sec-PaLM is optimized to detect cybersecurity vulnerabilities and Med-PaLM provides answers to medical questions.

How is PaLM 2 different from OpenAI’s GPT LLM?

OpenAI, creators of ChatGPT, have so far managed to outperform other language models in terms of functionality. But with the release of PaLM 2, can Google take the crown? However, synthetic benchmarks don’t necessarily reflect real-world performance.

So I asked Google Bard and Bing Chat some riddles to see which came first. Microsoft uses OpenAI’s GPT-4 as the foundation for Bing Chat. I used a bit of creative prompt engineering to avoid searching the internet for a response. Here is the result of the mystery I originally asked.

As you can see in the screenshot above, both Google Bard and GPT-4 were able to solve the riddle correctly. In fact, PaLM 2 also provided an alternative solution that Bing Chat did not.

But Google’s Bard chatbot refused to solve another riddle I threw at it. Bing Chat, on the other hand, provided detailed reasoning and correct solutions when asked the same riddle.

These two examples don’t test the full capabilities of PaLM 2, but they do prove that while Google is starting to catch up to its competitors, there’s still a lot of work to do.

At Google’s I/O developer conference, we got a glimpse of the company’s plans for the future of PaLM 2 and its AI development in general. The latest language model powers everything from the new “Help me Write” functionality in Google Docs and Gmail to AI image generation in Slides. Broadly, these capabilities fall under Google’s Duet AI for Workspace umbrella and will be rolled out in the coming months.

Google has also developed a version of PaLM 2 adapted for smartphones. Codenamed Gecko, this condensed language model could one day make generative AI fully offline on a typical Android smartphone.

Finally, software developers can now access PaLM 2 programmatically via an API. This means that other companies may rely on Google’s large-scale language model in the same way that Duolingo and Khan Academy have adopted his GPT-4 for their AI-powered features. means