AI is taking the world by storm. You can use Google Bard or ChatGPT, but you can also use one hosted locally on your Mac. Here’s how to use the new MLC LLM chat app:
Artificial intelligence (AI) is the new frontier in computer science and has caused quite a bit of hype in the computing world.
chatbot – AI-based apps that allow users to converse as domain experts are growing in popularity, to say the least.
Chatbots appear to have expertise on a variety of popular and specialized subjects and are being deployed everywhere at a rapid pace. One group, OpenAI, released his ChatGPT a few months ago and shocked the world.
ChatGPT appears to have infinite knowledge on virtually any subject and can answer questions in real time that would otherwise require hours or days of research. Both businesses and employees have found that AI can be used to reduce research time and speed up work.
Disadvantage
However, given all of this, some AI apps also have drawbacks. The main drawback of AI is that the results still need to be accurate. Verified.
AI usually provides mostly correct data, but it can provide incorrect or deceptive data leading to incorrect conclusions and outcomes.
Software developers and software companies are adopting “copilots” (specialized chatbots that can help developers write code by having artificial intelligence automatically outline functions and methods), person can verify it.
This saves a lot of time, but it also means that the co-pilot can create the wrong code. Microsoft, Amazon, GitHub, and NVIDIA all release Copilots for developers.
start a chatbot
To understand, at least at a high level, how chatbots work, you first need to understand the basics of AI, especially machine learning (ML) and large language models (LLM).
Machine learning is a branch of computer science dedicated to the research and development of attempts to teach computers to learn.
LLMs are essentially natural language processing (NLP) programs that use huge data sets and neural networks (NNs) to generate text. LLM works by training AI code on a large data model, learning from it over time and becoming an expert in a particular domain based on the accuracy of the input data.
The more input data (and the more accurate it is), the more accurate and accurate the chatbot using that model will be. LLM also depends on deep learning while being trained on the data model.
When you ask a question to a chatbot, the chatbot asks the LLM for the most appropriate answer based on its learning and stored knowledge of all subjects related to the question.
Essentially, chatbots have precomputed knowledge about a topic, and given enough LLM accuracy and enough learning time, they can provide correct answers much faster than most people.
Using a chatbot is like having a team of automated PhDs at your immediate disposal.
In January 2023, Meta AI released its own LLM. llama. A month later, Google introduced its own AI chatbot, Bard, based on its own LLM. LA MDA. Other chatbots have since emerged.
Generation AI
More recently, some LLMs have learned how to generate non-text-based data such as graphics, music, and even entire books. Companies are interested in generative AI to create things like corporate graphics, logos, titles, and even digital movie scenes that replace actors.
For example, the thumbnail image in this article was generated by AI.
As a side effect of generative AI, workers have become concerned about losing their jobs due to automation by AI software.
chatbot assistant
The world’s first commercially available chatbot (BeBot) was released by Bespoke Japan for Tokyo Station City in 2019.
Released as an iOS and Android app, BeBot guides you anywhere around the maze-like station, helps you store and retrieve your luggage, sends you to the information desk, tells you train times, ground transportation, Knows how to search for food and drink. A shop inside the station.
In addition, you can see which station platform you should go to for the fastest train ride to your destination in the city, by travel time. All done in seconds.
Tokyo Station, home of BeBot.
MLC chat app
of Machine learning compilation The (MLC) project is the brainchild of Apache Foundation deep learning researchers Siyuan Feng, Hongyi Jin, and others based in Seattle and Shanghai, China.
The idea behind MLC is precompiled Connect LLMs and chatbots to consumer devices and web browsers. MLC harnesses the power of consumer graphics processing units (GPUs) to accelerate AI results and searches, making AI available on most modern consumer computing devices.
Another MLC project, Web LLM, provides the same functionality for web browsers and is based on another project. WebGPU. Web LLM relies on code frameworks that support specific GPUs, so only machines with specific GPUs are supported.
While most AI assistants rely on a client/server model where the server does most of the AI heavy lifting, MLC embeds LLM in local code that runs directly on the user’s device, thus eliminating the need for an LLM server. disappears.
MLC setup
To run MLC on your device, it must meet the minimum requirements listed on the project and GitHub page.
To run on an iPhone, you need an iPhone 14 Pro Max, iPhone 14 Pro, or iPhone 12 Pro with at least 6GB of free RAM. Installing the app also requires Apple’s TestFlight app to be installed, but installation is limited to his first 9,000 users.
I tried running MLC on a base 2021 iPad with 64 GB of storage and it did not initialize. Results may vary on iPad Pro.
You can also build MLC from source and run it directly on your phone by following the instructions on the MLC-LLM GitHub page. To get the source, you need the git source code control system installed on your Mac.
To do this, create a new folder in the Finder on your Mac and select cd Go to that location in the terminal with the command, git clone Terminal commands listed on the MLC-LLM GitHub page:
https://github.com/mlc-ai/mlc-llm.git and press return. git will download all the MLC sources to the folder you created.
MLC running on iPhone. You can select or download the weight for your model.
Install prerequisites for Mac
For Mac and Linux computers, MLC is run from the Terminal command line interface. To use it, you need to install some prerequisites first.
- Conda or Miniconda package manager
- self made
- Vulkan graphics library (Linux or Windows only)
- git large file support (LFS)
For NVIDIA GPU users, the MLC instructions clearly state that the Vulkan driver should be manually installed as the default driver. Another graphics library for NVIDIA GPUs – CUDA – doesn’t work.
For Mac users, you can install Miniconda using the previously mentioned Homebrew package manager. Note that Miniconda conflicts with another Homebrew Conda formula. mini forge.
So if you already have miniforge installed via Homebrew, you need to uninstall it first.
Following the instructions on the MLC/LLM page, the rest of the installation steps are roughly:
- Create a new Conda environment
- Install git and git LFS
- Install command line chat app from Conda
- Create a new local folder, download the LLM model weights, and set the LOCAL_ID variable.
- Download the MLC library from GitHub
All of this is covered in great detail on the instructions page, so I won’t cover every aspect of the setup here. It may seem daunting at first, but with basic macOS Terminal skills, it’s actually just a few steps away.
In the LOCAL_ID step, simply set that variable to point to one of the three model weights you downloaded.
The model weights are downloaded from the HuggingFace community website, like AI’s GitHub.
Once everything is installed in your terminal, you can access the MLC in your terminal with the following command: mlc_chat_cli
instructions.
Using MLC in a web browser
MLC also has a web version of Web LLM.
Web LLM variants only run on Apple Silicon Macs. It won’t run on Intel Mac and will throw an error in the chatbot window if you try.
There is a pop-up menu at the top of the MLC web chat window that allows you to select the downloaded model weights to use.
Select one of the model weights.
Web LLM requires a Google Chrome browser (Chrome version 113 or higher). Earlier versions will not work.
You can find your Chrome version number from the Mac version of the Chrome menu. Chrome->About Google Chrome. If an update is available, update Click the button to update to the latest version.
You may need to restart Chrome after updating.
Note that the MLC Web LLM page recommends launching Chrome from the Mac Terminal using the command:
/Applications/Google\ Chrome.app/Contents/MacOS/Google\ Chrome --enable-dawn-features=allow_unsafe_apis,disable_robustness
“allow_unsafe_apis” and “disable_robustness” are two Chrome startup flags that allow the use of experimental features, which may or may not be unstable.
Once everything is set up, ask your question Please enter your message Click the field at the bottom of the chat pane on the Web LLM web page and select send button.
The era of true AI and intelligent assistants is just beginning. AI has its risks, but the technology promises to improve our future by saving a ton of time and eliminating a lot of work.
