Recent developments in natural language processing have enabled large-scale language models (LLMs) that can understand and generate human-like language. A particular LLM can hone a particular job in several ways through discussion as a result of learning a large amount of data. A good example of such an LLM is ChatGPT. Robotics is one of the exciting areas where ChatGPT will be employed, where it can be used to convert natural language commands into executable code for instructing robots. Generating robot programs from natural language commands is a desirable goal, and several studies are extant, some of which are based on his LLM.
Unfortunately, most of them lack human interaction features, are built with a limited scope, or rely on hardware. However, since much of this work relies on a specific dataset, we need to recall the data and retrain the model in order to adapt or extend it to different robot situations. Robotic systems that can be easily adapted to multiple applications and operating environments without the need for extensive data collection or model retraining are superior from a practical application perspective. An advantage of adopting ChatGPT for your robot application is that you can start with a moderate amount of sample data to tailor the model to your specific application and take advantage of its language recognition and interaction capabilities as an interface.
Although ChatGPT’s potential for robotic applications has been noted, there are currently no proven approaches that can be used in practice. In this study, Microsoft researchers demonstrate how ChatGPT can be applied to a few-shot situation to translate natural language commands into sequences of actions that a robot can perform (Figure 1). Prompts were created to be easily adaptable while meeting common specifications in many real-world applications.
To meet these requirements, they designed an input prompt that prompts ChatGPT to 1) output a predefined set of robot actions with descriptions in readable JSON format. 2) Express the operating environment in a formalized style. 3) It can infer and output an updated state of the operating environment and reuse it as input for the next, allowing ChatGPT to operate based solely on memory of the most recent operation. They conducted experiments to test the effectiveness of the proposed prompts in inferring the appropriate action of multi-stage verbal instructions in different environments. 1) Simple interaction with robot execution systems or visual recognition software. 2) Adaptability to a variety of home environments. 3) Ability to deliver any number of plain English instructions while mitigating the impact of ChatGPT’s token limit.
They also noted that ChatGPT’s conversational capabilities allow users to modify their output using natural language feedback. This is important for creating secure and resilient applications while providing the user her friendly interface. A collection of robot actions, environment expressions, and object names can all be easily modified and used as templates in suggested prompts. The contribution of this paper is to create and disseminate generic prompts that can be easily adapted to the needs of each experimenter, providing useful information to the robotics research community. They are open source, freely accessible on GitHub, and prompt to use.
check out paper and github. All credit for this research goes to the researchers of this project.Also, don’t forget to participate Our 18k+ ML SubReddit, cacophony channeland email newsletterWe share the latest AI research news, cool AI projects, and more.
🚀 Check out 100 AI Tools in the AI Tools Club
Aneesh Tickoo is a consulting intern at MarktechPost. He is currently pursuing a Bachelor’s Degree in Data Science and Artificial Intelligence from the Indian Institute of Technology (IIT), Bhilai. He spends most of his time on projects aimed at harnessing the power of machine learning. His research interest is image processing and his passion is building solutions around it. He loves connecting with people and collaborating on interesting projects.
