4 Prompt Techniques for Solving Difficult, Multi-Step Problems with LLM

Large Language Models (LLMs) have a terrible reputation when it comes to tackling inference-based problems. However, applying a simple method that does not require fine-tuning or task-specific validation can significantly improve inference performance. Chain of Thought (CoT) Prompt is the name of this method. Specifically, we use few-shot learning to enhance the deductive thinking ability of LLM. Many of the more advanced prompting strategies build on the Chain of Thought (CoT) prompting foundation and help address difficult multi-step problems in LLM.

Here, we present four methods of prompts to help LLM deal with complex multi-stage problems, presented by the joint efforts of researchers from Google, the University of Tokyo, Peking University, and Microsoft.

1. Zero shot CoT

In scenarios where traditional zero-shot strategies fail, zero-shot-CoT builds rational inference paths with zero-shot methods to find the correct solution. This is achieved without resorting to a few iterations of learning by inserting “think step by step” into the query. Unlike previous task-specific prompt engineering, which typically took the form of examples (few-shots) or templates (zero-shots), zero-shot CoT is flexible and task-agnostic, facilitating step-by-step answers. It can be performed across a wide range of reasoning tasks (including arithmetic, symbolic reasoning, common sense reasoning, and other logical reasoning tasks) without requiring rapid changes.

2. Least prompts to most prompts

LLM’s problem-solving technique is to openly decompose the problem into smaller, more manageable chunks and feed the results of each chunk into the next.

It has two different phases.

1. Decomposition: At this point, you are prompted with a question that requires decomposition, followed by a set of constant instances indicating the decomposition.
2. Problem Solving: At this point, the question to be answered is preceded by a series of constant instances indicating how the subproblem will be addressed, followed by previously answered subquestions and generated solutions. followed by a list of and finally the question itself.

Prompting from min to max can be used in conjunction with other methods such as inference chaining and self-consistency, but this is not required. Two phases of prompting from minimum to maximum can be combined into her one path for a particular activity.

3. Self-consistency

The language model’s reasoning power is further enhanced by using a unique decoding method called self-consistent instead of the greedy decoding method utilized in thought chain prompts. To achieve self-consistency, researchers work on the intuition that for the most complex reasoning tasks, there are several valid routes to a solution. The more time and effort you have to put into thinking and analyzing the problem, the more reasoning routes you have to reach the solution. The final decision is made by majority vote.

4. Diverse

In addition to being self-consistent, DiVeRSE uses a technique called prompt ensembles (groups of prompts all addressing the same problem) to guess/aggregate the correct answer from the various generated inference paths. Train the validation module for .

DIVERSE is a powerful and general strategy for improving the inference power of large language models. There are three main ideas for different things. Variety of prompts, voting validation features, and step-level accuracy. DIVERSE with codeavinci-002 outperforms the 540B PaLM model and previous prompting techniques combined to produce state-of-the-art results in most inference tests.

Please check Paper 1, Paper 2, Paper 3, and Paper 4.This article is inspired by Tweet.don’t forget to join 21,000+ ML SubReddit, Discord channeland email newsletterShare the latest AI research news, cool AI projects, and more. If you have any questions regarding the article above or missed something, feel free to email me. Asif@marktechpost.com

🚀 Check out 100’s of AI Tools at the AI ​​Tools Club