Chain of Thought Prompting

Chain of Thought (CoT) prompting is a technique that encourages LLMs to break down complex problems into intermediate reasoning steps before giving a final answer. Standard prompting asks the model to jump directly to an answer, which works well for simple tasks but often fails on multi-step reasoning tasks such as maths problems, logic puzzles, or multi-hop question answering. CoT prompting fixes this by instructing the model to "think step by step", producing a visible chain of reasoning that leads to a more accurate conclusion.

Chain of Thought was popularised in a 2022 Google research paper which showed that simply adding the phrase "Let us think step by step" to a prompt dramatically improved accuracy on reasoning benchmarks. There are two main variants: zero-shot CoT (just adding the magic phrase) and few-shot CoT (providing worked examples that show the reasoning steps). Few-shot CoT generally produces more reliable results because the model learns exactly what reasoning style you want.

CoT is especially effective for maths, logic, multi-step code analysis, and any task where getting intermediate steps right is critical to getting the final answer right. The below example shows the difference between standard prompting and chain of thought prompting on a multi-step word problem.

It gives the following output,

Standard prompt answer:
$259,200

Chain of Thought answer:
Step 1: Find widgets per shift.
  240 widgets/hour x 8 hours = 1,920 widgets per shift

Step 2: Find widgets per day.
  1,920 widgets/shift x 3 shifts = 5,760 widgets per day

Step 3: Find widgets per week.
  5,760 widgets/day x 5 days = 28,800 widgets per week

Step 4: Calculate weekly revenue.
  28,800 widgets x $4.50 = $129,600

The total weekly revenue is $129,600.

The standard prompt gave the wrong answer ($259,200) while chain of thought produced the correct answer ($129,600) by working through each step carefully. The below example shows few-shot CoT where you provide a worked example to guide the model on the reasoning style.

It gives the following output,

Few-shot CoT result:
Step 1: Travel time = 12:30 PM - 9:00 AM = 3.5 hours
Step 2: Distance = speed x time = 90 km/h x 3.5 hours = 315 km
Answer: 315 km

Chain of Thought prompting is now built into many LLM APIs. OpenAI o1 and o3 models, for instance, perform extended internal reasoning automatically before producing an answer, without needing any explicit CoT instruction in the prompt. For standard GPT models, adding "Let us think step by step" or "Think through this carefully before answering" remains a simple and highly effective technique for improving accuracy on any multi-step reasoning task.