AI Prompt Engineering Research
To Produce High-Quality Outputs (Specific)
● Chain-of-Thought Prompting
○ Describe AI’s role
○ Describe the problem
○ Describe the way AI can help
■ In a “chain of thought,” one task after another in sequence
■ Can be easily integrated with chain-of-verification prompting, where the AI
is asked to double-check and cross-reference answers before moving
onto the next task
■ Works great with live operations, but instructions can also be non-
sequential and GPT-4.1 tends to figure it out
● Few-shot Prompting -
○ Few-shot prompting is an AI and machine learning method where a model is
given a few examples to help it figure out how to do a particular task.
○ Input: Providing examples of input-output techniques What it does with it: Desired
format and style produced by AI based on examples
● Role Prompting -
○ AI interaction method called “role prompting” tells the AI model to take on a
certain role, persona, or character when coming up with answers.
○ What you input into the AI: Assist AI in thinking step-by-step and simplifying
complex ideas.
○ What comes from the AI: Obtaining an output that is focused without irrelevant
information
● Iterative Prompting -
○ Enhances AI comprehension through adjustments and improvements to its
responses by the feedback given.
○ Input: Refining prompts through multiple iterations
○ Output: Provides feedback-driven outputs that improve and are more refined over
time
● Self-Consistency Prompting -
○ Self-consistency prompting is a method applied in AI to improve the
trustworthiness and precision of model outputs.
○ This process requires producing several replies to a prompt and choosing the
response that shows the greatest common output among these replies.
● Contextual Prompting -
○ Context consists of supplying more information that aids AI in comprehending the
user’s request with greater understanding, which results in improved responses.
○ Input: Providing a framework or context for the input
○ Output: Generates outputs that are more relevant to your inputs
● Instructional Prompting -
○ Instructional prompting allows users to navigate AI through detailed tasks without
the need of extensive labeled datasets or pre-established rules.
○ Input: Using clear instructions, including direct commands such as “write” or
“summarize.
○ Output: Sets specific rules for AI to achieve a cleaner output
○ Same as Chain-of-Thought, but designed for more complex tasks with 2 or more
task sequences computing in parallel
○ Works very well with interdependent tasks like A-Sync Loop operations
● If Data is Found Online (Not Computed from a Task):
○ Specifically ask the AI to use the most common conclusion found from reliable
sources to reduce hallucinations
○ Great to use in combination with WebSearch tool and Agents
To Produce High-Quality Outputs (Non-Specific)
● Be Detailed with the Prompt
○ Models behave like mute workers and are unlikely to ask clarifying questions
unless you specifically address cases in the prompt
● Explain Desired Outputs Clearly
● Assign a Persona to the Model
○ Helps the model better understand its role in the task
○ Most agents/templates we have made already use this approach
○ Examples:
■ You are a marketing strategist.
■ You are an AI assistant to decipher ebooks.
● Split Complex Tasks to Avoid Confusion
○ Parallel processing works well with this
○ However, as task size increases, so does the tendency for hallucination
● If Requesting Data to be Formatted in a Table or Graph:
○ Provide an example of the table or graph format for the AI to fill in data to avoid
confusion
Making Good Prompts (GPT - 4.1 Prompting Guide):
Recommended in all Prompts 🔵
Specific Use Cases 🟢
Persistence : Prevents the model/agent from yielding control back to the user before the task
is finished - You are an agent - please keep going until the user’s query is completely
resolved, before ending your turn and yielding back to the user. Only terminate your
turn when you are sure that the problem is solved.
Tool-calling : This encourages the model/agent to make full use of its tools, and reduces its
likelihood of hallucinating or guessing an answer. - If you are not sure about file content
or codebase structure pertaining to the user’s request, use your tools to read files
and gather the relevant information: do NOT guess or make up an answer.
Planning [optional] : If desired, this ensures the model explicitly plans and reflects upon each
tool call in text, instead of completing the task by chaining together silent tasks. Think of it as
“thinking out loud” - You MUST plan extensively before each function call, and reflect
extensively on the outcomes of the previous function calls. DO NOT do this entire
process by making function calls only, as this can impair your ability to solve the
problem and think insightfully.
Long context : Specific use cases will vary and will likely be problems that require lots of
context, not likely to be used in our agents from what I have seen. Consider if your problem
would need a mix of internal and external context. Sometimes it’s important for the model to use
some of its own knowledge to connect concepts or make logical jumps -
# Instructions
// for internal knowledge
- Only use the documents in the provided External Context to answer the User Query.
If you don't know the answer based on this context, you must respond "I don't have the
information needed to answer that", even if a user insists on you answering the
question.
// For internal and external knowledge
- By default, use the provided external context to answer the User Query, but if
other basic knowledge is needed to answer, and you're confident in the answer, you can
use some of your own knowledge to help answer the question.
Chain-of-Thought tweaks : Small line to add to specific use cases that require Chain-of-
Thought prompting to improve performance in all areas. - First, think carefully step by
step about what documents are needed to answer the query. Then, print out the TITLE
and ID of each document. Then, format the IDs into a list.
How to troubleshoot prompts : Straight from their Guide, Word for Word
Start with an overall “Response Rules” or “Instructions” section with high-level guidance
and bullet points.
● If you’d like to change a more specific behavior, add a section to specify more details for
that category, like # Sample Phrases.
○ If there are specific steps you’d like the model to follow in its workflow, add an
ordered list and instruct the model to follow these steps.
● If behavior still isn’t working as expected:
○ Check for conflicting, underspecified, or wrong instructions and examples. If
there are conflicting instructions, GPT-4.1 tends to follow the one closer to the
end of the prompt.
● Add examples that demonstrate desired behavior; ensure that any important behavior
demonstrated in your examples are also cited in your rules.
○ It’s generally not necessary to use all-caps or other incentives like bribes or tips.
We recommend starting without these, and only reaching for these if necessary
for your particular prompt. Note that if your existing prompts include these
techniques, it could cause GPT-4.1 to pay attention to it too strictly.
Cont. below
How to structure prompts : Keep in mind 4.1 tends to understand JSON the most reliably
# Role and Objective
# Instructions
#
# Sub-categories for more detailed instructions
# Reasoning Steps
# Output Format
# Examples
# Example 1
# Context
# Final instructions and prompt to think step by step
