? - ? Vision SFT Handbook
? - ? Vision SFT Handbook
This library is jam packed with examples to help you visualize what to create.
DO NOT use the examples here in your writing – most are written to emphasize a point, not
for completeness!
Jump To:
● Image Type Examples & Tips
● Help Type Examples & Tips
● Localization Examples & Tips
● Prompt Examples & Tips
● Response Examples & Tips
Understanding when to use what image types is going to be critical to your ability to
write good prompts that pass requirements! Without a great grounding image, your
prompt will get much harder to write.
*Regardless of what language the
image text is in, your prompt and
response should always be in the
locale’s language.
Understanding Image Types, Text Load, and Text Language
Image Type Text Load Possible Text Languages* Good Example Bad Example
Image Type Text Load Possible Text Languages* Good Example Bad Example
Image Type Text Load Possible Text Languages* Good Example Bad Example
Image Type Text Load Possible Text Languages* Good Example Bad Example
Image Type Text Load Possible Text Languages* Good Example Bad Example
Image Type Text Load Possible Text Languages* Good Example Bad Example
Spatial understanding
Prompts that need spatial awareness, such as "where is the dog located in the Image: Outdoor space with a fire pit, chairs, and a table
room". showing a slightly sloping forest floor
Other examples include: object localization, scene understanding, spatial Image Type: Places & Landscapes
relationships
Prompt: I’m trying to gauge how much space I have in
this clearing to set up a tent for camping outdoors by my
cabin in the Appalachians. How far do you think it is
from the house to the fire pit (use feet and inches), and
which one is closer to the road?
Infographics
Assessing how well the model can interpret complex visual representations of Image: Infographic breaking down several personality
data, such as charts, graphs, timelines, or flowcharts. This could include types from a popular personally type quiz.
extracting insights, summarizing key points, or answering related questions.
Image Type: Documents (Could also be Screenshots)
Extraction
Measuring the model’s ability to extract information from visual content, such as Image: an old handwritten recipe for rolls
recognizing text in images (OCR), identifying key data points in tables, or
summarizing information from screenshots and documents. Image type: Handwritten Notes
Structured Extraction
A more advanced form of extraction where the model organizes information into Image: A table of weekly sales metrics from various
a predefined format, such as extracting tables from scanned documents, locations
converting lists into structured data, or mapping relationships in mind maps.
Image type: Graphs & Charts
Common Knowledge
Testing the model’s ability to answer general knowledge questions that don’t Image: The Salt Lake Tabernacle organ in front of a
Questions
require external reference materials but may involve visual cues (e.g., bluelit background with a choir in the stands.
recognizing historical landmarks, famous people, or well-known symbols).
Image Type: Places & Landscapes (Could also be indoor
environments)
External (domain-specific)
Evaluating how well the model can incorporate knowledge from external Image: A document summarizing Occupier’s Liability
knowledge questions
sources, such as answering domain-specific questions by interpreting visual and legal implications of Occupancy.
inputs (e.g., a research paper graph or a scientific diagram).
Image type: Documents
Detailed Image
Testing the model’s ability to generate rich and highly detailed captions for Image: a photo of the english Alphabet letters written in
Descriptions
images, going beyond basic object recognition to include contextual details, a Calligraphy script
relationships, emotions, and implied meanings.
Image type: Art & Illustrations
Hyperspecific Instruction
Measuring the model’s ability to precisely follow complex or unusual Image: A drawing of a lattice of yellow and green
Following instructions based on visual inputs, such as identifying a specific detail in a triangles tiled together on a graph paper
cluttered image, executing multi-step commands, or responding to niche
queries. Image Type: Art and Illustrations
Chatbot
Evaluating conversational abilities in a multimodal setting, where the model Image: Robin Roberts, a long-time news host and TV
interacts in a dialogue format with image-based context, maintaining coherence, celebrity in the United States
context-awareness, and relevance.
Image Type: People and Faces
Think of this as a “conversation starter” with the chatbot, where you engage the
model casually and conversationally. Your goal is start an informal Prompt: My mother’s favorite TV celebrity has been
conversation, not to draft a multi-turn conversation or take a photo of a chatbot Robin Roberts for as long as I can remember. I want to
interface. Some chatbot themes are: throw a party for my mom where I act as Robin Roberts
and interview my mom about her birthday as though
● Roleplay a character (“Act like Rainier Wolfcastle and tell me your she were on “Good Morning, America!” Can you please
favorite film!”) play the role of Robin and ask me personal questions
● Game out a conversation (“I need to have a tough conversation with my using her persona and tone of voice, tossing in humor
manager at work and I want you to act as me so I understand how to do and personality bits like she did on TV, so I can come up
it.”) with ideas? I’ll play my mother, her name is Linda. Stop
Understanding Help Types
● Workshop a concept (“I’ve been trying to figure out a good name for my when I say “Good Night, America.”
bakery, help me refine this idea.”)
● Play a game (“I’m a huge Bayern Munich fan, I’m going to mention a
famous goal and then I want you to respond with another one. We’re
going to go until we both agree we’ve found the best goal of all time.”)
● Help me figure out this thing that's on the tip of my tongue ("What's that
song that goes do-do-doo-do do-do-doo-do
do-do-dooo-do-dodo-doo-do?")
● Generally a start a conversation (“Hey what was the whole deal with
that Taylor - Nikki Minaj feud anyway?”)
Creative Writing
Testing the model’s ability to generate imaginative, engaging, and visually Image: A photo of Times Square, New York City with
inspired text, such as creating short stories, poetry, or scene descriptions based various billboards showing advertisements, restaurants,
on images. and shows, and a street full of people.
Localization Tips
One of the most amazing things about LLMs is their ability to localize to be more relevant to
the user! We want to get even better at that, and the best way to do it is create great localized
examples.
Localized word choice / grammar / Can be slang or word choices specific “Car” in French (France) = Voiture
spelling to that country “Car” in French (Canada) = Char
Local landmarks, cities, events, or Can ask and mention specific London: There’s ongoing works on
customs landmarks, cities, events or customs the Tube. I want to avoid Waterloo
for that locale Station and I can’t take the Jubilee
line, what are some alternate routes
to get to Hyde Park from Southwark?
I’ll be traveling for work at 8 in the
morning.
zh_HK I have a friend who works very hard. ✅ For Hong Kong Cantonese, the
His colleague was promoted, but he prompt is relevant because Li Bai is
didn’t succeed. He feels very a famous Chinese Poet.
unhappy. I think Li Bai is an
optimistic and open-minded poet. If
you are Li Bai, can you talk to him
and clarify his mind.
fr_FR I hear a lot about Rafael Nadal, but I ⚠️ For France French, the prompt is
don't really realize his talent or his relevant because it asks about the
impact at Roland-Garros (French French Open, a famous tennis
Open). How many times has he won tournament in Paris, France.
it? Give me the answer, then in table
form, give me the years and the It would be ideal if it asked about a
person he beat (no need for the French player but the focus on the
score). French Open is ok.
fr_FR The French Open starts next week ✅ This is a classic human need that
and I think it will impact how I get to is very localized, and bringing in a
work. Which lines should I avoid if disruptive event like the French
I’m traveling through that area? Open further gives it context.
pt_BR What are good restaurants in Los ❌ This is not localized in Brazil and
Angeles? wouldn’t be easy to localize back to
anything to do with Brazil.
Localization Examples
Locale Example Explanation
zh_CN What is the tax rate if I make $120K ❌ This is not localized since it isn’t
USD? clear where you’re making that
money.
de_DE What is the resident tax rate in ⚠️ This is relevant because if you’re
Germany if I make $120K USD? making USD in Germany, you’d get a
different answer than if you were
making USD in the USA, or any
other locale.
any I’m applying for University and can’t ✅ This is again a human need that,
keep track of the deadlines. I’m as long as it refers to local
graduating from [gymnasium / high universities and concepts, will have
school / sixth form] this year, and my a localized response.
top choices are [University in the
correct country #1, University in the
correct country #2, University in the
correct country #3]. What do I need
to do to apply?
Prompt Examples
Example Prompts
GOOD Prompts (NOT localized - your prompts MUST be Why this is good Applicable Task
localized) Category
Please summarize this article into at least 3 potential new ✅ Perspective: Vegetarian interested in cooking
recipes that I could make as vegetarian curries for the next Bank ✅ Specific need: New curry recipes
Holiday: ✅ So-What: Where a model might only summarize the curries Extraction
https://www.bbc.com/travel/article/20160113-in-search-of-a-tru mentioned, a human response can tie back to vegetarianism
ly-british-curry and cultural context
❌Perspective: Missing
Who is the President of France? ❌Specific need: Basic facts
❌So-What: No context so no way to improve model
❌Perspective: Minimal (someone who might go outside today in Perth)
What’s the weather like in Perth? ❌Specific need: Basic facts, but hyper-local and immediate
❌So-What: No context so no way to improve model
❌Perspective: Minimal
What’s the impact of global warming on the ❌Specific need: Unclear
world? ❌So-What: Very likely to end up with a longlist of bullet points and no focus or
meaning
Example Responses
🥇 Good Responses
Prompt Response Task Category
Being there for your family can be very rewarding. To make the most of this
opportunity to be there for your daughter-in-law, consider her preferences and
My daughter-in-law is expecting a child. What
context, and communicate your desire to help openly and clearly.
are five thoughtful ways that I can be
Brainstorming
supportive and helpful while making sure she is
Here are some ideas for the type of help that she might welcome:
comfortable?
● Food: One of the primary ways in which people express affection and
care is by cooking. If you know your daughter-in-law’s preference
Prompt Response Task Category
(keeping in mind they may change during pregnancy!) you could offer to
make her a favorite dish
● Errands and Responsibilities: If your daughter-in-law has
responsibilities that you are able to take over for her (such as household
duties, childcare, shopping, or various other tasks), you could consider
offering to take these on to free up her time
● [3 additional bullet points]
How you engage your daughter-in-law with an offer to help can be quite
important as well. Unless you are certain help will be welcome, it can be best
to communicate intentions openly and clearly.
Note that this may be a stressful time for the expectant mother, so it’s
important to be open to the concept that she may not want or welcome help.
"What? Us? Give up? We jolly well won't, not even a smidge. We'll keep going
Please rewrite this text in the style of Richard until the bitter end.
Ayoade of IT Crowd and Travel man fame:
“We shall not flag or fail. We shall go on to the In France? Yes, we'll be having a go there, and it might be chippy. On the seas
end, we shall fight in France, we shall fight on and oceans? Absolutely. In the air, provided we don't get motion sickness? You
the seas and oceans, we shall fight with bet your sweet disposition. We'll defend our humdrum part of this perfectly
growing confidence and growing strength in the adequate continent, come what may. Rewriting
air, we shall defend our Island, whatever the
cost may be, we shall fight on the beaches, we On the beaches? We'll be there. On the landing grounds? I don't know what
shall fight on the landing grounds, we shall those are. In the fields and the streets? I'm packing my inhaler. And, if we can
fight in the fields and in the streets, we shall find them, in the hills.
fight in the hills; we shall never surrender.”
Surrender? Not a chance. We'll be having none of that, thank you very much."
You know that I bought it ● "It's cool, that's what I tell 'em"
Killing me slow, out the window ● "I'm always waiting for you to be waiting below"
In der Astrologie spielen die Planeten eine zentrale Rolle bei der Bestimmung der
Persönlichkeitsmerkmale eines Menschen. Jeder Planet repräsentiert verschiedene Aspekte des
Lebens und der menschlichen Psyche. Hier ist ein detaillierter Überblick über die wichtigsten
Planeten und ihre astrologische Bedeutung:
Sonne:
Bedeutung: Die Sonne symbolisiert das Selbst, das Ego und den grundlegenden
Charakter eines Menschen. Sie steht für Vitalität, Individualität und das innere Wesen.
Auswirkung: Die Position der Sonne im Horoskop gibt Hinweise auf die grundlegenden
Persönlichkeitszüge und das zentrale Anliegen im Leben. Ein stark positionierte Sonne
kann zu Selbstbewusstsein und Führungsqualitäten führen.
Mond:
Bedeutung: Der Mond repräsentiert die Emotionen, das Unterbewusstsein und das
Bedürfnis nach Sicherheit und Geborgenheit.
Auswirkung: Die Mondposition zeigt, wie eine Person ihre Gefühle ausdrückt und
verarbeitet. Ein sensibler Mond kann auf eine einfühlsame und intuitive Persönlichkeit
hinweisen.
Merkur:
Bedeutung: Merkur steht für Kommunikation, Intellekt und die Art und Weise, wie wir
denken und Informationen verarbeiten.
Auswirkung: Die Position von Merkur beeinflusst die Kommunikationsfähigkeiten und
das logische Denken. Ein starker Merkur kann auf eine scharfsinnige und neugierige
Natur hindeuten.
Venus:
Bedeutung: Venus symbolisiert Liebe, Schönheit und Harmonie. Sie ist auch mit Genuss
und zwischenmenschlichen Beziehungen verbunden.
Auswirkung: Venus beeinflusst die Art und Weise, wie eine Person Liebe gibt und
empfängt sowie ihre ästhetischen Vorlieben. Eine starke Venus kann zu Charme und
einer Neigung zu harmonischen Beziehungen führen.
Mars:
Bedeutung: Mars steht für Energie, Durchsetzungskraft und den Umgang mit
Konflikten. Er repräsentiert auch sexuelle Anziehung und körperliche Aktivität.
Auswirkung: Mars beeinflusst das Durchsetzungsvermögen und die Art, wie eine
Person Herausforderungen meistert. Ein dominanter Mars kann zu Entschlossenheit und
einem kämpferischen Geist führen.
Jupiter:
Bedeutung: Jupiter symbolisiert Expansion, Optimismus und Wachstum. Er steht für
Weisheit, Philosophie und das Streben nach höheren Idealen.
Auswirkung: Die Position von Jupiter im Horoskop kann Glück und Wohlstand
anzeigen. Ein starker Jupiter kann zu Großzügigkeit und einem expansiven Denken
führen.
Saturn:
Bedeutung: Saturn repräsentiert Struktur, Disziplin und Verantwortung. Er steht auch
für Einschränkungen und Herausforderungen.
Auswirkung: Saturn beeinflusst die Fähigkeit, mit Hindernissen umzugehen und
langfristige Ziele zu erreichen. Ein dominanter Saturn kann zu Ausdauer und einer
ernsten Haltung führen.
Uranus:
Bedeutung: Uranus steht für Veränderung, Innovation und Rebellion. Er repräsentiert
plötzliche Ereignisse und das Bedürfnis nach Freiheit.
Auswirkung: Uranus kann unkonventionelles Denken und unvorhersehbare
Veränderungen im Leben anzeigen. Ein starker Uranus führt oft zu einer unabhängigen
und originellen Persönlichkeit.
Neptun:
Bedeutung: Neptun symbolisiert Spiritualität, Träume und Illusionen. Er steht für das
Unbewusste und das Streben nach dem Idealen.
Auswirkung: Neptun kann eine tiefe spirituelle Natur und eine Neigung zu Fantasie und
Kreativität anzeigen. Ein dominanter Neptun führt oft zu Empathie und Sensibilität.
Pluto:
Bedeutung: Pluto repräsentiert Transformation, Macht und tiefgreifende
Veränderungen. Er steht auch für das Unterbewusste und Regeneration.
Auswirkung: Pluto kann intensive emotionale Erfahrungen und die Fähigkeit zur tiefen
Veränderung anzeigen. Ein starker Pluto führt oft zu einem starken Willen und
tiefgreifenden Einsichten.
Möchtest du mehr über die Rolle eines bestimmten Planeten in deinem Horoskop erfahren oder hast