I often want the API to send back a response that is something I can parse programmatically. For example, I want it to summarize a passage of text, providing comments, where each comment is a member of a JSON array.
It’s frustrating. Something the response will have “JSON:” prepended to it. Sometimes it won’t. Sometimes there will be some other sort of preamble to explain that it’s JSON.
A pretty common feature of API’s is to be able to specify a response format. It would make things much more usable for developers here.
Have you tried putting in the prompt to respond with certain parts as JSON? For a similar problem with math expressions, asking for LaTeX is working fairly well. (ref)
It also seems that the more ChatGPT is asked to use LaTeX for math expressions the more it seems to know this, does it automatically and gets it correct, but that could be my wishful thinking.
Telling the model what to do(prompting) only 60% success rate or poor reliability. Give a few examples of your output style, you improve it to 90%. If you need 100% reliability in achieving JSON output and wish to parse it error-free, use 2 API calls, first one gets the text, and 2nd call formats the text in JSON format. If you get any parse error, you loop the generated text back into the 2nd API call the loop should only end if there is no error in parsing the JSON. This is one way I have used to ensure reliability when you are parsing the text to the front end and to avoid errors.
@RonaldGRuckus , that is interesting. I could train my own custom model to understand JSON output, and then use that model going forward. Is that possible with the chat endpoint? Seems like it’s not, or not yet:
The response is actually in JSON format, it just so happens that the text of the response is a text JSON value, so the completion response is a JSON text value.
I think it is more reliable to parse this text value and convert to JSON after the completion versus trying to prompt am stochastic language model to always output reliable JSON data.
I think you may be right. I’ve had more consistent luck asking it to respond in a numerical list, with each list item on it’s own line. That’s easy enough to parse into JSON with a little string hackery.
I have had reasonable success with asking ChatGPT to create/improve the questions that you ask it e.g. “Can you modify the question I asked you so that next time I ask it you will…” then, for example “… provide the answer in JSON format”. You can also ask chatGPT to compress the question. Here is an example I achieved using this technique. I wanted ChatGPT to provide topics and weightings for any input phrase I gave it, with the answer in the form of a JSON object. The resulting question was…
: Yesteday I bought myself a teapot. : Topics/keywords/people/places/things/dates/times. : 10+ topics. Output: JSON hash {topic: % weight} only. NoExplanation
The bit about “buying a teapot” is an example of the phrase I want it to categorize. The output I get is consistently just pure JSON e.g.
{
“teapot”: 0.8,
“yesterday”: 0.4,
“buying”: 0.3,
“self”: 0.3,
“household items”: 0.2,
“kitchenware”: 0.2,
“shopping”: 0.2,
“personal purchase”: 0.2,
“home goods”: 0.1,
“retail”: 0.1,
“April 16, 2023”: 0.1
}
I personally found few-shot examples and English description of the schema to be unreliable for my use case. Specifying the schema as a TypeScript interface finally did the trick for me. You can see some examples at ResuLLMe/__init__.py at main · IvanIsCoding/ResuLLMe · GitHub
In general, I recommend the following prompt style:
You are going to write a JSON <insert the rest of your instructions>
Now consider the following TypeScript Interface for the JSON schema:
interface MessageItem {
receiver: string;
sender: string;
content: string;
}
interface Message {
messages: MessageItem[];
}
Write the basics section according to the Message schema.
On the response, include only the JSON.
Typescript is the way to go. I also find using zod works really well too because you can validate the outputs at runtime and if there is an error feed it back in and let GPT reflect on it’s mistake.
How are you expressing line breaks, in your code, when providing your prompt? Currently I just have “\r\n” inside of a string and it does seem to work. But it’s confusing because obviously if I just log the request, I don’t see a new line, I see the literal “\r\n”.
From my personal experience, the stability of outputing JSON array is worse than JSON object, so instead of ["a","b","c"], I always ask the model to output {"0": "a", "1": "b", "2": "c"}, which has a better stability.
: Yesteday I bought myself a teapot. 
: Topics/keywords/people/places/things/dates/times. 
Output: JSON hash {topic: % weight} only. 
NoExplanation