AI is one of those technologies that almost seems too simple to use. There's no fancy API. You just ask the LLM what you want and it spits out a result. For something so incredibly capable and complex, the APIs that you use are shockingly simple. But this, for simplicity, can be a bit of a curse. I was amazed at how easy it was to start integrating basic AI into my applications. And then amazed at how hard it was to build anything complex from there. This document contains some of the tips and tricks that I've learned over the months of working with AI. I'll keep updating it as I get a better understanding of how to do things properly. # Working with the output By default, AI's output results as unstructured strings of text. This is super convenient when you want to use the results yourself or feed them directly to users. Data structures become really important when you need to manipulate data. There are a few ways of forcing structured data ## Returning JSON The best way to parse and manipulate data is to receive a result in JSON format. There are two challenges to this. The first is receiving only JSON data. And the second is understanding the structure of the data that you receive. The easiest way to force GPTs to return JSON is to clarify that you are expecting a JSON response. Something along the lines of `"Return only a JSON response"`. Defining the structure also helps the GPT know to return just JSON. Most modern GPTs now also have the option to force a JSON response, which does a pretty good job of removing non-JSON cruft. I've struggled a little with how to define a structure for the JSON response that the LLM will consistently use. The surprise solution for me was to specify the response in a non-JSON format. Both TypeScript and Python typing seem to work very well. ``` The JSON must have the following format: interface Suggestions {{ suggestions: string[]; }} ``` or ``` The JSON must have the following format: class Response: suggestions: List[str] ``` ## Handling unexpected responses A well-defined query should return JSON that can be parsed and that can matches the proper structure, but there are still occasions when you still get an unexpected response or broken JSON. The best solution I've heard of is to wrap your JSON parser into an exception handler, and run a second query if you detect an error. This second query can be a repeat of the original query - ideally, with a counter and upper repeat limit so you don't end up in an infinite loop. In the event it's a broken query. Another solution that I like is to run an separate query that takes the broken response and asks the GPT to try to format it into valid JSON. OpenAI now has the ability to call functions via responses. This is a really good solution to handling responses programmatically. would probably be the way to go for most GPTs in the future.