[IsaacL]

I pretty much agree. The "scientific" approach the author pushes for in the article -- running experiments with multiple similar prompts on problems where you desire a short specific answer, and then running a statistical analysis -- doesn't really make much sense for problems where you want a long, detailed answer.

For things like creative writing, programming, summaries of historical events, producing basic analyses of countries/businesses/etc, I've found the incremental, trial-and-error approach to be best. For these problems, you have to expect that GPT will not reliably give you a perfect answer, and you will need to check and possibly edit its output. It can do a very good job at quickly generating multiple revisions, though.

My favourite example was having GPT write some fictional stories from the point of view of different animals. The stories were very creative but sounded a bit repetitive. By giving it specific follow-up prompts ("revise the above to include a more diverse array of light and dark events; include concrete descriptions of sights, sounds, tastes, smells, textures and other tangible things" -- my actual prompts were a lot longer) the quality of the results went way up. This did not require a "scientific" approach but instead knowledge of what characterized good creative writing. Trying out variants of these prompts would not have been useful. Instead, it was clear that:

- asking an initial prompt for background knowledge to set context - writing quite long prompts (for creative writing I saw better results with 2-3 paragraph prompts) - revising intelligently

Consistently led to better results.

On that note, this was the best resource I found for more complex prompting -- it details several techniques that you can "overlap" within one prompt:

https://learnprompting.org/docs/intro