How the AI cookie crumbles…

An experiment in AI content development

With more and more AI content writers and marketers flocking to sites like Fiverr and Upwork to work on AI content gigs, we wanted to try out a little experiment on how AI content stacks up to human content and see if we could provide some insight to our readers on managing AI content development pitfalls. Hence, this blog.

We experimented by generating an AI-generated salted butter and chocolate chunk shortbread recipe using ChatGPT. Then, we compared it against Alison Roman’s Salted Butter and Chocolate Chunk Shortbread recipe, commonly described among food bloggers as “The Cookie.”

A talented amateur baker (who makes pizza and bread every weekend) cooked both cookies on the same day using the same equipment. The same test group tasted both cookies on the same day without being told which cookie was AI. Below are some takeaways and advice for content marketers, as well as some honest feedback and quotations from taste testers and the amateur baker.

Here is the AI recipe, if you would like to compare:

Salty Dark Chocolate Chunk Shortbread Cookies

Instructions:

Absolutely! Here’s a recipe for salty dark chocolate chunk shortbread cookies, which are rich, buttery, and perfectly complemented by chunks of dark chocolate and a sprinkle of sea salt.

Salty Dark Chocolate Chunk Shortbread Cookies

Ingredients:

• 1 cup (2 sticks) unsalted butter, softened
• 1/2 cup powdered sugar
• 2 cups all-purpose flour
• 1/4 cup cocoa powder (preferably Dutch-processed)
• 1/2 teaspoon salt
• 4 ounces dark chocolate (60-70% cocoa), chopped into small chunks
• Flaky sea salt, for sprinkling

Instructions:

1. Prepare the Dough:
   • In a large mixing bowl, cream together the softened butter and sugar until smooth and fluffy using a hand mixer or stand mixer fitted with the paddle attachment.
2. Incorporate Dry Ingredients:
   • In a separate bowl, whisk together the flour, cocoa powder, and salt until well combined.
   • Gradually add the dry ingredients to the butter mixture, mixing on low speed until a dough forms and begins to come together.
3. Add Dark Chocolate Chunks:
   • Fold in the chopped dark chocolate chunks using a spatula or mixer on low speed until evenly distributed throughout the dough.
4. Shape the Cookies:
   • Divide the dough in half. Place each half on a sheet of plastic wrap or parchment paper. Shape each portion into a log about 1.5 inches in diameter.
   • Wrap the logs tightly in the plastic wrap or parchment paper, twisting the ends to seal. Chill in the refrigerator for at least 1-2 hours, or until firm (you can also freeze them for later use).
5. Preheat and Prepare Oven:
   • Preheat your oven to 350°F (175°C). Line baking sheets with parchment paper or silicone baking mats.
6. Slice and Bake:
   • Remove the chilled dough logs from the refrigerator. Unwrap and slice into rounds about 1/4 to 1/2 inch thick.
   • Place the slices onto the prepared baking sheets, spacing them about 1 inch apart.
7. Bake:
   • Bake the cookies in the preheated oven for 12-15 minutes, or until the edges are just starting to firm up and the tops look set.
   • Remove from the oven and let the cookies cool on the baking sheet for 5 minutes, then transfer them to a wire rack to cool completely.
8. Finish:
   • Once cooled, sprinkle the tops of the cookies with a pinch of flaky sea salt.

AI Content Development Risks and Advice

In this section, we are going to start with the result and then provide some advice on what you should know about AI content development. The result was that everyone disliked the AI recipe cookies. Aesthetically, they were also disgusting. You would be poorly reviewed if someone hired you to write content that a blogger could use to bake quickly.

Here are some choice quotations from this experiment:

• “Half – disintegrating mush in my mouth”
• “Edible but taste very weird” (our amateur baker explained that this had to do with several of the poorly written instructions)
• “Something is very very off. If I downloaded this recipe from a baking blog I would be pissed. Butter and chocolate are expensive.”

3 Key Takeaways for AI Content Development

When you use AI to develop content, be careful

Let’s figure out why, shall we? Here are the top takeaways for AI content writers.

TAKEAWAY 1

Using AI to develop content for specialized fields is more risky. Don’t use AI to generate content for specialized fields quickly. Avoid it as much as possible, or use the content you generate as a first draft and do extensive research before you submit it for your contract. In a specialized field, you must provide many steps in sequence. As a result, a poor result could backfire and cost people money.

Cooking is one example, and woodworking or DIY renovation is another. If the content is one where the human writer must have years of experience under their belt to develop content, don’t rely on AI to develop your content quickly.

If you need to develop content quickly using AI, you should spend time doing extensive research first, which means you use AI as a first draft, do the research, and then rewrite the content. As an example, the baker said the mistakes started at Step 1. Many of the steps defied very specialized baking rules. Every baking recipe should start with a direction to preheat the oven (which AI listed as Step 5) and include scraping down the sides of the bowl after whipping (not suggested at all). The use of powdered sugar contributed to the crumbling effect, and even though the prompt said “salted” chocolate shortbread, the butter in the recipe was unsalted. Do you see what we mean by specialized writing?

TAKEAWAY 2

Be careful about inspiration. The more specialized the area of expertise, the more likely it is that AI has scraped the original work. Alison Roman’s work is evident in the AI recipe but fell short in results. It is essential to be careful when using AI for content production in specialized fields. Another way to think about a specialized field is to consider whether the field has a very famous leader. For instance, do some quick research on this using AI or Google search.

TAKEAWAY 3

Use an AI Detector like AI Detector Pro before you submit your content. Because of how we’ve designed our AI Detector Pro, we can show you exactly where you can improve your content with editing. By reviewing where your content sounds “most AI,” you can quickly switch out vocabulary or do more research on the subject matter to rewrite the piece.