🤖 AI vs Humans in Marketing

Blake Allsmith and Alex Bowman explore the dark arts of digital marketing and, in this first episode of the Allsmith Growth Show, find out who'll come out on top when AI content generators ChatGPT and Midjourney are pitted against a pair of human writers and graphic designers. Can Alex tell which of Blake's ads came from the humans and which ones came from a robot? And whose ads will end up getting the most clicks and conversions once they go live? Blake and Alex find out the answers to these questions and more as they look at how AI is transforming the creative industry, at the limitations and hidden biases it can reveal, and what it means for the future of performance marketing.

AI in the Headlines

As we start the episode, Blake and Alex share their experiences with ChatGPT and AI-generated content, from a popular AI recreation of Seinfeld on Twitch and a Decemberists song with lyrics and music by ChatGPT to Blake’s use of the app to help streamline a fundraising article. They discuss a recent Wall Street Journal article by Henry Kissinger, Eric, Schmidt, and Daniel Huttenlocher, and the article’s warning that we could end up trusting AI algorithms with too much of our everyday lives since their decisions can be so opaque and seemingly unquestionable. “It is scary,” Alex notes, “not because of what AI can actually do, but what I think people perceive AI can do.”

Let’s Play Humans vs. Robots

The experiment begins as Blake explains the rules of the contest: two human ad designers are working separately against ChatGPT and Midjourney's AI output on a very real advertising campaign for InsurGrid. For round one, ChatGPT's text output is presented alongside the human efforts. Can you guess which ad copy came from a robot?

If you thought that option three came from an AI, then congratulations: you correctly agreed with Alex. As he notes, the third option has ChatGPT’s distinctively rambling tone along with some odd and generic phrasing that helps give it away. The next test compared the AI art generator Midjourney’s output with two visual ads that were created by humans. Both the picture and the slogans on each ad came from either a human or Midjourney’s algorithms. Can you tell which one is which?

Don’t worry if you didn’t recognize the first option as the robot. Alex thought it might be the third choice since the sunglasses on the dancing man it portrays had an odd reflection, or perhaps the second one, which is just a straightforward Facebook carousel ad with a picture of a smartphone. The first choice, featuring an image of a professional young woman with a laptop, turned out to be Midjourney’s creation. Blake and Alex discuss some of the AI ad’s giveaways, such as a slogan that sounds generic compared to its human competition, but how the ad layout and choice of a smiling office worker nonetheless struck Alex as a very professional and intuitive design choice, too much so for a computer. Blake admits that his instruction to Midjourney to create an ad featuring a young woman may have given the AI an advantage that it wouldn’t have figured out on its own.

The final test isn’t to decide which one’s the robot - we’ve already figured that out - but which of the three ads, two made by human designers and one created by combining ChatGPT and Midjourney’s output, will actually succeed in generating the most conversions on a $500.00 budget. Which ad do you think will come out on top when it comes to buyers?

We know that the first option was generated by AI, but which ad did the best when they all went live? Do you agree with Alex that option three, with its lively image, short, simple phrasing, and eye-catching design, would have the highest conversion rate? If so, then you may be surprised by the result: the third option fared the worst of the three, with a cost-per-click rate of $1.28 and a conversion rate of just 9%. Options one and two were closer, but option one, the AI’s creation, did the best, with a conversion rate of 12% and cost-per-click of $1.13 compared to the second ad’s 12.5% conversion and $1.18 CPC.

The game concludes with Alex spotting the AI copy, missing the AI art, and with the robots outperforming the humans.

What Did We Learn?

Blake and Alex talk about some of the factors behind the AI’s surprising win. For one thing, as Blake notes, the first ad’s use of an attractive young woman could be a performance tiebreaker when it comes to InsurGrid’s target market. “I know that young female ads for this business have worked well in the past,” he explains, “because a lot of these [ads] are [targeted at] young men who are going to stop and scroll because of that. That’s stereotypical, but also supported by data.” They discuss how option three’s consciousness around diversity and inclusion seemed to be undercut by the AI’s more mercenary tactics: “It’s almost like a reversion to the 1950s Mad Men approach, which was like ‘hey, whatever it takes to sell more product.’”

“My personal belief,” Blake adds, “is that, maybe somewhere in the future, there is a world where AI is going to replace designers and creatives entirely. I don't think we're in that era yet. I do think that creatives who embrace AI are going to annihilate creatives who don't.” Alex agrees with Blake, and he points out that designers and robots don’t necessarily have to be at odds. “What would’ve happened,” he asks, considering the game’s outcome, “if you got those design ideas from ChatGPT and the designer went off with those instead? They may have had a better version than what ultimately won.”

Letting the Robots Decide

As the episode comes to a close, Alex and Blake discuss the future of AI in the creatives industry, and how a classic essay by Wendell Berry, “Why I Am Not Going to Buy a Computer,” warns us of the cost of letting robotic efficiency rule all our decisions. “AI does a really good job of filling in gaps,” Alex observes, “and you mentioned this earlier, but we don’t understand why it makes the decisions that it makes.” Blake agrees that we still need to use our own judgment, and he adds that “every time I download an app, I ask the question ‘what is this going to replace in my life, and is that a good thing?’ I think, in this instance, we need to do that with each use case of AI, but not AI as a whole.”

Episode Timestamps

• 00:00:00 Introduction
• 00:00:40 Can ChatGPT Replace Any Job?
• 00:01:40 AI in the Headlines
• 00:03:45 Having Faith in the Robots
• 00:05:39 Better Fundraising through AI
• 00:06:47 The Rules of the Game
• 00:08:09 How ChatGPT Creates Ad Copy
• 00:08:46 Using Midjourney to Generate Images
• 00:10:00 Let’s Play Robot vs. Humans
• 00:12:00 ChatGPT’s Ad Copy Revealed
• 00:12:32 Which Image is from Midjourney?
• 00:13:51 Revealing the AI Image
• 00:14:50 Which Ad Performed the Best?
• 00:16:36 The Final Score
• 00:18:04 Human Biases in Machine Learning
• 00:19:09 What Did We Learn?
• 00:21:04 Combining Human and AI Expertise
• 00:23:37 Short-term Profit vs. Long-term Branding
• 00:25:38 Wrap-up
• 00:27:12 Knowing When the Robot’s Wrong
• 00:27:46 Balancing Happiness and AI Efficiency
• 00:29:01 Send in Your Experiment Requests
  

Referenced Articles

“ChatGPT Heralds an Intellectual Revolution,” by Henry Kissinger, Eric Schmidt, and Daniel Huttenlocher (WSJ)

“I Had ChatGPT Write a Decemberists Song,” by Colin Meloy (Substack)

“Why I Am Not Going to Buy a Computer,” by Wendell Berry (Matthew J. Brown)

Other Resources

Introducing ChatGPT, a prototype GPT3.5 chatbot and text generator

Midjourney, an AI art generator that creates images based on text descriptions

Nothing, Forever, a ChatGPT-generated Seinfeld Twitch stream

‍