Here are 5 to look for when selecting a creative testing vendor:
1. Control group
Similar to tests in science or healthcare, the way to isolate impact is to include a control group. In ad testing, this means a group who sees a non-ad piece of content (the placebo) and answers the same questions as the group who sees your real ad.
Without a control group, it might be possible to tell if audiences have interest in your brand after seeing your ad. But it will be impossible to tell if that interest is due to your ad or if it already existed, which negates the purpose of ad testing.
Randomized controlled trials (RCTs) are a specific type of test that always includes a control group.
2. Speed. Can you get the test done before you decide which media to run?
Tests are only useful if they help you make better decisions.
In the past, marketers had to rely on post-campaign studies to tell how their campaigns shifted perception. With today’s pre-testing options, marketers can choose which campaigns and ideas to invest in before they launch, allowing them to make better decisions (and save media dollars).
3. Real participants — Are real people being surveyed or AI?
If you want to find out how your ad will perform for an audience, the best way to do so is simply to ask them. Some companies claim to replace respondents with artificial intelligence to predict how effective ads are. AI can do some amazing things (have you seen ChatGPT?). But even the best AI models can only ever pattern-match against what has done well in the past — they have no way to know what new creative ideas might work (which is where great advertising comes from).
4. Statistical power (aka, sample size and clear margins of error)
Statistically significant results require large sample sizes, typically in the thousands, and sophisticated analysis to prove impact and understand margins of error. Ask your vendor what their sample sizes are and how they calculate margins of error.
5. Real world outcomes
Tests are only as good as what they can predict. Ask your vendor to share validation studies linked to real-world outcomes. E.g., data which proves that their ad recommendations grew topline sales.
How does Swayables’ ad testing platform stack up?
- ✔️ Control group — Our tests always include a randomly allocated group who sees a placebo ad (at no extra cost), so you can isolate the impact of your creative assets.
- ✔️Speed — We test 1000s of respondents in just 24 hours, allowing brands like Truff, Paramount, and Amazon to measure campaigns quickly and launch new creative in market, only days after test.
- ✔️Real participants — Every test includes a large group of survey participants, along with an array of qualitative comments to explain not just impact, but why participants felt the way they did.
- ✔️Statistical power — All Swayable tests include a large sample size, clearly denoted margins of error, and levels of statistical significance.
- ✔️Outcomes — You bet. Bain & Company independently confirmed a 7% lift in sales after one of their CPG clients switched to messaging recommended by Swayable tests.
Interested in testing out Swayable’s creative testing for yourself? Request a demo here, and get results in 24 hours.
