Performance Max creative A/B testing: what to test and when to trust the result

A Performance Max asset A/B test can tell you whether one set of creative beat another inside one asset group. It can't tell you much if the question was vague, the assets changed partway through or someone read the result before purchases finished arriving. So settle three things before you launch: the creative decision the result will change, the business metric that decides it, and what you'll do if the answer comes back inconclusive. Keeping your current creative after an inconclusive test is a perfectly good outcome.
Timing matters more than usual this time of year. Google recommends running these tests for at least four to six weeks, and the assets under test are locked for the duration, so a test launched in late October runs straight through Black Friday (November 27) and Cyber Monday (November 30). Below is how we'd plan one for a hypothetical homewares retailer, and how we'd read the result.
What Google's asset test compares
Google's asset A/B testing experiment is still labeled beta. It splits one asset group's traffic between two sets of creative:
- Control: a selection of your existing assets, used as the reference.
- Treatment: the alternative set, which can include other existing assets or new ones you upload for the test.
- Common: every asset in the group you don't assign to either arm. These keep serving to all of the campaign's traffic.
Each experiment tests a single asset group, and there are two ways to set one up. From Campaigns, open Experiments, add a new one, choose Assets, then Performance Max and Any assets, and pick the campaign and asset group. Or open the asset group's assets for editing and turn on "Save asset group changes as an A/B test." Either way you set the traffic split and schedule, and Google suggests an end date that you can change.
Google's troubleshooting notes list a few snags worth checking before you pay for a photo shoot. A shared budget has to come off the campaign if it's part of a portfolio bid strategy or uses Smart Bidding Exploration, and an overlapping experiment can block a new one. Both arms also count toward the asset group's asset limits.
Choose a difference worth paying to learn about
The best test is one where either answer changes what you produce next. Product-only photography against lifestyle photography is a good example: if lifestyle wins clearly, you commission more of it, and if it doesn't, you've saved that budget. Benefit-led copy against customer proof works too, as long as the proof is real and you're cleared to use it.
Keep the difference between the arms to one idea. If the treatment changes the photos, the headlines and the offer at once, a win tells you the package worked and nothing about which part did the work. Sometimes the package is the decision you actually face, and that's fine, as long as the brief says so and nobody later reads the result as a verdict on photography.
Pick the deciding metric before launch too. For most ecommerce campaigns we'd use purchase conversion value, provided purchase values are tracked reliably, with ROAS and margin as guardrails. Add-to-carts and page views answer a different question. The experiment report shows clicks, CTR, cost, impressions and all conversions by default, so add the conversion metrics you agreed on before anyone starts reading it.
A worked plan: product photos versus lifestyle photos
Here's a hypothetical homewares retailer deciding whether next season's photography brief should show products in use. The example uses no client data and predicts no result.
Its hypothesis is that lifestyle images help shoppers picture the product at home, which should raise purchase conversion value while ROAS stays above the floor finance has agreed. Both arms show the same products accurately.
| Planning choice | What the retailer writes down |
|---|---|
| Control | The current product-only images. |
| Treatment | Lifestyle images of the same products, in the same aspect ratios. |
| Common assets | The same approved headlines, descriptions, logos and videos. |
| Held steady | Product selection, offer and landing pages. Anything that has to change gets logged with its date. |
| Traffic split | 50/50, which gives each arm as much data as the budget allows. |
| Deciding metric | Purchase conversion value, with ROAS and margin as guardrails. |
| Adopt the treatment if | The completed comparison shows higher purchase value with acceptable uncertainty, and both guardrails hold. |
| If the result is inconclusive | Keep the control, record what stayed uncertain and decide whether a second test is worth the spend. |
The retailer also writes down the smallest lift that would justify reshooting the catalog. That threshold comes from what the photography costs. A lift can be statistically real and still too small to pay for new production.
One thing to understand about the split: Google divides auction opportunities between the arms, so a 50/50 split doesn't guarantee equal spend or impressions, and the percentage can't be changed once the experiment is set up. Compare the arms in the experiment report instead of eyeballing which one booked more revenue.
Give purchases time to arrive
Google recommends at least four to six weeks for these tests, longer when your conversion delay is long, and its experiments FAQ suggests waiting one to two conversion cycles before judging. New experiments also have a ramp-up period that Google puts at 7 to 14 days. Higher volume helps, but no amount of calendar guarantees a clear answer.
Check the campaign's time to conversion before you schedule anything. If a good share of purchases land several days after the click, the last stretch of any window will understate both arms. Read a mature window, the same one for both arms, and note which recent days are still filling in.
Set the review date before launch and hold to it. During the test, watch for the things that actually break it: a purchase tag that stops firing, a treatment asset that gets disapproved, a hero product that sells out. An early lead for the treatment isn't on that list, and it's no reason to stop early.
Check the holiday calendar before locking creative
Once the experiment starts, you can't add, edit or remove assets in the asset group being tested, and its asset pages go view-only until the test ends. New treatment assets go through standard policy review, and a disapproved asset won't serve in the experiment.
That lock makes a late-October launch an operations decision as much as a marketing one. Lay the planned end date over every offer change, price change and creative expiry between now and early December. A treatment image carrying a Black Friday badge will be wrong by December 1, so that test can't run as designed. Either make the assets accurate for the whole window or wait until January.
Evergreen photography, like our hypothetical retailer's, can usually run through the holidays. Read the result for the period it covers, though. Gift buyers responding to a holiday offer may prefer different imagery from full-price shoppers in February, so a winner in this window is evidence about holiday creative. If you want a year-round answer, plan a second test in a normal trading period, and resist trimming the finished data down to the few days where the treatment looked best.
Keep the test readable while the account keeps running
Google's Smart Bidding documentation says budget, ROI-target and creative changes never trigger the formal Learning status or remove historical data. Its experiments FAQ is more cautious about tests in particular. It recommends making as few changes as possible while an experiment runs and warns that budget or target changes can contaminate the results. We read those as answers to different questions: a budget change won't reset the bid strategy, and it can still shift the comparison you're trying to read.
So keep a change log. If you raise budgets for Cyber Week, move a ROAS target or swap a landing page during the test, record the date and narrow the conclusion to match. Our Google Ads channel-prioritization article covers where that spend goes.
Review the evidence before adopting the treatment
Open the experiment summary and confirm the dates and the deciding metric first. The report shows the estimated difference between the arms with a confidence interval, and it marks a statistically significant result with a blue asterisk. Google's reporting documentation says the default confidence level is 80% and that you can choose your own. Choose it before launch. We'd use a stricter level than the default when the decision means reshooting a catalog, and whatever you pick, changing it afterward to produce a winner changes your decision rule.
Suppose the retailer sees purchase value up in the lifestyle arm, but the interval runs from a loss to a gain. The direction hasn't been settled. The team may still prefer the lifestyle photos, which is a fine reason to shoot some, but the test hasn't shown that they sell more.
| What the completed comparison supports | What we'd do |
|---|---|
| The treatment improves the deciding metric with adequate statistical evidence, and the guardrails pass | Adopt it, then keep watching performance under normal trading conditions. |
| The point estimate is positive, but the interval includes no improvement | Record an inconclusive result. Extend only if conditions stay stable and the remaining budget can buy a useful comparison. |
| Purchase value rises, but ROAS or margin falls below the floor | Keep the guardrail in the decision and work out where the tradeoff comes from before adopting anything. |
| Tracking failed, or the treatment couldn't serve as planned | Treat the comparison as compromised. Fix the input before drawing any creative conclusion. |
Watch the auto-apply setting. When you create the test from the asset group, "Apply these changes to the original asset group if the treatment arm wins" is on by default, and it takes effect when the experiment ends if the treatment performed better than the control. Turn it off if you want a person to make the call. Choosing Apply updates the original asset group. Choosing End stops the test without changes, discards the new treatment assets and returns the group to its original state, and Google says experiments ended manually aren't auto-applied. Name the person who owns that decision before launch, especially when the end date lands in the middle of a promotion.
Turn the result into the next creative brief
When the test ends, write one short record: the question, the assets in each arm, the window you read, the result with its interval, what changed in the account along the way and what you decided. If the result was inconclusive, add what would make a follow-up worth running. That record is what stops a promising chart from becoming a production mandate six months later, when nobody remembers the caveats.
LimeLight's paid media team can help you scope a creative test, fit it around your promotion calendar and read the result. If you're weighing a test against holiday operations or next year's production budget, talk with us about the decision.


