A/B Testing for Push & Native: A Practical Framework

Why push and native need discipline

Push and native traffic is cheap per click and noisy per placement. A landing page that "wins" after 300 clicks often loses after 3,000. Most bad decisions in these verticals come from stopping a test too early or changing several things at once.

What to test first

Test the change with the largest expected effect, one at a time:

1. The offer. Two offers on the same landing page can differ by 2–3× in EPC.
2. The landing page angle. Quiz versus article versus direct, not button colours.
3. The pre-landing step. Whether to have one at all, and how long it is.
4. Details. Headlines, images and calls to action, once the big pieces are settled.

Running the test

In XPNSR Tracker an A/B test belongs to a campaign. Choose what you compare (landing pages, offers or the whole path), add the variants and set the split, 50/50 or 70/30 when you want to protect a proven control.

• Bind visitors. Switch it on so a returning visitor always sees the same variant; otherwise one person counts in both.
• Keep everything else fixed. Same GEO, sources, bids and schedule for all variants, for the whole test.
• Filter bots first. Bot clicks never convert and dilute every variant; run anti-fraud before you read results.

How much data is enough

Look at conversions, not clicks. As a rule of thumb you want at least 100 conversions per variant before trusting a difference of 20–30%. Smaller differences need far more. The tracker reports statistical significance for each test, so wait for it to say the result is significant instead of eyeballing the conversion rate.

After the test

Send all traffic to the winner, archive the loser and write down what you learned. Then test the next thing on the list. If you would rather not run each split by hand, the optimiser on Pro and higher shifts stream weights toward better performers automatically, and automation rules can pause variants that fall below an ROI threshold.