Landing Page Variant
A landing page variant is an alternate version of a page tested against the original (control) to see which one converts better.
Key takeaways
- The control supplies the baseline; the variant differs from it in a defined way.
- Random bucketing with sticky assignment stops traffic mix from explaining the difference.
- Every extra arm splits the sample and lengthens the time to a decision.
- Required sample follows the smallest lift worth acting on, not calendar convenience.
- A winner on blended traffic can lose inside individual sources or devices.
In depth
A variant is one rendered version of a page held in an experiment beside the control. Arriving visitors are bucketed by a hashed identifier, the assignment is stored so a return visit sees the same version, and every conversion is credited to the bucket that produced it. Because assignment is random, differences in device mix, traffic source and time of day distribute evenly across arms, so the remaining gap in conversion rate is attributable to the change itself rather than to who happened to arrive.
Traffic sets the ceiling on how many variants are useful. Splitting the same visitors across four arms leaves each arm a quarter of the sample and stretches the time to a decision considerably. The size of the change matters just as much. A headline that reframes the offer moves the metric enough to surface quickly; a button shade rarely does, and testing it burns weeks of traffic. Bold variants resolve faster but teach you less about which element did the work.
Working practice is to name the control, declare one primary metric before launch, compute the sample needed for the smallest lift worth acting on, and run through whole business cycles so weekday and weekend behaviour are both represented. A scorecard funnel suits this because its stages separate cleanly: the intro page that frames the quiz, the question sequence and its length, and the lead-capture step. Testing one stage while the others hold steady keeps the baseline stable and the reading interpretable.
The method breaks down on thin traffic. Below a few hundred conversions per arm most observed differences are noise, and running the test anyway produces confident-looking nonsense that hardens into team folklore. Variants also cannot judge effects that appear after the measurement window, such as a version that wins more form fills while attracting leads who never buy. Novelty distorts returning-visitor segments early on, and a winner on blended traffic can be a loser inside individual sources.
Example in practice
How to measure it
Read conversions per arm rather than visitors per arm, since the conversion count drives how precise the estimate is. Alongside the point difference, look at how wide the interval around it is: a variant showing a large lift with an interval spanning zero has not decided anything. Check the split actually delivered matches the split you configured, because a mismatch usually signals a bucketing or redirect bug rather than a real result.
After the test closes, segment the same data by source, device and new versus returning to see whether the result holds everywhere or was carried by one slice. Then follow the winner downstream. Compare qualified leads, booked calls or revenue per thousand visitors, not just submissions, so a variant that lifted form fills by lowering the bar gets caught before it becomes the new control.
Common mistakes
The dominant failure is peeking. Teams watch the dashboard daily and stop the moment the variant pulls ahead, but early gaps wander widely, so an arbitrary stopping point manufactures winners that evaporate in production. Decide the sample size and end date before launch and hold to them, or adopt a sequential method designed for interim looks. Either approach is defensible; watching a fixed-horizon test in real time and stopping on a good day is not.
The second is the variant that changes six things at once and then gets promoted wholesale. The team learns that a page won without learning which part earned it, and quietly carries the losing elements into every future page. Client-side swapping adds a related problem: the control paints first and flickers into the variant, which depresses the variant's own numbers. Render variants server side, or accept that you are measuring the flicker too.
Frequently asked questions
What should a variant change?
Test high-impact elements first: the headline, the primary call-to-action, the hero, or form length. Changing too many things at once means you cannot tell which change drove the result.
How many landing page variants can you test at once?
As many as your traffic supports, which for most B2B pages means two, occasionally three. Each additional arm divides the sample, so four arms need roughly four times the traffic of a straight A/B to reach the same precision. If a test would take more than a month or two to resolve, cut the arms rather than the required sample.
How long should a landing page test run?
Until it reaches the sample size you calculated before launch, and never less than one full business cycle. Weekday and weekend traffic behave differently, and stopping mid-week bakes that difference into the result. Set the end date up front. If the calculated sample will take far too long, the change you are testing is probably too small to be worth testing.
What is the difference between a variant and a redesign?
A variant is a controlled change measured against a live baseline; a redesign replaces the baseline outright. Redesigns can still be tested as one large variant, and often should be, but the result tells you only whether the new page is better overall. It cannot say which of the twenty changes carried it or which quietly hurt.
How do you know a variant has actually won?
When the test reaches its pre-declared sample, the difference on the primary metric is larger than the interval around it, and the delivered traffic split matches the configured one. Then check that the result survives segmentation by source and device. A variant that wins only on one channel is a channel-specific finding, not a page-wide winner.
Should you test one element or a whole new page?
Test one element when you want to learn a transferable lesson, and a whole page when you need a decision fast and have limited traffic. Single-element tests isolate cause but require larger samples because the effect is smaller. Bold page-level variants resolve quicker at the cost of ambiguity about what actually moved the number.
What do you do with a losing variant?
Retire it and record why it lost, including the segment breakdown. A loss on the primary metric is still information: it narrows what your audience responds to and stops the same idea returning next quarter. Keep the losing version accessible for a while, since a variant that lost overall occasionally won decisively in a single source worth pursuing separately.