Campaign A/B walkthrough
Run a campaign subject-line A/B test in RouteKite.
Prepare two subject and preview-text variants, choose a test audience and evidence threshold, and understand automatic winner selection, fallback, and blocked outcomes.
Reviewed October 4, 2026
A and B use the same published template. Subject and preview text can differ.
The launch requires the normal sender confirmation for open and click tracking.
The test launches a real cohort. A/B campaigns cannot be scheduled.
Ask one question the test can answer
RouteKite compares the observed engagement of two inbox presentations for one frozen campaign audience. It does not test different body designs, prices, products, or Automation branches.
For a subject-only experiment, keep the body and preview text the same and change only the subject. If you change subject and preview together, interpret the result as a comparison of those combined presentations.
Use a verified Sending Address, a published marketing template, completed required variables, and a permissioned audience. Check eligibility rather than relying on the list's raw contact count.
Normal campaigns default to analytics Off. Enable it deliberately for this experiment and review whether tracking is appropriate for your recipients. Neither a collected open nor a classified click proves who performed the action or that a person read the email.
Set up A and B in campaign creation
Sign in to the intended workspace, open Campaigns, and create a campaign. The A/B controls stay inside this workflow rather than in a separate testing product.
- Prepare Variant A
Complete the normal campaign details, audience, template, Subject, and Preview text. Variant A uses those existing Content values.
- Turn analytics On and enable the test
Select Test two subject lines. Add Variant B's subject and optional preview text. At least one must differ from Variant A.
- Choose the test audience
Select 10, 20, 30, 40, or 50 percent of eligible recipients. RouteKite assigns an even test cohort between A and B; the untouched remainder waits for the decision.
- Choose the evidence rule
Set Evaluation delay from 1 to 24 hours, Minimum delivered per variant from 25 upward, and the Winner metric. Choose Variant A or B as the Inconclusive fallback.
- Create and review the draft
Review sender, recipients, template variables, analytics, both variants, and the test settings. Creating a draft does not send the experiment.
Use the counts to judge whether the test is practical
A supported minimum is a validation limit, not a guarantee of statistical power or a useful business conclusion. Small or weakly engaged audiences often produce inconclusive results.
Illustrative setup: 2,000 eligible recipients, a 20 percent test, a 4-hour evaluation delay, a minimum of 100 delivered per variant, and Variant A as fallback. The cohort contains 400 recipients, 200 assigned to each variant, with 1,600 initially waiting.
Assigned is not Delivered. Bounces, current opt-outs, or sending restrictions can reduce the evidence and the eligible remainder. Review the real counts rather than copying the example into a smaller campaign.
Likely-human click rate uses classified click evidence relative to delivered recipients. Confirmed opener rate uses deduplicated eligible open or click evidence. Privacy proxies, automated requests, classification uncertainty, and missing observations limit what either rate establishes.
Confirm the real send, then follow the decision
Launching starts real delivery to the assigned test cohort and authorizes the selected variant to be sent to the eligible remainder. It is not a simulation or an email rendering test.
Complete the required open-and-click tracking confirmation before Confirm and send. Recipient messages use the normal sending allowance. After launch, the audience assignments and decision are immutable; the campaign cannot be converted back to a draft.
Campaign detail reports test, evaluation, and remainder progress. The evaluator waits for the configured window and fresh, classifier-correct evidence. Performance selection uses a fixed 95 percent threshold under RouteKite's statistical model, alongside the minimum delivered count.
That threshold is not a 95 percent guarantee of more sales, inbox placement, or the same outcome next time. Record the result with its audience, metric, window, and actual delivered counts.
Distinguish fallback from unavailable evidence
An inconclusive comparison and a failed evidence check are different outcomes. The system must not invent engagement or a performance winner to finish the campaign.
- Performance selection
- Fresh evidence meets the minimum and statistical threshold. The selected variant is sent to the currently eligible frozen remainder.
- Inconclusive fallback
- A tie, insufficient delivered sample, insufficient variation, or insufficient confidence selects the fallback you chose. This is not a proven performance win.
- Stale or unavailable evidence
- Evaluation retries within bounded limits. Persistent failure can block the remaining audience without sending it. Check the displayed reason; do not treat unavailable metrics as zero.
Inspect the current campaign and message records before taking another action. Do not relaunch the full audience to work around an uncertain result, because some recipients may already have received a message.