The idea
An A/B test randomly assigns eligible units to alternatives. Define the unit, primary metric, duration and guardrails before examining results. Users who can enter both variants complicate interpretation. Randomization helps reduce confounding but does not excuse unethical exposure or poor implementation.
Worked example
A fictional checkout test assigns each eligible user consistently to version A or B. The primary metric is completed purchases per assigned user; an error-rate guardrail catches a variant that harms reliability even if some purchases increase.
Try it
Write an experiment brief naming the unit, eligibility, primary metric and one guardrail. Define consistent assignment for returning users. Identify a harmful change you would not test and a configuration check needed before collecting outcomes.