Recipe
A/B test platform design
A reference architecture for building a high-throughput experimentation engine with deterministic assignment, real-time telemetry, and statistical rigor.
Core principles
- Deterministic hashing — user ID + experiment key → bucket via xxHash64. No database lookups on the hot path.
- Sticky bucketing — once assigned, a user stays in the same variant for the experiment lifetime. Salt rotation on re-launch.
- Server-side evaluation — variant resolution happens at the edge or API layer. Client SDKs receive a pre-computed flags payload.
Data model
| Entity | Key fields |
|---|---|
| experiment | id, key, status, traffic_pct, variants[], start_at, end_at |
| assignment | user_id, experiment_key, variant, bucket, assigned_at |
| exposure_event | user_id, experiment_key, variant, timestamp, metadata |
Evaluation flow
- 1Request hits edge. Extract user ID from session token.
- 2For each active experiment, compute
hash(user_id + salt + experiment_key) % 10000. - 3Map bucket to variant range. Persist assignment if new.
- 4Return resolved flags. Client renders variant component.
Statistical engine
Use sequential testing with always-valid p-values to avoid peeking bias. Compute relative lift with 95% confidence intervals via bootstrap. Minimum detectable effect gates experiment duration — power analysis runs before launch.
Meridian note: This recipe pairs with the feature-flag system. Use the same hashing infrastructure for both. Keep assignment logs in a columnar store for fast cohort analysis.