← Docs
Recipe

Personalization engine design

Architecture for a real-time recommendation pipeline that adapts to user behavior without leaking raw telemetry.

Ingredients

  • Clickstream aggregator — Kafka + Flink windowed counts
  • Feature store — Redis Sorted Sets keyed by user cohort
  • Scoring service — Go binary, gRPC, latency budget 12ms p99
  • Fallback ranker — static popularity index with exponential decay

Prep

  1. Define event schema: impression, click, dwell, dismiss.
  2. Provision feature-store namespace per tenant.
  3. Deploy scoring service behind Envoy with circuit breaker.
  4. Seed fallback index from 90-day aggregate logs.

Execution

The clickstream aggregator flushes 30-second windows into the feature store. On each request, the scoring service fetches the user's cohort vector, blends it with session context via a lightweight factorization machine, and returns a ranked candidate set. If the scoring service exceeds its latency budget, the gateway transparently degrades to the fallback ranker. All telemetry is anonymized at the edge — only cohort IDs leave the tenant boundary.

Key decision

Factorization machines over deep models — they train in under 90 seconds on a single vCPU and degrade gracefully under sparse features, which is the norm during cold starts.