Recipe

Log anomaly detection

Ship a real-time pipeline that flags suspicious patterns in your application logs using a sliding-window statistical model.

Overview

This recipe wires up a lightweight anomaly detector that ingests structured JSON logs, maintains a rolling baseline of event frequencies, and fires alerts when any bucket exceeds three standard deviations. No external services required — everything runs in your existing Node.js process.

Ingredients

Structured JSON log emitter (Pino, Winston, or Bunyan)
Sliding-window counter (60-second buckets, 15-minute retention)
Z-score threshold calculator
Alert dispatcher (webhook, Slack, or stdout)

Steps

Instrument logs. Emit every request, error, and auth event as a flat JSON line with an event_type field.
Build the window. Partition time into 60-second buckets. On each tick, push the current bucket into a ring buffer and expire buckets older than 15 minutes.
Compute baseline. Calculate the mean and standard deviation of per-event-type counts across all active buckets.
Evaluate. For each incoming event, compute its z-score. If |z| > 3.0, mark it anomalous.
Dispatch alert. Throttle alerts to one per event type per 5 minutes. Post to your webhook with the event type, observed count, expected range, and timestamp.

Gotchas

• Cold start: the first 15 minutes produce no alerts while the window fills.
• Low-volume event types need a minimum-count gate (default: 5 per window) to avoid false positives.
• Deployment restarts reset the window; persist buckets to a temp file if you need continuity.

← Back to docs More recipes