Resilience

Retries + exponential backoff

Meridian SDK automatically retries failed requests with exponential backoff and jitter. Configure retry behaviour to match your application's tolerance for latency and transient failures.

Default behaviour

By default, the Meridian SDK retries requests that receive 429 (rate limit) or 5xx (server error) status codes up to 2 times with exponential backoff. Each retry waits progressively longer, reducing load on the API during outages.

import Meridian from "meridian";

const client = new Meridian({
  apiKey: process.env.MERIDIAN_API_KEY,
  // max_retries defaults to 2
});

// Transient 503 triggers automatic retry
const result = await client.inference.create({
  model: "meridian-4",
  prompt: "Explain exponential backoff",
});

Configuring `max_retries`

Set max_retries at client construction to control how many times the SDK retries before surfacing the error. Set to 0 to disable retries entirely.

const client = new Meridian({
  apiKey: process.env.MERIDIAN_API_KEY,
  max_retries: 5,   // retry up to 5 times
  timeout: 60_000,  // 60s total timeout including retries
});

try {
  const result = await client.inference.create({
    model: "meridian-4",
    prompt: "Generate a haiku about resilience",
  });
  console.log(result.choices[0].text);
} catch (err) {
  // Raised after all retries exhausted
  console.error("Request failed after retries:", err.message);
}

Backoff strategy

Meridian uses exponential backoff with full jitter. The delay before retry n is calculated as:

delay = random(0, min(cap, base * 2^n))

// base = 500ms, cap = 60s
// Retry 1: 0–1000ms
// Retry 2: 0–2000ms
// Retry 3: 0–4000ms
// Retry 4: 0–8000ms
// Retry 5: 0–16000ms

Full jitter spreads retries across clients, preventing thundering herd problems when many instances recover simultaneously. The cap ensures retries never wait longer than 60 seconds.

Retryable errors

The SDK retries on these conditions:

1HTTP 429 Too Many Requests — respects Retry-After header when present
2HTTP 500, 502, 503, 504 server errors
3Connection errors (ECONNRESET, ETIMEDOUT, socket hang-up)

Client errors (4xx except 429) are not retried — they indicate a problem with the request itself that retrying cannot fix.

← Quickstart Streaming →