KV Cache Tuning

Problem

Default KV cache settings often waste memory on infrequently accessed keys while evicting hot data prematurely. This causes unnecessary round-trips and higher latency under load.

Solution

Profile your access patterns, then tune TTL, maxmemory, and eviction policy together. Use allkeys-lru for general caching and volatile-lru when mixing persistent and ephemeral keys.

Steps

1. Enable Upstash metrics and record hit/miss ratio over 24h.
2. Identify keys with sub-10% hit rate — shorten their TTL or remove caching.
3. Set maxmemory-policy allkeys-lru if all keys are cacheable.
4. Bump maxmemory until eviction rate drops below 5%.
5. Monitor p99 latency — target under 10ms for cached reads.

Verification

Run a 1-hour load test at 2x peak throughput. Confirm hit ratio stays above 90% and no OOM evictions appear in logs. Adjust maxmemory upward if evictions spike.