BlogValkey

How to Run Valkey in Production: Memory, Replication, Failover

Scott Sternloff

By Scott Sternloff, Senior Enterprise Architect

LinkedIn · Updated · 5 min read

How to Run Valkey in Production: Memory, Replication, Failover
On this page

Short answer

A production Valkey deployment needs four decisions made on purpose: how much memory to allow (maxmemory) and what to evict, how large the replication backlog is, whether and how to persist, and how failover is tested. Defaults suit a laptop. In production, read four INFO sections before an incident: memory, replication, persistence and stats.

A production Valkey deployment needs four decisions made on purpose: how much memory to allow (maxmemory) and what to evict, how large the replication backlog is, whether and how to persist, and how failover is tested. Defaults suit a laptop. In production, read four INFO sections before an incident: memory, replication, persistence and stats.

The defaults below come from the Valkey 9.0 valkey.conf shipped with the server and the valkey.io documentation. If a cluster is already stalling or resyncing, our valkey consulting team begins with the INFO output described here.

How much memory should Valkey be allowed to use?

Set maxmemory below the machine's RAM, and leave headroom for the fork that snapshots and full replication syncs need. A fork is copy-on-write, so a write-heavy primary can use extra memory while a child process runs. Measure the headroom you need during a save.

The default eviction policy is noeviction: once the limit is reached, commands that need more memory (SET, HSET, LPUSH and similar) return an error, and reads continue. Use that for data you cannot lose. For a cache, choose an allkeys-lru or allkeys-lfu policy, or a volatile- policy if only keys with a TTL may be removed.

Read these fields from INFO memory and INFO stats:

Field What it tells you
used_memory vs maxmemory How close you are to the limit
mem_fragmentation_ratio used_memory_rss divided by used_memory; it includes more than fragmentation, so confirm with allocator_frag_ratio, and ignore a high ratio when the byte difference is only a few MB
evicted_keys Keys removed by the eviction policy (should be 0 under noeviction)
mem_clients_normal, mem_clients_slaves Memory held by normal clients and by replicas (replica buffers share memory with the backlog, so this can read 0)
mem_replication_backlog Memory held by the replication backlog

On Linux, the Valkey admin guide asks for two kernel settings: vm.overcommit_memory = 1 and Transparent Huge Pages disabled (echo never > /sys/kernel/mm/transparent_hugepage/enabled). Without the first, a background save can fail with a fork error. With THP on, latency and memory use suffer.

How do I size the replication backlog?

The backlog is a buffer of recent writes. When a replica reconnects, the primary replays the writes it missed from the backlog if they are still there (a partial resync). If they are not, the replica asks for a full resync, which is expensive.

The default repl-backlog-size is 10mb. The backlog is only allocated while at least one replica is connected, and it is released repl-backlog-ttl (3600) seconds after the last replica disconnects.

  1. Measure the write rate. Take master_repl_offset from INFO replication twice, 60 seconds apart, and divide the difference by 60 (bytes per second).
  2. Choose the outage window a replica should survive without a full resync, for example a 10 minute network blip or restart.
  3. Set repl-backlog-size to rate times window, plus margin. At 2 MB per second and 10 minutes that is about 1.2 GB, far above the 10mb default.
  4. Watch INFO stats: rising sync_full while sync_partial_ok stays flat means replicas are not getting partial resyncs. sync_partial_err counts partial resync requests the primary denied.
  5. Compare repl_backlog_size with repl_backlog_histlen in INFO replication. A histlen that stays pinned at the size means the backlog is full and is rolling over.

Check client-output-buffer-limit replica too. It defaults to 256mb 64mb 60: a replica is disconnected if its output buffer passes 256mb, or stays above 64mb for 60 seconds. A replica that cannot keep up gets dropped, reconnects, and may start another full sync. That loop is the usual cause of a replica that "keeps resyncing". Valkey ignores a replica limit set below repl-backlog-size.

Full syncs are diskless by default (repl-diskless-sync yes, with repl-diskless-sync-delay 5 seconds so several replicas can share one transfer). The primary forks, so a full sync costs the same memory headroom as a snapshot.

Which persistence setting should I use?

appendonly is no by default. The two options trade durability against cost:

Option Data you can lose Cost
RDB snapshots only Everything since the last snapshot (minutes) One fork per snapshot; fast restarts
AOF with appendfsync everysec (the default fsync mode) About 1 second Larger files, AOF rewrites (also fork)
AOF with appendfsync always Almost nothing A write to disk before each reply, so slower
None Everything on restart No fork, a pure cache

The persistence documentation warns that fork() can stop Valkey serving clients for some milliseconds, or up to a second on a very large dataset on slow CPUs. INFO persistence shows latest_fork_usec, the duration of the last fork in microseconds. Also check rdb_last_bgsave_status, which should be ok.

If you need to limit lost writes during a replica outage, min-replicas-to-write and min-replicas-max-lag stop the primary accepting writes when too few replicas are in sync. The documentation is explicit that this limits the exposure window but does not guarantee a write reached a replica.

How do I make failover work?

Choose Sentinel for a single shard and Valkey Cluster when data must be split across shards. Then test failover with real clients connected.

  • Sentinel: the sentinel monitor <name> <ip> <port> <quorum> line sets how many Sentinels must agree a primary is down. Run at least three, on separate failure domains.
  • Cluster: cluster-node-timeout (15000 ms) sets how long a node can be unreachable before it is considered failing. A lower value fails over faster but reacts to short stalls, including fork pauses, so test it against your latest_fork_usec.
  • Replicas: replica-serve-stale-data is yes by default, so a replica that lost its primary link keeps answering reads from old data. Decide whether your application can tolerate that.

Drill failover in staging: kill the primary and measure client recovery time.

What are the best practices for monitoring a Valkey instance?

Treat the Valkey configuration as incomplete until it has alerts. Alert on evicted_keys rising when you did not plan to evict, rdb_last_bgsave_status not ok, sync_full growing, latest_fork_usec crossing your latency budget, and replica lag. If you want an independent review of these thresholds before an incident, a valkey consultant can read your INFO output and valkey.conf with your team.

Sources

Frequently Asked Questions

What is the default maxmemory-policy in Valkey?

`noeviction`. When memory is full, write commands that need more memory return an error and reads still work. Cache workloads normally switch to an `allkeys-lru` or `allkeys-lfu` policy.

Why does my Valkey replica keep doing full resyncs?

Usually the replication backlog (10mb by default) is smaller than the writes made while the replica was disconnected, or the replica's output buffer passed `client-output-buffer-limit replica` and the primary dropped it. Compare `sync_full` with `sync_partial_ok` and size the backlog from your write rate.

Does Valkey need persistence turned on?

Only if you cannot rebuild the data. `appendonly` is off by default. A pure cache can run with no persistence, while a primary data store should use AOF with `everysec` and tested restores.

Scott Sternloff

Author

Scott SternloffSenior Enterprise Architect

Senior Enterprise Architect at AceMQ. Writes the technical coverage here: RabbitMQ, Kafka and Redis architecture, clustering and migrations.

Talk to an engineer

Need a hand with Valkey?

Senior engineers who run it in production. Get a quote or a free consultation, no sales pitch.

Free Consultation

Get Expert Eyes on Your Valkey Environment

Whether you're troubleshooting a production incident, planning a migration, or want a second opinion on your architecture — our team is ready. No pitch, just answers.

Email Us