On this page
Short answer
Repair, compaction and tombstones are one system in Cassandra: a delete writes a tombstone, repair spreads it to every replica, and compaction removes it after gc_grace_seconds. If any of the three runs late, reads slow down or deleted data comes back. This guide gives the schedule, the commands and the numbers to check. It is written for teams that run their own clusters and want to know when to call for Cassandra support.
Repair, compaction and tombstones are one system in Cassandra: a delete writes a tombstone, repair spreads it to every replica, and compaction removes it after gc_grace_seconds. If any of the three runs late, reads slow down or deleted data comes back. This guide gives the schedule, the commands and the numbers to check. It is written for teams that run their own clusters and want to know when to call for Cassandra support.
How often should you run Cassandra repair?
Run repair on every node at least once inside gc_grace_seconds, which defaults to 864,000 seconds (10 days). The Apache Cassandra docs suggest repairing every node at least every 7 days with that default, which leaves slack for delays. The same page gives a starting point of an incremental repair every 1 to 3 days plus a full repair every 1 to 3 weeks.
| Command | What it does |
|---|---|
nodetool repair | Incremental repair (the default type in the 5.0 docs) |
nodetool repair --full | Full repair, to catch corruption and operator error |
nodetool repair -pr | Only the node's primary ranges; run on every node in every datacenter |
nodetool repair -prv | Preview: estimates streaming without doing it |
nodetool repair -vd | Validate: compares repaired data across nodes without streaming |
Cassandra does not run repair by itself in earlier versions. Cassandra 5.0.8 added CEP-37 Auto Repair, a built-in scheduler, but it needs the JVM property cassandra.autorepair.enable=true set on every node before any schema change, and the project notes say the property cannot be reversed once enabled. Treat it as a planned change, not a toggle.
What to check: a scheduled repair is not a completed one. Compare the last completed repair per table against gc_grace_seconds.
How do you choose a compaction strategy?
Match the strategy to the write pattern, then check that compaction keeps up. The compaction docs list four:
| Strategy | Fits | Avoid when |
|---|---|---|
| Unified (UCS) | The docs recommend it for most and new workloads, time series or update-heavy | You are on a version without it (it arrived in 5.0) |
| Size tiered (STCS) | Default; general write-heavy tables | Read latency matters and SSTable counts climb |
| Leveled (LCS) | Read-heavy tables, many updates and deletes | Immutable time-series data |
| Time window (TWCS) | TTL'd, mostly immutable time series | Tables that are updated or deleted out of order |
Three commands show whether compaction is winning:
nodetool compactionstatsshows active and pending compactions. A pending count that only climbs means compaction is behind.nodetool compactionhistorylists recent compactions, so you can see whether big SSTables are actually being merged.nodetool setcompactionthroughputchanges the throttle. The defaultcompaction_throughputis 64MiB/s, and it also applies to validation compactions for repair, so repair and compaction compete for the same budget.
Changing strategy rewrites data: go one table at a time and watch disk headroom.
How do tombstones work, and when do they hurt?
A delete is an insert of a time-stamped marker. A cell with a TTL becomes a tombstone when the TTL ends. Compaction can drop a tombstone only when it is older than gc_grace_seconds and the SSTables holding the older data it shadows are in the same compaction.
The read-path guard rails live in cassandra.yaml: tombstone_warn_threshold is 1000 and tombstone_failure_threshold is 100000 per query. A query that crosses the first logs a warning; one that crosses the second fails. If you see those log lines, find the table before you raise the thresholds.
Checks, in order:
nodetool tablestats <keyspace>.<table>for tombstones scanned per slice and live cells per slice.tombstone_threshold(default 0.2): the share of an SSTable that must be tombstones before single-SSTable compaction is considered.unchecked_tombstone_compaction(default false) andtombstone_compaction_interval(default 86,400 seconds) to see why a tombstone-heavy SSTable is not being rewritten.sstableexpiredblockerswhen an SSTable holds only tombstones and still will not drop: it lists which other SSTables block it.
Queue-like tables, short TTLs on wide partitions and bulk deletes are the usual sources. Raising thresholds hides the symptom; fix the data model, for example time-bucketed partitions with TTL on a TWCS table.
Why do deleted rows come back?
They come back when a replica misses a delete and the tombstone is purged elsewhere before repair reaches it. The Cassandra docs call this zombie data: if a node stays down longer than gc_grace_seconds, its old copy is treated as new and repaired back into the cluster. Hints are best-effort, and max_hint_window defaults to 3 hours, so a node down for longer needs repair, not hints.
Rules that prevent it:
- Repair every table inside
gc_grace_seconds, including tables nobody reads. - Never lower
gc_grace_secondson a table whose repair history you have not verified. - After a node has been down longer than the hint window, repair it before it serves traffic you care about.
A weekly operations checklist
| Check | Command or setting | Healthy sign |
|---|---|---|
| Repair completion | repair history per table | Every table inside gc_grace_seconds |
| Compaction backlog | nodetool compactionstats | Pending tasks return to near zero |
| SSTables per read | nodetool tablehistograms | Stable, not rising |
| Tombstone warnings | system log | No new warn or failure lines |
| Hint backlog | nodetool tpstats, hints directory | Drains after a node returns |
When to call for help
Call when repair has not completed on any table for longer than gc_grace_seconds, when pending compactions have climbed for days, or when read timeouts track a table with heavy deletes. AceMQ provides 24/7 Cassandra support across the Americas, EMEA and APAC, with response targets from 15 minutes, by plan, and support typically runs $24,000-72,000 a year (USD) by SLA, scope and deployment. For the incident view of the same problems, see Cassandra support options, Cassandra pricing and alternatives, and the use cases Cassandra tombstone and read timeout remediation and Cassandra repair and compaction support.
Sources
- Apache Cassandra docs, Repair: cassandra.apache.org/doc/latest/cassandra/managing/operating/repair.html, read 17 Nov 2026.
- Apache Cassandra docs, Compaction overview: .../operating/compaction/overview.html, read 17 Nov 2026.
- Apache Cassandra docs, Tombstones: .../operating/compaction/tombstones.html, read 17 Nov 2026.
- cassandra.yaml, branch cassandra-5.0 (thresholds, compaction_throughput, max_hint_window): github.com/apache/cassandra/blob/cassandra-5.0/conf/cassandra.yaml, read 17 Nov 2026.
- Cassandra NEWS.txt, branch cassandra-5.0 (CEP-37 Auto Repair in 5.0.8; full repair after incremental pre-upgrade): github.com/apache/cassandra/blob/cassandra-5.0/NEWS.txt, read 17 Nov 2026.
Frequently Asked Questions
How often should I run nodetool repair?
Every node must finish a repair inside `gc_grace_seconds` (10 days by default). The Apache docs suggest every 7 days at least, as incremental every 1 to 3 days with a full repair every 1 to 3 weeks.
What is the default compaction strategy in Cassandra?
Size tiered (STCS) is the default. For new workloads the 5.0 docs recommend the Unified Compaction Strategy; TWCS is for TTL'd time series.
What are the default tombstone thresholds?
`tombstone_warn_threshold` is 1000 and `tombstone_failure_threshold` is 100000 in `cassandra.yaml` for 5.0.
Is it safe to lower gc_grace_seconds?
Only if every repair on that table completes well inside the new value. Otherwise you risk resurrected deletes.
Does Cassandra 5.0 run repair automatically?
From 5.0.8, an optional Auto Repair scheduler exists (CEP-37). It needs a JVM property set on all nodes and cannot be switched back off at that level.
Talk to an engineer
Need a hand with Cassandra?
Senior engineers who run it in production. Get a quote or a free consultation, no sales pitch.
