Back to all use cases
Manufacturing / AutomotiveRemediationOn-Premises

Fixing the access pattern, not just raising the tombstone threshold

CV
Connected Vehicle Telematics Provider

Overview

A telematics provider used a Cassandra table as a work queue, writing rows and deleting them after processing. Read queries began timing out as coordinators scanned tens of thousands of tombstones to return a handful of live rows. AceMQ was engaged under the emergency SLA.

Challenge

Tombstones remain readable until gc_grace_seconds elapses and compaction removes them, which exists to prevent deleted data from resurrecting on a node that missed the delete. Repairs had not been completing across the ring, so lowering gc_grace_seconds carried real risk of resurrection. The team had been raising the tombstone warning and failure thresholds, which hid the symptom while the underlying scan cost kept growing.

Environment

Multi-datacenter Apache Cassandra cluster on bare metal ingesting vehicle telemetry at high volume.

Approach

AceMQ separated the immediate relief work from the durable fix. Getting repair completing reliably was the prerequisite for any change to gc_grace_seconds, and the data model change was necessary because a queue on a wide partition generates tombstones faster than compaction can retire them regardless of settings.

Solution

  • Identified the specific partitions and query shapes scanning the largest tombstone counts
  • Restored reliable repair completion across the ring as a prerequisite for touching gc_grace_seconds
  • Moved the queue workload off the wide-partition pattern to time-bucketed partitions with TTL-based expiry
  • Switched the affected tables to a compaction strategy matched to the write and delete pattern
  • Reverted the raised tombstone thresholds so the condition surfaces again rather than being suppressed
  • Ran targeted compaction to retire the existing tombstone backlog during a controlled window

Outcome

Read timeouts on the affected tables stopped, and p99 read latency came down substantially as tombstone scan counts fell by orders of magnitude. Repair now completes on schedule, which also closed the resurrection risk the customer had been carrying unknowingly.

Technologies

Apache CassandraKubernetesPrometheusLinux

Ready to Get Started?

Whether you need architecture advisory, 24/7 support, or full managed services, AceMQ has the expertise to help.

Contact Us