Back to all use cases
Financial Services / PaymentsRemediationOn-Premises

Getting a stalled Elasticsearch cluster back to green without data loss

GP
Global Payments Processor

Overview

A payments platform lost a data node during a hardware fault and the cluster never recovered to green. Replica shards stayed unassigned for days while search traffic ran on single copies, leaving the index one failure away from real data loss.

Challenge

The surviving nodes had crossed the flood-stage disk watermark, so Elasticsearch had applied a read-only-allow-delete index block and refused to allocate the missing replicas. The allocation explain output pointed at the watermark, but the team could not free space because the block also prevented the delete-by-query cleanup they had planned. Meanwhile the translog on the hot indices kept growing.

Environment

Elasticsearch cluster of roughly 20 data nodes on bare metal, hot-warm tiering, several TB of indexed transaction and audit data.

Approach

AceMQ engineers started from the allocation explain API rather than guessing, then worked the disk pressure down in an order that let the block clear safely. We moved older indices to the warm tier, force-merged read-only segments to reclaim deleted-document space, then cleared the index block and let allocation resume under a throttled recovery rate so the cluster did not saturate its own network during rebuild.

Solution

  • Read cluster allocation explain output for each unassigned shard to confirm the watermark, not corruption, was the blocker
  • Relocated aged indices to warm-tier nodes and force-merged read-only indices to reclaim space from deleted documents
  • Cleared the read-only-allow-delete block only after headroom was restored, avoiding a repeat trip over flood stage
  • Throttled indices.recovery.max_bytes_per_sec during rebuild so replica recovery did not starve live query traffic
  • Set explicit low/high/flood watermarks with realistic headroom and added disk alerting ahead of the high watermark
  • Documented the recovery runbook so the customer's on-call can clear the same condition without escalation

Outcome

The cluster returned to green with every replica allocated and no index loss. Because the watermark alerts now fire well before flood stage, the customer has not hit the read-only block again, and node replacement is now a routine maintenance event instead of an incident.

Technologies

ElasticsearchKibanaLinuxKubernetes

Ready to Get Started?

Whether you need architecture advisory, 24/7 support, or full managed services, AceMQ has the expertise to help.

Contact Us