A healthcare analytics provider needed long capacity-planning history and multi-year retention for certain operational metrics, but was achieving it by attaching very large block volumes to Prometheus. AceMQ designed an object-storage-backed architecture with downsampling instead.
Local retention was set to well over a year, which meant enormous provisioned volumes per instance, slow compaction, and painful instance moves. Queries over long ranges timed out because they scanned raw-resolution samples across the whole window. There was also no real durability story — the retention lived on volumes attached to single instances, so an instance loss meant losing history that the business had committed to keeping.
Prometheus on Kubernetes across multiple AWS accounts, object storage available, regulated retention obligations on a subset of metrics.
We separated the two jobs Prometheus was being asked to do: recent high-resolution data for alerting, and long-range history for planning and audit. Prometheus keeps a short local window; blocks ship to object storage where a downsampling compactor produces coarser resolutions for long-range queries. The retention obligation is then enforced by object lifecycle policy rather than instance configuration.
Provisioned block storage per instance dropped by roughly an order of magnitude, and year-range capacity queries that used to time out now return in seconds against downsampled data. Retention is now enforceable and auditable without depending on any individual instance surviving.
Assessment of a Prometheus federation topology that had grown past its limits, causing gaps and duplicated data across sites.
Ongoing support for large Prometheus instances where restarts caused extended monitoring blind spots due to slow write-ahead log replay.
Whether you need architecture advisory, 24/7 support, or full managed services, AceMQ has the expertise to help.