Deciding what to send before deciding what to pay for
KubernetesTerraformIngested volume fell by roughly forty percent with no dashboard or alert losing its data source. Because ingest is now attributed per team and budgeted, growth is a visible decision rather than a…
Overview
A SaaS provider's telemetry ingest was growing well ahead of its business, with no clear owner for the growth. AceMQ helped them attribute volume, cut what nothing consumed, and put controls at the collection layer so the reduction holds.
Challenge
Ingest was dominated by a small number of sources: verbose framework logs forwarded at debug level, custom events emitted per request that were only ever aggregated, and infrastructure metrics collected at a much finer interval than any dashboard displayed. Nobody could see which team's telemetry drove which share of the bill, so cost conversations went nowhere and the growth continued each quarter.
Environment
New Relic across a multi-account AWS estate with Kubernetes workloads and several autonomous product teams.
Approach
AceMQ does not operate the vendor platform — we help customers decide and control what leaves their infrastructure. The engagement attributes ingest by team and source, checks what is genuinely queried, and implements drop rules and sampling at the collection layer where they apply regardless of what individual applications emit.
Solution
- 1Ingest volume attributed by team, service, and telemetry type so cost has an owner rather than a total
- 2Query and dashboard usage cross-referenced to identify telemetry that is ingested but never consumed
- 3Drop rules applied at the collection layer for provably unconsumed log and event classes
- 4Log forwarding levels corrected at source, removing debug output left enabled from past incidents
- 5Infrastructure metric collection intervals aligned to the resolution dashboards and alerts actually use
- 6Per-team ingest budgets with monitoring so a logging change surfaces as an alert rather than an invoice
Outcome
Ingested volume fell by roughly forty percent with no dashboard or alert losing its data source. Because ingest is now attributed per team and budgeted, growth is a visible decision rather than a quarterly surprise.
Technologies
KubernetesTerraformRelated Use Cases
New Relic Instrumentation Coverage Assessment
Assessment of which services, dependencies, and code paths are genuinely covered by APM instrumentation versus assumed to be.
New Relic Agent Overhead Remediation
Remediation of application latency and memory growth traced to APM agent configuration on the customer's own JVM and container estate.
Datadog Custom Metric and Ingest Cost Governance
Consulting engagement to govern custom metric cardinality, log ingest, and trace volume so observability spend tracks value instead of accident.
Datadog Monitor and Alert Noise Support
Ongoing support for a Datadog monitor estate producing more alerts than the on-call rotation could meaningfully act on.
New Relic Distributed Tracing Support
Ongoing support for distributed tracing across a mixed New Relic and OpenTelemetry estate where traces broke at instrumentation boundaries.
OpenSearch Multi-Tenant Search Architecture Design
Consulting engagement to design tenant isolation, index strategy, and query governance for OpenSearch serving thousands of customer tenants.
Need New Relic Architecture Guidance?
AceMQ's senior New Relic engineers have handled this exact type of engagement before. Whether you need architectural guidance, hands-on remediation, or an ongoing managed partnership, we're ready to help.