Tenant isolation that survives the customer who queries ten times harder than everyone else

Overview

A B2B SaaS provider's index-per-tenant model had stopped scaling as their customer count grew. AceMQ designed a tenant architecture that keeps isolation guarantees while cutting the shard count the cluster has to track.

Challenge

With one index per tenant and default shard settings, most indices held very little data while the cluster carried the metadata cost of all of them. Cluster state grew large enough to slow every allocation decision. At the same time, a small number of large tenants could monopolize the search thread pool and cause queue rejections for everyone else, so the isolation the index-per-tenant model was supposed to provide was not actually holding under load.

Environment

OpenSearch on Kubernetes, thousands of tenants ranging from a few thousand documents to hundreds of millions.

Approach

We designed a tiered model: small tenants share pooled indices with a routing key and document-level security, while large tenants keep dedicated indices sized to their data. Alongside that, we added search-level governance so no single tenant can exhaust shared thread pools, and defined the promotion path a tenant follows as it grows.

Solution

• Tiered index strategy separating pooled shared indices for small tenants from dedicated indices for large ones
• Custom routing and document-level security to keep tenant isolation intact within shared indices
• Defined promotion criteria and a tested reindex path for moving a tenant from pooled to dedicated as it grows
• Search thread pool sizing, queue limits, and per-tenant concurrency controls so one heavy tenant cannot starve the rest
• Query governance including timeouts, result-size caps, and rejection handling surfaced back to the application tier
• Cluster and index templates managed as code with tenant onboarding automated end to end

Outcome

Technologies