Emergency Incident Response
The scheduler stops scheduling, or a business-critical DAG chain is stuck. A senior engineer joins a live bridge within 15 minutes with direct access to diagnose — not a ticket acknowledgement.
Apache Airflow Support
Last reviewed by Scott Sternloff
AceMQ supports Apache Airflow in production — schedulers that stop scheduling with no error, DAG parsing delays that quietly push back every downstream run, executor slot exhaustion from zombie tasks, and metadata database connection exhaustion under high concurrency. Every ticket reaches a named senior engineer who already knows your DAG topology.
Planning a build, upgrade or migration instead? See AceMQ Apache Airflow consulting and services.
Customers




Partners


Incident Triage
These are real symptoms from real Apache Airflow production environments — and the first thing our engineers check when one comes in.
| Symptom you're seeing | What we check first | Typical resolution |
|---|---|---|
| DAGs stop scheduling with no error in the UI or logs | Scheduler health and heartbeat, whether max_active_runs is saturated on the affected DAG, and whether executor slots are consumed by zombie tasks that never released cleanly after a worker crash. | Under 1 hour |
| Everything runs late, scheduling across the whole instance drifts | DAG file parsing time in the scheduler logs. Heavy top-level imports or API calls executed at parse time — not inside a task — run on every scheduler heartbeat and delay scheduling for every DAG, not just the offending one. | 1–3 hours |
| A task sits in 'queued' state and never starts | Executor and worker pool alignment — queue name mismatches between the task's assigned queue and the workers actually listening on it, or a worker pool with zero available slots. | Under 2 hours |
| XComs are enormous and the metadata DB is bloating or slow | What's actually being passed through XCom versus external storage. Passing dataframes or large payloads through XCom instead of a reference to S3/GCS is the most common cause, and it degrades DB performance for every DAG, not just the culprit. | 1–2 hours |
| Metadata database connections exhausted under load | Connection pool sizing on the scheduler, webserver, and workers against DAG concurrency settings, plus whether Airflow's own connection pooling (SQLAlchemy pool_size) is configured for the actual DAG parallelism. | 2–4 hours |
| Celery/Kubernetes executor workers crash and lose task state | Worker pod resource limits against task memory usage, and whether task state is being tracked correctly through executor restarts. KubernetesExecutor pod evictions under resource pressure are a common silent cause. | 2–4 hours |
| A DAG ran at the wrong time or ran twice | schedule_interval and timezone configuration versus the DAG's actual start_date and catchup setting. Timezone-naive schedules and catchup=True combined with a backfill are the most frequent causes of unexpected double-runs. | Under 1 hour |
| Custom operators broke after upgrading to Airflow 2.x | DAG serialization requirements and provider package compatibility. Airflow 2.x's DAG serialization model rejects certain dynamic patterns that worked fine under 1.x, and provider package version pins are a frequent silent break. | 2–4 hours |
Resolution times reflect typical Apache Airflow engagements under an active AceMQ support contract. Every P1 closes with a written root-cause analysis.
What's Included
No add-on pricing for incidents. No per-ticket charges. One contract covers the whole surface.
The scheduler stops scheduling, or a business-critical DAG chain is stuck. A senior engineer joins a live bridge within 15 minutes with direct access to diagnose — not a ticket acknowledgement.
Every P1 closes with a written RCA: what failed, why, the fix applied, and the specific DAG, config, or infrastructure change that prevents recurrence.
max_active_runs, pool sizing, executor slot allocation, and DAG parse time reduction tuned against your actual DAG count and task concurrency.
Guidance on major version upgrades, provider package compatibility, and custom operator migration — including the DAG serialization changes that break naive 1.x-to-2.x upgrades.
Quarterly reviews of metadata database connection load, XCom growth, and worker resource headroom so scheduler performance doesn't degrade as DAG count grows.
Executor migration (Celery to Kubernetes or vice versa), consolidating legacy cron and Pentaho-style batch jobs into Airflow, and DAG design review for teams scaling past a few hundred DAGs.
Why AceMQ
The same senior engineers stay on your account. They know your DAG topology, your executor setup, and which pipelines are fragile — so a P1 starts with diagnosis, not twenty minutes of explaining your environment.
You reach a senior Airflow engineer directly by phone, email, or Slack. No help desk collecting information to pass along, no escalation approval process standing between you and someone who can fix it.
Most Airflow incidents are metadata database, executor infrastructure, or DAG design problems wearing an Airflow error. We diagnose across the whole stack, not just the scheduler UI.
Engineers across 26+ countries and every time zone. Your overnight batch failure is someone's mid-afternoon — no overnight skeleton crew, no waiting for a region to wake up.
Quarterly health checks on DAG parse time and metadata DB load, plus shared intelligence across our support base. When a provider package bug surfaces on one customer's DAGs, every affected customer hears about it before it reaches their production.
Airflow failures are frequently caused by DAG authoring patterns — top-level API calls, unbounded XCom, missing idempotency — that logs alone won't reveal. We review the actual DAG code when that's where the problem lives.
Cloud, Kubernetes, bare metal, hybrid, and air-gapped — including environments where you can't give us outbound network access.
Escalation Path
Every tier reaches a senior Apache Airflow engineer. There is no tier-1 triage layer to get through.
P1 escalation path
A replay of one P1, from the first page to the written root-cause analysis
15-minute P1 response, 24/7.
A response time, not a resolution time.
P1 opened, on-call engineer paged
Phone, email or Slack reaches the on-call senior engineer directly, whether the incident is in RabbitMQ, Kafka, Redis, PostgreSQL or Kubernetes. No web form, no tier-1 queue.
Senior engineer on your bridge
A senior engineer who already knows your environment joins a live bridge, so nobody has to re-explain your topology.
Specialists across the stack join
When an incident crosses components, the engineers who know each one join. For RabbitMQ, the path runs to the core team.
Written root-cause analysis
Root cause, the fix applied and the prevention steps, delivered after every P1.
Pricing
AceMQ Apache Airflow support contracts typically run $24,000–72,000 a year, depending on SLA level, scope (number of clusters or nodes, and environments) and deployment (self-managed, cloud-managed or Kubernetes).
| What moves the price | How it changes the contract |
|---|---|
| SLA level | Business-hours cover costs less than 24/7 cover with a 15-minute emergency response. |
| Scope | More clusters or nodes, and more environments, make a larger contract. |
| Deployment | Self-managed, cloud-managed and Kubernetes estates need different amounts of hands-on work. |
Covering several technologies under one contract, for example RabbitMQ, Kafka and Redis, earns an aggressive discount.
Support In Practice
Apache Airflow support work from AceMQ. Each card links to the full write-up: the Airflow estate, what AceMQ diagnosed or changed, and the result.
Global Insurance Group
Resolving pipeline runtime blowouts caused by queries spilling to remote storage on undersized warehouses while concurrent jobs queued behind them.
Retail Analytics Provider
Ongoing support for Snowpipe, stream, and task failures including stale streams past their retention window and silent partial-load conditions.
Financial Market Data Provider
Assessment of clustering keys, micro-partition pruning, and table design on large tables where queries had begun scanning most of the data.
National Insurance Carrier
Restoring scheduling on Airflow deployments where zombie tasks hold executor slots and pools until nothing new gets queued.
Digital Retail Platform
Fixing scheduler delay caused by DAG files that make network or database calls at parse time, blocking every DAG in the deployment.
Regional Utility Operator
Reviewing an Airflow deployment for reliability, DAG authoring practice, secrets handling, and upgrade readiness before it becomes unmaintainable.
Support options
AceMQ offers Apache Airflow support in four tiers, each handing more of the day-to-day work to AceMQ: 24/7 support and incident management for platforms your team runs, proactive monitoring that finds parse delays and slot exhaustion first, managed Airflow on your own infrastructure, and fully managed Airflow on AceMQ infrastructure. Airflow 3 and Airflow 2 are covered, including MWAA, Cloud Composer and Astronomer environments.
You run it, we back you up
Best for: Data platform teams that run Airflow themselves and need senior escalation cover when a pipeline chain stops.
You run it, we watch it with you
Best for: Teams that want the stale dashboard prevented, not explained afterward.
We run it in your cloud or data centre
Best for: Regulated estates that must keep orchestration and credentials inside their own network.
We host it and run it, as a service
Best for: Teams that want Airflow without a platform rota or an upgrade project of their own.
| Who does what | Tier 1 24/7 support | Tier 2 Proactive monitoring | Tier 3 Managed, your infrastructure | Tier 4 Fully managed by AceMQ |
|---|---|---|---|---|
| Who runs Apache Airflow day to day? | Your team | Your team, with AceMQ reviews | AceMQ | AceMQ |
| Where does it run? | Your infrastructure | Your infrastructure | Your cloud account or data centre | AceMQ infrastructure |
| 24/7 incident response | 15-minute P1 response | 15-minute P1 response | 15-minute P1 response | 15-minute P1 response |
| Health checks and CVE alerts | On request | Scheduled | Included | Included |
| Patching and upgrades | Your team, AceMQ advises | Planned with AceMQ | Done by AceMQ | Done by AceMQ |
| Monitoring and on-call | Your team | Your team | AceMQ | AceMQ |
| Data, keys and accounts | Yours | Yours | Yours | Your data, hosted by AceMQ |
Every Tier can start as an ad hoc engagement or an annual partnership, and you can move up or down a Tier as your team changes.
One partner for Apache Airflow
Support is one part of what AceMQ does for Apache Airflow. The same engineers deliver consulting, health checks, migrations and end-of-life cover, and support the systems that usually run next to it.
Our Airflow support covers 24/7 emergency incident response with a 15-minute SLA, root cause analysis on every P1, scheduler and executor tuning, upgrade and compatibility advisory, quarterly capacity reviews, and migration or architecture support. Every ticket is handled by a named senior engineer — there is no tier-1 triage layer and no per-incident charge.
Yes — this is one of the most common Airflow incidents we handle. It's almost always one of three things: scheduler heartbeat failure, max_active_runs saturation on a specific DAG, or executor slots consumed by zombie tasks that never released after a worker crash. We check all three in the first diagnostic pass.
Yes, alongside self-managed Airflow on Kubernetes or bare infrastructure. Managed variants remove some operational surface but not the ones that cause most incidents — DAG design, executor slot exhaustion, and metadata DB connection pressure are still yours to manage, and still where we spend most diagnostic time.
Yes. As an extension of the support contract, AceMQ runs managed Airflow on your own infrastructure: Airflow on Kubernetes with the official Helm chart, on virtual machines or in Docker stays in your account, and we take on the day-2 operations. That means monitoring the scheduler, executors, workers and metadata database, holding the pager, applying patches, planning and running upgrades, and reviewing capacity as DAG volume grows. Your team keeps writing DAGs; keeping the platform under them healthy becomes our job.
Astronomer's Astro, Amazon MWAA and Google Cloud Composer are managed Airflow services: the provider runs the Airflow environment on its platform, offers the Airflow versions it supports, and bills for the environment. They are a good fit for a new deployment or a team already standardised on that cloud. Moving an existing deployment onto one is a migration, and DAG design, dependency conflicts and executor tuning still sit with you. AceMQ's managed operations keep Airflow where it already runs, on versions and configuration you choose, and cover that DAG-level layer too. If you already use one of those services, our support covers it as it is.
P1 emergencies — the scheduler is down or a business-critical DAG chain is stuck — get a 15-minute first response with a senior engineer joining a live bridge. P2 critical issues are 1 hour, P3 high-priority 4 hours, and P4 standard questions next business day. These are contractual, not best-effort targets.
Yes. Heavy top-level code in a single DAG file — API calls or expensive imports executed at parse time rather than inside a task — runs on every scheduler heartbeat and delays parsing for every other DAG in the instance. We identify the offending DAG and refactor the parse-time logic out of it.
Yes. Airflow 2.x's DAG serialization requirements break certain dynamic patterns that worked under 1.x, and provider package version pins are a frequent source of breakage after an upgrade. We validate and fix custom operators as part of upgrade support.
Apache Airflow is the open source platform created by the community to programmatically author, schedule and monitor workflows as Python DAGs. The Airflow community supports current versions with bug fixes and offers help on GitHub and Slack, with no SLA; Airflow 2 reached end of life in April 2026. AceMQ's support service covers self-managed Airflow on Kubernetes with the official Helm chart, on virtual machines, and in Docker, plus Amazon MWAA, Google Cloud Composer and Astronomer where the provider runs the platform and nobody runs your DAGs. In scope are the scheduler, executors, workers and metadata database, task instances stuck in queued, slow DAG parsing, dependency conflicts between providers and your own Python packages, versioning and upgrade planning across Airflow versions, and security and compliance reviews of connections, secrets and access.
Yes. Airflow 3 is an architectural release: tasks no longer talk to the metadata database directly, the REST API changed, SubDAGs and other deprecated features were removed, datasets became assets, and a newer Python version is required. The project recommends moving to the last 2.x release first and clearing deprecation warnings there, and ships a linter that flags incompatible DAG code, which turns the migration into a measurable backlog. Estates still on 1.10 go through 2.x on the way. We run the migration with your team: environment build, DAG code review, provider version alignment, a parallel run against production schedules, and a cutover with a tested rollback.
Yes. AceMQ works on Apache Airflow in two ways: ad hoc engagements for a defined piece of work, such as a health check, an upgrade or an incident, or an annual partner engagement that covers 24/7 support, proactive reviews and named engineers. Many teams start ad hoc and move to an annual partnership once the first engagement is done.
Every AceMQ proposal is tailored to your requirements: your Apache Airflow environment, the SLA you need, the hours of cover and the scope of work. Most contracts run $24,000–72,000 a year, depending on the SLA, the scope and how Apache Airflow is deployed, with aggressive discounts when several technologies share one contract. Request a quote and AceMQ replies within 1 hour.
Let's talk
Reactive cover, proactive reviews, or AceMQ running Apache Airflow for you on your infrastructure or ours. A senior engineer scopes it with you, and quote requests are answered within 1 hour.
Production incident right now?Email an Apache Airflow Architect
Send us a message and we'll follow up within 1 hour — or book a free 30-min consultation directly.
305-204-2607info@acemq.comPrefer to talk now? Call us directly or use the consultation tab to find a time that works.