Upsert a Slot Pool
A 5-step recipe with Apache Airflow. We can help you get access.
Recipe · 5 steps
Upsert a Slot Pool
with Apache Airflow
Create an Airflow pool or resize it if it already exists.
- Look the pool up by name. A 200 means it exists and should be resized; a 404 means it must be created. Both are expected outcomes, so the branch is decided on the status code.
- Resize the existing pool. The update_mask restricts the patch to slots and description so the pool is not accidentally renamed, which is what happens when the name field is included in the mask.
- Create the pool when the lookup found nothing under this name.
- Read the pool back and require the slot count to match what was requested, returning the live occupied, used, queued, and open slot accounting.
- List every pool so the caller can see the deployment's full concurrency budget alongside the pool just written.
What you'll need to have ready
- poolName: The pool name DAG tasks reference via the pool argument (e.g. "warehouse_writes").
- slots: The number of concurrent task slots the pool allows. Shrinking a pool below its currently used slots does not evict running tasks; it only prevents new ones fro
- description: A human readable description of the resource this pool protects.
For your developer
Arazzo workflow apache-airflow:upsert-pool, 5 operations: getApiV2PoolsByPoolName → patchApiV2PoolsByPoolName → postApiV2Pools → getApiV2PoolsByPoolName → getApiV2Pools. Specs on APIs.io.