capabilities.work

Collect DAG Run Results from XCom

A 4-step recipe with Apache Airflow. We can help you get access.

Recipe · 4 steps

Collect DAG Run Results from XCom

with Apache Airflow

Retrieve the XCom value a finished DAG run's task published.

  1. Poll the DAG run until Airflow stamps an end_date. XCom values are only dependable once the run has reached a terminal state.
  2. Read the producing task instance and require the success state. A failed or skipped producer publishes no XCom, so stopping here yields a clear error instead of an empty result.
  3. List every XCom entry the producing task published in this run so the caller can see what is available before reading a specific key.
  4. Read the requested XCom key with deserialize enabled so the response carries the actual value. Note that deserialize is rejected unless the Airflow deployment has enabled allow_deserialize_xcom_via_api.
What you'll need to have ready
  • dagId: The DAG identifier the run belongs to.
  • dagRunId: The run identifier to collect results from.
  • taskId: The task within the DAG that publishes the result to XCom.
  • xcomKey: The XCom key to read. Airflow stores a task's plain return value under the key "return_value", which is the sensible default.
For your developer

Arazzo workflow apache-airflow:collect-dag-run-results, 4 operations: getApiV2DagsByDagIdDagRunsByDagRunId → get_task_instance → getApiV2DagsByDagIdDagRunsByDagRunIdTaskInstancesByTaskIdXcomEntries → getApiV2DagsByDagIdDagRunsByDagRunIdTaskInstancesByTaskIdXcomEntriesByXcomKey. Specs on APIs.io.

More recipes with Apache Airflow

Audit DAG Runs Across the FleetTrace Dataset-Driven LineageDeployment Preflight CheckDiagnose a Failed DAG RunForce a Task Instance StateOnboard a DAGProvision a User with a RoleRegister a Connection After Testing ItRetry Failed Tasks in a DAG RunTrigger a DAG Run and Wait for Completion