External Asset Event Pipeline
A 3-step recipe with Dagster. We can help you get access.
Recipe · 3 steps
External Asset Event Pipeline
with Dagster
Report observation, materialization, and check events for one external asset key.
- Record an observation of the external asset, capturing the data version and metadata about the source before it is materialized.
- Record the materialization of the asset, reusing the data version from the observation so the events are correlated in Dagster.
- Record the asset check evaluation against the freshly materialized asset, carrying severity through when the check did not pass.
What you'll need to have ready
- apiToken: Dagster+ user API token sent in the Dagster-Cloud-Api-Token header.
- assetKey: The external asset key driven through the event pipeline.
- dataVersion: A user-supplied data version shared by the observation and materialization.
- partition: The asset partition associated with the events.
- metadata: Free-form key/value metadata attached to each reported event.
- checkName: The identifier of the asset check evaluated after materialization.
- passed: True if the asset check passed, false otherwise.
- severity: Severity level reported when the check failed (WARN or ERROR).
For your developer
Arazzo workflow dagster:external-asset-event-pipeline, 3 operations: reportAssetObservation → reportAssetMaterialization → reportAssetCheck. Specs on APIs.io.