capabilities.work

Launch Flex Template and Monitor

A 3-step recipe with Google Cloud Dataflow. We can help you get access.

Recipe · 3 steps

Launch Flex Template and Monitor

with Google Cloud Dataflow

Launch a Flex Template job, poll until done, and read its metrics.

  1. Launch a Dataflow job from the Flex Template container spec in the requested region.
  2. Fetch the launched job and confirm it has reached the terminal JOB_STATE_DONE state before reading metrics.
  3. Retrieve the execution metrics for the completed job, including counters and distributions emitted by the pipeline.
What you'll need to have ready
  • accessToken: Google Cloud OAuth 2.0 access token used as a Bearer credential.
  • projectId: The Google Cloud project id that owns the job.
  • location: The regional endpoint that contains the job (e.g. us-central1).
  • jobName: The name to assign to the launched job.
  • containerSpecGcsPath: Cloud Storage path to the Flex Template container spec file (gs://...).
  • parameters: Map of runtime template parameter name/value pairs.
For your developer

Arazzo workflow google-cloud-dataflow:launch-flex-template-and-monitor, 3 operations: launchLocationFlexTemplate → getLocationJob → getLocationJobMetrics. Specs on APIs.io.

More recipes with Google Cloud Dataflow

Cancel Running JobCapture Worker Debug DataCleanup Job SnapshotsCreate Job From Template and TrackDiagnose JobDrain Running JobLaunch Classic Template and MonitorList Jobs and SnapshotSnapshot Streaming Job