capabilities.work

Start Cluster Then Run Job

A 3-step recipe with Databricks. We can help you get access.

Recipe · 3 steps

Start Cluster Then Run Job

with Databricks · IT Infrastructure Management

Start a terminated cluster, wait for RUNNING, then run a job.

  1. Start the terminated cluster using its prior configuration. Starting is asynchronous, so a poll loop confirms readiness.
  2. Read the cluster status and inspect the life cycle state. Loop back while the cluster is still PENDING; continue once it is RUNNING.
  3. Trigger the job now that the cluster is RUNNING. Returns the run_id of the triggered run.
What you'll need to have ready
  • cluster_id: The terminated cluster to start.
  • job_id: The job to run once the cluster is RUNNING.
  • notebook_params: Optional notebook parameters for the job run.
For your developer

Arazzo workflow databricks:start-cluster-and-run-job, 3 operations: startCluster → getCluster → runJobNow. Specs on APIs.io.

More recipes with Databricks

Audit Cluster Lifecycle EventsCancel a Job's Active RunCancel Job Run and Confirm TerminalInspect and Recursively Delete a DirectoryClone a Job From an Existing OneCreate Job and Trigger First RunDrain Active Runs Then Delete JobEdit Cluster Configuration and VerifyExport a Notebook and Re-Import as a CopyFind Terminated Cluster by Name and Start It