capabilities.work

Provision Cluster and Wait Until Running

A 2-step recipe with Databricks. We can help you get access.

Recipe · 2 steps

Provision Cluster and Wait Until Running

with Databricks · IT Infrastructure Management

Create a cluster and wait until it is RUNNING.

  1. Create the Spark cluster. The response returns the cluster_id used to poll for readiness; creation itself is asynchronous.
  2. Read the cluster details and inspect the life cycle state. Loop back while the cluster is still PENDING; finish once it is RUNNING.
What you'll need to have ready
  • cluster_name: Human-readable name for the new cluster.
  • spark_version: The Spark runtime version (e.g. a Databricks Runtime key).
  • node_type_id: The node type for worker and driver nodes.
  • num_workers: The number of worker nodes for the cluster.
  • autotermination_minutes: Idle minutes before the cluster auto-terminates.
For your developer

Arazzo workflow databricks:provision-cluster, 2 operations: createCluster → getCluster. Specs on APIs.io.

More recipes with Databricks

Audit Cluster Lifecycle EventsCancel a Job's Active RunCancel Job Run and Confirm TerminalInspect and Recursively Delete a DirectoryClone a Job From an Existing OneCreate Job and Trigger First RunDrain Active Runs Then Delete JobEdit Cluster Configuration and VerifyExport a Notebook and Re-Import as a CopyFind Terminated Cluster by Name and Start It