capabilities.work

Resize a Running Cluster and Wait

A 4-step recipe with Azure Databricks. We can help you get access.

Recipe · 4 steps

Resize a Running Cluster and Wait

with Azure Databricks · IT Infrastructure Management

Resize a running cluster's worker count and wait for it to settle.

  1. Read the cluster to capture the cluster_name, spark_version, and node_type_id required as part of the edit request body.
  2. Edit the cluster with the new worker count. A running cluster is restarted with the new configuration applied.
  3. Retrieve the current cluster state. Repeat until the cluster is RUNNING again after the resize; branch to failure if it terminates.
  4. Read the cluster once more to capture the termination reason when the resize did not return to RUNNING.
What you'll need to have ready
  • token: Databricks personal access token for the Authorization header.
  • clusterId: The id of the running cluster to resize.
  • numWorkers: The new number of worker nodes to scale the cluster to.
For your developer

Arazzo workflow microsoft-azure-databricks:resize-running-cluster, 4 operations: getCluster → editCluster → getCluster → getCluster. Specs on APIs.io.

More recipes with Azure Databricks

Back Up a Notebook by Export and Re-importCancel an Active Job RunClean Up the Latest Completed Job RunCluster Health DiagnosticsCreate a Directory and Import a NotebookCreate a Job and Run It to CompletionSafely Delete a Workspace DirectoryImport a Notebook and Run ItPin the First Listed ClusterPreflight and Create a Cluster