capabilities.work

Stage Notebook Then Create and Run a Job

A 3-step recipe with Databricks. We can help you get access.

Recipe · 3 steps

Stage Notebook Then Create and Run a Job

with Databricks

Import a notebook, wire a job to it, and run it.

  1. Import the base64-encoded notebook into the target path, overwriting any existing object.
  2. Create a job whose notebook task runs the freshly imported notebook on the supplied existing cluster.
  3. Trigger the first run of the newly created job and capture the run_id.
What you'll need to have ready
  • notebook_path: The absolute path to import the notebook to and run.
  • content: The base64-encoded notebook content.
  • language: The notebook language (SCALA, PYTHON, SQL, or R).
  • existing_cluster_id: The cluster the job task should run on.
  • job_name: The name for the new job.
  • task_key: The unique task key within the job.
For your developer

Arazzo workflow databricks:stage-notebook-and-create-job, 3 operations: importWorkspaceObject → createJob → runJobNow. Specs on APIs.io.

More recipes with Databricks

Audit Cluster Lifecycle EventsCancel a Job's Active RunCancel Job Run and Confirm TerminalInspect and Recursively Delete a DirectoryClone a Job From an Existing OneCreate Job and Trigger First RunDrain Active Runs Then Delete JobEdit Cluster Configuration and VerifyExport a Notebook and Re-Import as a CopyFind Terminated Cluster by Name and Start It