capabilities.work

Open Spark Session and Run Statement

A 3-step recipe with Azure Synapse Analytics. We can help you get access.

Recipe · 3 steps

Open Spark Session and Run Statement

with Azure Synapse Analytics

Start a Spark session, wait for it to be idle, and run a code statement.

  1. Create a new interactive Spark session on the pool.
  2. Poll the session until its livy currentState is idle. Loop back while it is still starting; proceed once it is ready.
  3. Submit a code statement to the ready Spark session.
What you'll need to have ready
  • sparkPoolName: The name of the Spark pool to host the session.
  • sessionOptions: The SparkSessionOptions body. Must include name at minimum.
  • statement: The SparkStatementOptions body, including code and kind (e.g. spark, pyspark, sql).
For your developer

Arazzo workflow microsoft-azure-synapse-analytics:open-spark-session-and-run-statement, 3 operations: SparkSession_CreateSparkSession → SparkSession_GetSparkSession → SparkSession_CreateSparkStatement. Specs on APIs.io.

More recipes with Azure Synapse Analytics

Build Data Integration PipelineDeploy and Run PipelineDeploy and Start TriggerDeploy and Execute Spark Job DefinitionProvision and Resume SQL PoolProvision Linked Service and DatasetProvision Spark PoolPublish and Verify NotebookPublish and Verify SQL ScriptSubmit and Poll Spark Batch Job