capabilities.work

Build Data Integration Pipeline

A 4-step recipe with Azure Synapse Analytics. We can help you get access.

Recipe · 4 steps

Build Data Integration Pipeline

with Azure Synapse Analytics

Create a linked service, a dataset bound to it, a pipeline, and run the pipeline.

  1. Create or update the linked service that defines the connection to the backing data store.
  2. Create or update the dataset that references the linked service.
  3. Create or update the pipeline that orchestrates movement using the dataset and linked service above.
  4. Submit a run of the newly built pipeline, passing any optional run-time parameters.
What you'll need to have ready
  • apiVersion: The Synapse data plane API version (e.g. 2020-12-01).
  • linkedServiceName: The name of the linked service to create.
  • linkedService: The LinkedServiceResource definition.
  • datasetName: The name of the dataset to create.
  • dataset: The DatasetResource definition; its linkedServiceName reference should point at the linked service created here.
  • pipelineName: The name of the pipeline to create.
  • pipeline: The PipelineResource definition.
  • runParameters: Optional run-time parameters passed to the pipeline run.
For your developer

Arazzo workflow microsoft-azure-synapse-analytics:build-data-integration-pipeline, 4 operations: LinkedService_CreateOrUpdateLinkedService → Dataset_CreateOrUpdateDataset → Pipeline_CreateOrUpdatePipeline → Pipeline_CreatePipelineRun. Specs on APIs.io.

More recipes with Azure Synapse Analytics

Deploy and Run PipelineDeploy and Start TriggerDeploy and Execute Spark Job DefinitionOpen Spark Session and Run StatementProvision and Resume SQL PoolProvision Linked Service and DatasetProvision Spark PoolPublish and Verify NotebookPublish and Verify SQL ScriptSubmit and Poll Spark Batch Job