capabilities.work

Validate and Deploy a Connector

A 5-step recipe with Apache Kafka. We can help you get access.

Recipe · 5 steps

Validate and Deploy a Connector

with Apache Kafka

Validate a connector configuration against its plugin, then deploy it.

  1. Read the Connect worker root to confirm the worker version and which Kafka cluster it is attached to before deploying anything to it.
  2. List the connector plugins installed on the worker so the caller can confirm the requested plugin class is on the plugin path.
  3. Run the supplied configuration through the plugin's validation endpoint. The step only succeeds when the plugin reports zero configuration errors, which stops a bad config from ever reaching the cluster.
  4. Create the connector on the worker using the configuration that has just passed validation.
  5. Read the connector status back so the deployment is confirmed by the worker rather than assumed from the create response.
What you'll need to have ready
  • pluginClass: The fully qualified connector plugin class to validate against (e.g. org.apache.kafka.connect.file.FileStreamSourceConnector).
  • connectorName: The name to register the connector under on the worker.
  • connectorConfig: The connector configuration as a flat map of string key/value pairs, including connector.class, tasks.max, and any plugin-specific keys.
For your developer

Arazzo workflow apache-kafka:validate-and-deploy-connector, 5 operations: getWorkerInfo → listConnectorPlugins → validateConnectorConfig → createConnector → getConnectorStatus. Specs on APIs.io.

More recipes with Apache Kafka

Update a Connector ConfigurationDecommission a ConnectorTriage a Failed ConnectorPause and Resume a Connector for MaintenanceReset Connector OffsetsRewind Connector OffsetsReview Consumer Group LagGrant a Principal Access to a TopicReview Cluster and Topic CapacityDecommission a Topic