Push Batch Job Metrics to the Pushgateway
A 5-step recipe with Prometheus. We can help you get access.
Recipe · 5 steps
Push Batch Job Metrics to the Pushgateway
with Prometheus
Push a completed batch job's metrics and verify they are stored.
- Read the Pushgateway build information. Recording the version alongside a push is worth the one request, because text-format and OpenMetrics handling has changed across Pushgateway releases.
- Confirm the Pushgateway is ready to accept pushes before the job discards its in-memory results.
- Atomically replace every metric for this job with the results of this run. Using PUT rather than POST is what stops a metric emitted by yesterday's run, and not by today's, from lingering forever and being scraped as if it were current.
- Add the completion metric to the group with POST, which leaves the metrics written by the previous step untouched.
- Read back every stored metric group and confirm this job's group is present with a fresh lastPushedAt. This is the only positive confirmation the job gets that its telemetry survived its own exit.
What you'll need to have ready
- jobName: The job label for this metric group, and the primary grouping key (e.g. batch-importer).
- metricsPayload: The run's metrics in Prometheus text exposition format, including HELP and TYPE lines. Must end with a trailing newline.
- completionPayload: A follow-up metric in text exposition format, typically the last-success timestamp used to alert on jobs that stop running.
For your developer
Arazzo workflow prometheus:push-batch-job-metrics, 5 operations: getStatus → checkReady → replaceJobMetrics → pushJobMetrics → listAllMetrics. Specs on APIs.io.