Serving Compare a Candidate Version Against the Default
A 4-step recipe with TensorFlow. We can help you get access.
Recipe · 4 steps
Serving Compare a Candidate Version Against the Default
with TensorFlow · Artificial Intelligence Management
Run identical instances through a candidate model version and the default version.
- Confirm the candidate version has finished loading and reports AVAILABLE. A candidate still in the LOADING state would otherwise fail the comparison for reasons that have nothing to do with model quality.
- Read the candidate version's signature metadata to confirm it accepts the same inputs as the version currently serving default traffic.
- Score the evaluation instances against the pinned candidate version.
- Score the identical instances against the model's default version, the one currently answering unpinned production traffic, to produce the baseline the candidate is judged against.
What you'll need to have ready
- modelName: The name the model was registered under in the ModelServer.
- candidateVersion: The version number of the newly trained candidate to evaluate (e.g. 2).
- signatureName: Optional signature to target on both versions. Must be the same on each side for the comparison to be meaningful.
- instances: The evaluation instances sent unchanged to both versions. Holding the input constant is what makes the two responses comparable.
For your developer
Arazzo workflow tensorflow:version-canary-compare, 4 operations: getModelVersionStatus → getModelVersionMetadata → predictModelVersion → predictModel. Specs on APIs.io.