TGI Inspect and Generate
A 3-step recipe with Hugging Face. We can help you get access.
Recipe · 3 steps
TGI Inspect and Generate
with Hugging Face · Artificial Intelligence Management
Inspect a TGI server, tokenize a prompt, and generate text from it.
- Read the TGI server info to learn the loaded model id and its maximum input length before submitting a prompt.
- Tokenize the prompt to measure how many tokens it consumes against the server's input limit.
- Generate text from the prompt using the loaded model with details enabled so the finish reason and token count are returned.
What you'll need to have ready
- hfToken: Hugging Face access token used as a Bearer credential.
- prompt: The input prompt to tokenize and generate from.
- maxNewTokens: Maximum number of new tokens to generate.
- temperature: Sampling temperature.
For your developer
Arazzo workflow hugging-face:tgi-inspect-and-generate, 3 operations: getInfo → tokenize → generate. Specs on APIs.io.