How to monitor an LLM API (OpenAI, Anthropic, Gemini) in Spot Downtime
A step-by-step guide to LLM API monitoring: use a separate key, pick the model, set the prompt and time limit, and keep the cost of checks to a fraction of a cent.
· 5 min read · By the Spot Downtime team
A provider's status page speaks for its whole platform, not for your key, your model or your quota. An LLM API monitor sends a short prompt to the exact model your product uses, with your own key, and checks that a real answer comes back in time. A revoked key, an empty balance, a retired model or slow replies all fail the check, just as they would fail your app.
OpenAI · gpt-5-mini · every 5 minutes
Reply with the single word OK.
OK
Up · 1.2 sWhen it fails, the alert says why:
Step 1: create a separate API key
In your provider's console, create a key just for monitoring, with a low spending limit. If it ever leaks, it can't run up a bill, and you can revoke it without touching production.
Step 2: create the monitor
- Open Monitors, click Add monitor and choose LLM API.
- Pick the Provider: OpenAI, Anthropic, Google Gemini, Mistral, Groq, DeepSeek, xAI, OpenRouter or Together AI. For Azure OpenAI, vLLM, LiteLLM, Ollama or any OpenAI-compatible server, choose Custom and enter its Base URL.
- Paste the API key. Once saved, only its last 4 characters are ever shown.
- Pick the Model from the list, or type its ID. Use the same model your product calls.
Add a monitor
Provider
OpenAI
API key
sk-…a91F
Model
gpt-5-mini
Prompt
Reply with the single word OK.
Reply must contain
OK
Step 3: set what counts as a good answer
| Field | Default | What to change |
|---|---|---|
| Prompt | Reply with the single word OK. | Usually nothing. A short prompt keeps checks cheap. |
| Reply must contain | OK | Not case-sensitive. A blank or different reply counts as down. |
| Max reply tokens | 64 | Raise to 500 or more for reasoning models (see below). |
| Down if slower than | No limit | Set it if slow replies hurt your users, such as 5 seconds. |
Reasoning models
Step 4: choose how often
Pick Check every and click Add monitor. With the defaults each check costs a tiny fraction of a cent on most models. Every 5 minutes is about 8,640 requests a month, and the form shows the number for the interval you pick. A failure is retried once, not repeated from every region.
Step 5: read an alert
Alerts say what went wrong in plain words, for example:
- Key rejected: the key was revoked or mistyped.
- Rate limited or out of credits (HTTP 429): top up, or raise your limits.
- Provider overloaded: often temporary. Check the provider's status too.
- Reply missing your text: the alert shows what came back instead.
Pair it with a third-party service monitor on the same provider: one tells you about your key and model, the other about outages the provider declares.
LLM API latency testCompare OpenAI, Claude and Gemini response times now, free.More in this series
Get new guides by email
Practical posts on monitoring, outages, error fixes and on-call, when they're published. No spam.
Keep reading
- How-to · Monitoring · APIsHow to configure gRPC health checkingLearn how to implement and configure gRPC health checking for reliable service discovery and load balancing: server status, Kubernetes probes, client-side health checks, Envoy, cloud load balancers and Consul.November 7, 2026 · 7 min read
- How-to · APIs · MonitoringHow to monitor an API that needs a login, with multi-step checksChain requests in one monitor: log in, save the access token, and use it to call the endpoints that matter, with JSON assertions on every step.October 28, 2026 · 5 min read