LLM observability:
see inside every call.
LLM observability gives you traces, cost-per-query, and hallucination scores for every language model call your app makes — so you can debug, optimize, and trust your AI in production.
What is LLM observability?
You can't debug what you can't see.
Traditional software observability — logs, metrics, traces — was designed for deterministic systems. Language models are probabilistic: the same input can produce different outputs, costs vary with token count, and failures are often semantically wrong rather than thrown exceptions. LLM observability extends the classic observability stack to handle this.
Traces
A full call tree for every agent run — each LLM call, tool invocation, and retrieval step captured as a span with inputs, outputs, latency, and token count.
Cost tracking
Token usage translated to dollars, broken down by query, operation, and tenant. Know your unit economics before the bill arrives.
Quality scoring
Sentence-level hallucination detection — each claim in an LLM response is verified against your retrieved context and labeled supported, contradicted, or unsupported.
Why you need LLM observability
Four problems that only observability can solve.
The problem
Wrong answers with no error thrown
Traces show exactly what the model received.
An LLM returns wrong output because a tool returned null, and the model hallucinated around it. There is no exception. Without a trace showing the tool output and the model input, you spend hours guessing.
The problem
API costs growing faster than usage
Cost-per-query trend finds the culprit.
Unbounded conversation history, redundant retrieval, or one expensive feature can double your token spend without touching your traffic. Cost-per-query trending upward is the signal.
The problem
Hallucinations you only hear about from users
Sentence-level scoring surfaces them before users do.
A 0–1 faithfulness score tells you something was wrong. Claim decomposition tells you which sentence was wrong and what context it contradicted — so you can fix the prompt or the retrieval.
The problem
Slow responses you can't profile
The waterfall shows where time actually goes.
Teams swap models first. The trace waterfall almost always shows the bottleneck is a slow tool call or a cache miss, not the model. You can't find this without a span for every step.
How to add LLM observability
Two lines before your first import. Zero configuration.
Peekr patches LLM clients at the class level — every OpenAI(), AsyncOpenAI(), and anthropic.Anthropic() instance is captured. No wrappers. No proxy. No framework-specific setup.
- Auto-instruments OpenAI, Anthropic, Gemini, Bedrock, LangChain
- Async and streaming calls captured and rolled up
- Zero latency overhead — background thread export
- Works with LlamaIndex, CrewAI, and any agent framework
# Step 1 — call before any other imports
import peekr
peekr.instrument(
tenant_id="acme",
exporter=peekr.HTTPExporter(
endpoint="https://peekr.starkspherelabs.com",
api_key="pk_live_…",
),
evaluators=[peekr.eval.Hallucination(detailed=True)],
)
# Step 2 — your existing code, unchanged
from openai import OpenAI
client = OpenAI() # ← traced automaticallyGo deeper
AI Agent Observability →
Multi-step agents, tool calls, and cross-agent trace waterfalls — observability for agentic workflows.
LLM Observability Platform →
Shared dashboards, per-tenant cost, and team alerts — the full platform for teams shipping AI together.
LLM Guardrails →
Observability scores; guardrails block. Add in-process enforcement to HIPAA, FDCPA, or GDPR rules.
Add LLM observability in two lines.
Free up to 10k spans per month. No credit card. Auto-instruments OpenAI, Anthropic, Gemini, and Bedrock — no proxy, no wrappers, no lock-in.