Open Source: 491
Bring your AI app
to production.
Observability, analytics and tests for AI agents and chatbots.
Integrate now — it's free
free up to 1k events / day
Log LLM calls
Log all your prompts and results, see how agents are performing in production.
- Edit calls in the Playground
- Search & filter instantly
- Live tail
- Label data for fine-tuning

Stay on top of costs
Monitor requests and costs segmented by user and model.
- Auto cost & token calculations
- Optimize costs
- Costs by user, prompts & agents

Debug complex agents
Replay agent executions with traces and find out what went wrong and where.
- Trace agent executions
- Error stack traces
- Filter & instant search

Be alerted when things go wrong
Write evaluations in English or code and run them against your queries.
- AI-powered assertions & tests
- Automated Alerts
- CI/CD pipeline integration
- Sentiment Analysis

Replay user chats
Record user conversations and identify gaps in your chatbot's knowledge.
- Capture user feedback
- React hooks
- Dev-friendly frontend integration

Create training datasets
Label your outputs based on tags and feedback then easily export datasets to fine-tune your models, improving your app's quality and reducing costs.

Model-agnostic tracking
Monitor all your LLMs whether they're the latest OpenAI models or an open-source model rising in the benchmarks.


















Integrate in seconds.
Our libraries provide automatic instrumention to get setup in seconds.
Explore the docs →
We recorded a 1 minute demo video.
(expect better as we ship a lot)
What our users say
Join our Open Source community.
Everything we build is 100% open-source. Check out the source-code or our product roadmap.
llmonitor: 491
Developers also love these
Live TailNew logs and runs are shown in realtime.
Instant SearchSearch across your whole data in milliseconds.
Label dataAdd tags to your data to easily filter and fine-tune.
Self HostSetup on your own servers for maximum data security.
PlaygroundCollaboratively iterate on prompts.
Data APIFetch your data to integrate it in other services.
AlertsSetup alerts to be notified for outlier results and errors.
React HooksTrack chats and feedback direcly in your frontend.
AI EvaluationsUse GPT-4 to auto-evaluate and judge responses.
Get started in minutes.
Run it on your own or get started in minutes with our hosted version.
Prompt Playground
Evaluate & Test
Alerts
Public API
Exports
Open Source
Self Hostable
Agent Tracing
Analytics
Chat Replays
User Tracking
Feedback Tracking
LangChain Support
Prompt Playground
Evaluate & Test
Alerts
Public API
Exports
Open Source
Self Hostable
Agent Tracing
Analytics
Chat Replays
User Tracking
Feedback Tracking
LangChain Support