Log LLM calls
Log all your prompts and results, see how agents are performing in production.
- Edit calls in the Playground
- Search & filter instantly
- Live tail
- Label data for fine-tuning
Stay on top of costs
Monitor requests and costs segmented by user and model.
- Auto cost & token calculations
- Optimize costs
- Costs by user, prompts & agents
Debug complex agents
Replay agent executions with traces and find out what went wrong and where.
- Trace agent executions
- Error stack traces
- Filter & instant search
Be alerted when things go wrong
Write evaluations in English or code and run them against your queries.
- AI-powered assertions & tests
- Automated Alerts
- CI/CD pipeline integration
- Sentiment Analysis
Replay user chats
Record user conversations and identify gaps in your chatbot's knowledge.
- Capture user feedback
- React hooks
- Dev-friendly frontend integration
Create training datasets
Label your outputs based on tags and feedback then easily export datasets to fine-tune your models, improving your app's quality and reducing costs.
Monitor all your LLMs whether they're the latest OpenAI models or an open-source model rising in the benchmarks.
Integrate in seconds.
Our libraries provide automatic instrumention to get setup in seconds.
Explore the docs →
We recorded a 1 minute demo video.
(expect better as we ship a lot)
Developers also love these
Live TailNew logs and runs are shown in realtime.
Instant SearchSearch across your whole data in milliseconds.
Label dataAdd tags to your data to easily filter and fine-tune.
Self HostSetup on your own servers for maximum data security.
PlaygroundCollaboratively iterate on prompts.
Data APIFetch your data to integrate it in other services.
AlertsSetup alerts to be notified for outlier results and errors.
React HooksTrack chats and feedback direcly in your frontend.
AI EvaluationsUse GPT-4 to auto-evaluate and judge responses.
Get started in minutes.
Run it on your own or get started in minutes with our hosted version.
Evaluate & Test
Evaluate & Test