Runs are the core unit of LLMonitor: everything boils down to a run.
It can be an LLM request, an agent execution, a tool execution, a workflow, etc.
Each run has an input and most often, an output.
Runs are the billing unit of LLMonitor. You can see the number of runs you have consumed in the billing page.
An LLM call refers to a request made to a language model, such as a GPT-4 query.
Chains denote sequences of connected runs, tools, and LLM calls.
They help visualize the flow and dependencies in complex tasks, clarifying the interactions between different components of the system.
An agent is usually composed of tools and LLM calls.
It autonomously interacts with various components, and might iterate over tasks until it finds a solution.
A tool is a piece of code that your AI agent can use to perform a specific action.
A tool usually doesn't make AI queries itself.
It is the same notion as in LangChain for example.
Example of tools:
- Search on Google
- Random number generator
A trace is a collection of runs that are related to each other.
An agent will generate a trace everytimes it executes.
Diving into traces on the dashboard helps you understand how your code is behaving and how the different LLM pieces are interacting with each other.
Using our SDKs, runs are automatically hierarchized into traces.
A user is someone that uses your app. With all our SDKs you can identify users.