DocsGuides
Observability & metrics
Inspect every tool call and follow token usage over time.
Agents are easier to trust when you can see what they did. Rusty records what happens during runs and lets you inspect it.
Tool Execution Observability
Click Tools in the top bar to open Tool Execution Observability. The window's header summarizes what is on disk, for example 215 calls across 24 executions, and the list is grouped by run.
Find a run or a call
- Search looks through prompts, tools, responses, arguments, results and errors.
- All statuses filters by status: queued, waiting for permission, running, succeeded, failed or cancelled.
- All sources filters by the source of a run, such as
agent_chat.
Read a run
Each run is a card showing the prompt that started it, where it came from (for example agent_chat), the model it used, the time, the number of calls, how long it took and the tokens it used. Inside, a timeline lays out the run step by step: each assistant turn and each tool call, with its time offset from the start of the run. Expand Event Trace under a run to see its raw events.
Select a call in the timeline to see its details on the right:
| Section | What it shows |
|---|---|
| Call Output | The result the tool returned, with a Copy result button. |
| Tool Execution | Which model executed the call and its provider, then a timestamped trace of the steps. |
| Run Context (Initiating Prompt) | The user request that started the run, with Copy prompt. Every call in a run shares it. |
| Run Token Usage | Tokens by model: input, output, cache read, cache write and total. |
| Call Metadata | Details about the call, such as its source and surface. |
Who ran each call
Smart tools hand their reading to a separate selector model. For those calls, Tool Execution names the selector as the executor, along with its provider and a purpose such as Smart Read selector. The trace shows each step of the selection: the file loaded for the selector, the selector invocation starting and completing, and the ranges it selected. The selector's tokens are tracked separately from the main run's.
The same view records decisions made by the decision model (JEV). An automatic model selection decision appears as a jev_model_selection call in the timeline, with the model that made it.
Flow decisions
When Auto flows choose a workflow for a message, or check whether a running workflow should hand over, each decision is recorded as its own run with one call. A choice appears as AUTO flow selection, and a boundary check as AUTO flow switching · workflow. Open one to see the request sent to the decision model, the option it chose with its confidence, the ranked probabilities, what the decision cost, and, if it failed, why. The steps read Asked … which workflow fits the message or Asked … whether "…" should carry on or hand over.
With automatic model selection on a workflow, each step's model choice is recorded the same way, labelled AUTO model selection · step.
JEV decision review
If you turn on JEV decision review (experimental) under Settings → Intelligence, JEV scores every tool call as Proceed, Revise or Stop. It runs in shadow mode: the call always runs, and the verdict is recorded on the call. In the trace it appears as a line such as JEV shadow: Revise with confidence 0.34. Not enforced. Each scored call is one Decisions API request to your OpenRouter decision model.
The footer
The bottom of the window shows History on disk: how many calls and runs are shown, the folder they live in (.rusty/observability) and its size. All-time tokens totals your usage by type. Use Show to choose how far back to list, such as 30 days, and Clear history to delete the records.
Records are stored locally in your workspace. Rusty adds a .gitignore there so they are not committed by accident. See The .rusty folder.
Token Metrics
Open Token Metrics in the left rail to follow usage. The rail icon also shows today's total, in compact form, so you can watch it as you work.
Choose a period at the top of the tab:
| Period |
|---|
| Today |
| Yesterday |
| 7 days |
| 30 days |
| All time |
| Custom |
Use it to see how your model choices add up. For instance, whether automatic model selection is keeping routine work on your cheaper models.
Metrics are kept locally in .rusty/metrics/, which is also excluded from Git automatically.