How We Measure
A quick mental model for what actually becomes part of your graph, and how to put your scores to work.
What counts as a node
Two things build your graph: agents (a system prompt) and tools (a description). Everything else — plain LLM calls with no system prompt — isn't a decision point, so it's not counted.
Has a system prompt — a real decision point.
Has a description — same treatment as an agent.
No system prompt — treated as plumbing, not a decision.
Turning scores into improvements
Score + explanation isn't a report card — it's feedback you can act on.
Agent skips a step, scores low, explanation says why — feed that back into your orchestrator (or that agent's instructions) and retry. decision_identifier lets you confirm scores actually improve.
A strategy for building trust over time
Start with observability, then gate the moments that matter.
- New network — observability only (
ALWAYS), no manual gates yet. Watch the scores to see how it performs before touching anything. - Once you spot the risky step — add a manual gate right before it, not before every step.
A bad find or a rough draft costs nothing to redo. An unwanted purchase does — so that's where the gate goes, not earlier.
See the Usage Patterns section for the code behind a gate.
Authentication
API keys authenticate your agent's calls to Trellar. You'll need one before you can call the client library.
- Create a Trellar account (or sign in if you already have one).
- Once you're in the dashboard, go to API Keys and click Create API Key.
- Copy the key right away — for security, you won't be able to see it again.
Installation
trellar is the Python client for Trellar's confidence evaluation API. Install it, then set your API key so the client can authenticate.
trellar is the TypeScript / Node.js client for Trellar's confidence evaluation API, with the same features as the Python package. Install it next to the agent framework you use, then set your API key so the client can authenticate.
Install
pip install "trellar[langchain]"
Requires Python 3.9+. The langchain extra is required because agent runs are captured via a LangChain callback handler.
pip install "trellar[strands]"
Requires Python 3.10+ (Strands Agents needs it). The strands extra installs strands-agents and is required because agent runs are captured via a Strands hook provider. Using both frameworks in one project? Install "trellar[langchain,strands]".
Set your API key
export TRELLAR_API_KEY=your-api-key
Or pass it directly: evaluate_confidence(api_key="...").
Install
npm install trellar @langchain/core @langchain/langgraph
Requires Node.js 20+ (tested on 20, 22, 24 and 26). @langchain/core is required because agent runs are captured via a LangChain callback handler.
npm install trellar @strands-agents/sdk
Requires Node.js 22+ (the Strands Agents SDK needs it). @strands-agents/sdk is required because agent runs are captured via a Strands plugin.
@langchain/core and @strands-agents/sdk are optional peer dependencies — install only the framework you use. The package ships ESM and CommonJS builds with type declarations.
Imports
Each framework's integration lives in its own entry point, next to the shared core:
import { evaluateConfidence, ObservabilityMode, runWithTrellarAgent } from "trellar";
import { trellarLangchainAgent, trellarLangchainSingleCall } from "trellar/langchain";
import { evaluateConfidence, ObservabilityMode, runWithTrellarAgent } from "trellar";
import { trellarStrandsAgent, trellarStrandsSingleCall } from "trellar/strands";
Set your API key
export TRELLAR_API_KEY=your-api-key
Or pass it directly: evaluateConfidence({ apiKey: "..." }). The client always talks to the managed Trellar backend — the endpoint is fixed and can't be overridden.
Quick Start
LangChain / LangGraph
Attach the guard to your LangChain / LangGraph run, then call
evaluate_confidence() from inside a node while the run is still in progress — trace and agent context are picked up automatically.
Strands Agents
Register the guard as a hook on your Strands agents and graph, then call
evaluate_confidence() from inside a graph node (or a tool) while the run is still in progress — trace and agent context are picked up automatically.
LangChain / LangGraph
Start by observing: set your API key, add the Trellar callback to your run, and Trellar evaluates the run automatically when it finishes. To read the score in your code or stop the graph when it's low, call
evaluateConfidence() from inside a node while the run is still in progress — trace and agent context are picked up automatically.
Strands Agents
Register the Trellar agent as a plugin on your Strands graph, and Trellar evaluates the run automatically when it finishes. To read the score in your code or stop the graph when it's low, call
evaluateConfidence() from inside a graph node (or a tool) while the run is still in progress — trace and agent context are picked up automatically.
from trellar import get_agent_guard, evaluate_confidence
# agent_name must be a stable, unique name for this agent graph — the
# backend uses it to track the graph's network profile across runs.
guard = get_agent_guard("research-agent")
# call evaluate_confidence() from a node, while the run is still in progress —
# context, trace_id, and agent_name are picked up from the guard automatically
def report_confidence(state):
result = evaluate_confidence()
print(result.score) # int, 1-10
print(result.explanation) # str, human-readable reasoning
print(result.decision_identifier) # uuid, for tracing & BI
return state
graph.invoke(inputs, config={"callbacks": [guard]})
evaluate_confidence() from inside a node while the graph is still running — calling it after graph.invoke() returns raises ValueError.
Single LLM calls
Calling a chat model directly with no wrapping graph? Use get_single_call_guard instead. A bare llm.invoke() has no node to call evaluate_confidence() from mid-call, and the guard is released the instant the call finishes — so a manual call is never supported here. Pass an observability_mode and read the outcome back off the guard instead.
from trellar import get_single_call_guard, ObservabilityMode
guard = get_single_call_guard("single-llm-call", ObservabilityMode.ALWAYS)
llm.invoke(messages, config={"callbacks": [guard]})
result = guard.trellar_evaluate_result # AgentLoopResult, or None if not yet evaluated
error = guard.trellar_evaluate_error # the exception, if the auto-triggered call failed
Agents built with create_react_agent are already compiled graphs, so they work with get_agent_guard as usual — get_single_call_guard is only for a genuinely bare model call. Requests made through this guard are tagged single_call: true in the backend payload, so it can tell the two guard types apart.
import asyncio
from strands import Agent
from strands.agent.agent_result import AgentResult
from strands.multiagent import GraphBuilder
from strands.telemetry.metrics import EventLoopMetrics
from trellar import get_strands_guard, evaluate_confidence
# agent_name must be a stable, unique name for this agent network — the
# backend uses it to track the network's profile across runs.
guard = get_strands_guard("research-agent")
# A non-LLM graph node that scores the run so far. evaluate_confidence() is a
# blocking HTTP call, so run it in a thread — context, trace_id, and agent_name
# are picked up from the guard automatically.
class ConfidenceNode:
name = "report_confidence"
async def invoke_async(self, prompt=None, **kwargs):
result = await asyncio.to_thread(evaluate_confidence)
print(result.score) # int, 1-10
print(result.explanation) # str, human-readable reasoning
print(result.decision_identifier) # uuid, for tracing & BI
# Pass the previous step's output along to the next node.
return AgentResult(
stop_reason="end_turn",
message={"role": "assistant", "content": [{"text": str(prompt)}]},
metrics=EventLoopMetrics(),
state={},
)
def __call__(self, prompt=None, **kwargs):
return asyncio.run(self.invoke_async(prompt, **kwargs))
async def stream_async(self, prompt=None, **kwargs):
yield {"result": await self.invoke_async(prompt, **kwargs)}
# Register the guard on EVERY agent...
searcher = Agent(name="searcher", hooks=[guard])
reporter = Agent(name="reporter", hooks=[guard])
builder = GraphBuilder()
builder.add_node(searcher, "searcher")
builder.add_node(ConfidenceNode(), "report_confidence")
builder.add_node(reporter, "reporter")
builder.add_edge("searcher", "report_confidence")
builder.add_edge("report_confidence", "reporter")
builder.set_entry_point("searcher")
# ...and on the graph itself, so the whole run is one trace.
builder.set_hook_providers([guard])
graph = builder.build()
graph("Find me something to report on")
evaluate_confidence() from inside a graph node while the graph is still running — calling it after graph(...) returns raises ValueError. Register the same guard on every Agent and on the graph (set_hook_providers); otherwise the run is not captured as a single trace.
Single agent calls
Calling one Agent directly with no wrapping graph? Use get_strands_single_call_guard instead. The run is over the moment the call returns, so a manual evaluate_confidence() is never supported here. Pass an observability_mode and read the outcome back off the guard instead.
from strands import Agent
from trellar import get_strands_single_call_guard, ObservabilityMode
guard = get_strands_single_call_guard("faq-agent", ObservabilityMode.ALWAYS)
agent = Agent(name="faq_agent", hooks=[guard])
agent("What time does the office open?")
result = guard.trellar_evaluate_result # AgentLoopResult, or None if not yet evaluated
error = guard.trellar_evaluate_error # the exception, if the auto-triggered call failed
Requests made through this guard are tagged single_call: true in the backend payload. Not covered: agent.structured_output() and calling a Strands Model directly — Strands fires no model-call hooks for them, so there is nothing to capture.
The response object
Each call to evaluate_confidence() returns a result with the following fields:
The confidence score for this run — 1 is low confidence, 10 is high confidence.
A human-readable summary of the reasoning behind the score — handy for logs, audits, or showing a reviewer why a run was flagged.
A unique identifier for this evaluation. Use it to trace and correlate the decision across your own BI dashboards or third-party systems.
NetworkHaltedError
Not returned as a field, and not something that can trigger on its own. It only fires if you've created a rule with Stop Execution enabled and attached it to this agent's network — if that rule matches, evaluate_confidence() raises NetworkHaltedError instead of returning normally. No rule configured means this never happens. Catch it to halt the run gracefully.
from trellar import evaluate_confidence, NetworkHaltedError
try:
result = evaluate_confidence()
except NetworkHaltedError:
# Trellar determined this agent's network should stop — halt the run
return {**state, "halt": True}
from trellar import NetworkHaltedError
# Don't catch it inside the gate node — let it propagate out of the graph.
# Nodes after the gate never run.
try:
graph("Find me something to report on")
except NetworkHaltedError:
# Trellar determined this agent's network should stop — the run was halted
print("Halted by a Trellar Stop Execution rule")
In Strands, a NetworkHaltedError raised in a graph node bubbles up through the graph and stops it, so catch it around the graph call rather than returning a flag in state.
import { createAgent, tool } from "langchain";
import { z } from "zod";
import { ObservabilityMode } from "trellar";
import { trellarLangchainAgent } from "trellar/langchain";
const searchDocs = tool(async () => "The office opens at 9am.", {
name: "search_docs",
description: "Search the internal docs.",
schema: z.object({ query: z.string() }),
});
const graph = createAgent({ model, tools: [searchDocs] });
// agentName must be a stable, unique name for this agent graph — the
// backend uses it to track the graph's network profile across runs.
const trellarAgent = trellarLangchainAgent("research-agent", ObservabilityMode.ALWAYS);
// The run is captured and evaluated when invoke() finishes.
// Context, trace_id, and agent_name are picked up automatically.
await graph.invoke(
{ messages: [{ role: "user", content: "What time does the office open?" }] },
{ callbacks: [trellarAgent] },
);
The run is sent to Trellar when invoke() returns, and the result appears in your dashboard. Trellar scores the run from the events it captured, so the run should include at least one agent call or tool call.
config into nested runnables (model.invoke(msgs, config)) so child runs are reported to the Trellar agent — the same rule as for any LangChain callback. Call evaluateConfidence() while the run is still in progress: once the root invoke() returns, the run is closed and a later call throws.
Single LLM calls
Calling a chat model directly with no wrapping graph? Use trellarLangchainSingleCall instead. A bare llm.invoke() has no node to call evaluateConfidence() from mid-call, and the Trellar agent is released the instant the call finishes — so a manual call is never supported here. The run is evaluated automatically (ObservabilityMode.ALWAYS is the default; pass ObservabilityMode.NONE to only record). Read the outcome back off the Trellar agent.
import { trellarLangchainSingleCall } from "trellar/langchain";
const trellarAgent = trellarLangchainSingleCall("single-llm-call");
await llm.invoke(messages, { callbacks: [trellarAgent] });
const result = trellarAgent.trellarEvaluateResult; // AgentLoopResult, or null if not yet evaluated
const error = trellarAgent.trellarEvaluateError; // the error, if the auto-triggered call failed
Agents built with createAgent are already compiled graphs, so they work with trellarLangchainAgent as usual — trellarLangchainSingleCall is only for a genuinely bare model call. Requests made through it are tagged single_call: true in the backend payload.
import { Agent, Graph } from "@strands-agents/sdk";
import { ObservabilityMode } from "trellar";
import { trellarStrandsAgent } from "trellar/strands";
// agentName must be a stable, unique name for this agent network — the
// backend uses it to track the network's profile across runs.
// ALWAYS: the run is evaluated automatically when it finishes.
const trellarAgent = trellarStrandsAgent("research-agent", ObservabilityMode.ALWAYS);
// Give every agent a stable name and a unique id.
const searcher = new Agent({ id: "searcher", name: "searcher" });
const reporter = new Agent({ id: "reporter", name: "reporter" });
// Register the Trellar agent as a plugin on the Graph: the whole run is one
// trace, and every node's Agent is bound automatically.
const graph = new Graph({
nodes: [searcher, reporter],
edges: [["searcher", "reporter"]],
plugins: [trellarAgent],
});
await graph.invoke("Find me something to report on");
agent.id, which defaults to "agent" — give each agent in a graph a unique id. Without an ObservabilityMode, nothing is evaluated unless you call evaluateConfidence() from inside the run (a graph node or a tool), while the graph is still executing.
Single agent calls
Calling one Agent directly with no wrapping graph? Use trellarStrandsSingleCall instead. The run is over the moment the call returns, so a manual evaluateConfidence() is never supported here. The run is evaluated automatically (ObservabilityMode.ALWAYS is the default; pass ObservabilityMode.NONE to only record). Read the outcome back off the Trellar agent.
import { Agent } from "@strands-agents/sdk";
import { trellarStrandsSingleCall } from "trellar/strands";
const trellarAgent = trellarStrandsSingleCall("faq-agent");
const agent = new Agent({ name: "faq_agent", plugins: [trellarAgent] });
await agent.invoke("What time does the office open?");
const result = trellarAgent.trellarEvaluateResult; // AgentLoopResult, or null
const error = trellarAgent.trellarEvaluateError; // the error, if the auto-triggered call failed
A standalone Agent (no Graph) always needs new Agent({ plugins: [trellarAgent] }). Requests are tagged single_call: true in the backend payload. Not covered: agent.structuredOutput() and calling a Strands Model directly — there is nothing to capture.
The response object
Each call to evaluateConfidence() resolves to a frozen AgentLoopResult with the following fields:
The confidence score for this run — 1 is low confidence, 10 is high confidence.
A human-readable summary of the reasoning behind the score — handy for logs, audits, or showing a reviewer why a run was flagged.
A unique identifier for this evaluation. Use it to trace and correlate the decision across your own BI dashboards or third-party systems.
NetworkHaltedError
Not something that can trigger on its own. It only fires if you've created a rule with Stop Execution enabled and attached it to this agent's network — if that rule matches, evaluateConfidence() rejects with NetworkHaltedError (which carries explanation, score and decisionIdentifier) instead of resolving normally. No rule configured means this never happens. Catch it to halt the run gracefully.
import { evaluateConfidence, NetworkHaltedError } from "trellar";
async function confidenceGate(state: State) {
try {
const result = await evaluateConfidence();
return { ...state, halt: false, score: result.score };
} catch (err) {
if (err instanceof NetworkHaltedError) {
// Trellar determined this agent's network should stop — halt the run
return { ...state, halt: true, reason: err.explanation };
}
throw err;
}
}
import { evaluateConfidence, NetworkHaltedError } from "trellar";
try {
const result = await evaluateConfidence();
console.log(result.score, result.explanation);
} catch (err) {
if (err instanceof NetworkHaltedError) {
// Trellar determined this agent's network should stop — halt the run
console.log("Halted by a Trellar Stop Execution rule:", err.explanation);
}
throw err;
}
evaluateConfidence({ apiKey?, timeout? }) also rejects with a plain Error if there is no active Trellar agent or no API key, and with TrellarHTTPError (status, statusText, url, body) on a non-2xx response. timeout is in seconds (default 30).
Observability Modes
Don't want to remember to call evaluate_confidence() yourself every time? Pass an observability_mode to
get_agent_guard() and it auto-triggers the call for you when the graph's root run finishes.
Don't want to remember to call evaluate_confidence() yourself every time? Pass an observability_mode to
get_strands_guard() and it auto-triggers the call for you when the graph's root run finishes.
Don't want to remember to call evaluateConfidence() yourself every time? Pass an ObservabilityMode to
trellarLangchainAgent() and it auto-triggers the call for you when the graph's root run finishes.
Don't want to remember to call evaluateConfidence() yourself every time? Pass an ObservabilityMode to
trellarStrandsAgent() and it auto-triggers the call for you when the graph's root run finishes.
from trellar import get_agent_guard, ObservabilityMode
guard = get_agent_guard("research-agent", ObservabilityMode.IF_NOT_EVALUATED)
graph.invoke(inputs, config={"callbacks": [guard]})
# evaluate_confidence() has already run automatically if no node called it.
from trellar import get_strands_guard, ObservabilityMode
guard = get_strands_guard("research-agent", ObservabilityMode.IF_NOT_EVALUATED)
# ...register `guard` on every Agent and via builder.set_hook_providers([guard])...
graph("Find me something to report on")
# evaluate_confidence() has already run automatically if no node called it.
Available modes
Never auto-call. Identical to not passing observability_mode at all — call evaluate_confidence() yourself wherever you need a score.
Always call evaluate_confidence() when the run finishes — in addition to any manual calls already made during the run.
Call evaluate_confidence() when the run finishes, but only if it wasn't already called successfully earlier in the run (e.g. from a gate node).
NetworkHaltedError — is caught and logged instead of propagating out of graph.invoke()your graph(...) call. A manual call to evaluate_confidence() still raises normally.
observability_call: true in the payload sent to the backend (false for a normal, manually-invoked call), so it can tell automatic pings apart from explicit ones.
get_strands_single_call_guard never supports a manual evaluate_confidence() call — always pass ALWAYS or IF_NOT_EVALUATED and read the outcome off guard.trellar_evaluate_result (an AgentLoopResult, or None if not yet evaluated) and guard.trellar_evaluate_error (the exception, if the auto-triggered call failed).
get_single_call_guard never supports a manual evaluate_confidence() call — always pass ALWAYS or IF_NOT_EVALUATED and read the outcome off guard.trellar_evaluate_result (an AgentLoopResult, or None if not yet evaluated) and guard.trellar_evaluate_error (the exception, if the auto-triggered call failed).
import { ObservabilityMode } from "trellar";
import { trellarLangchainAgent } from "trellar/langchain";
const trellarAgent = trellarLangchainAgent("research-agent", ObservabilityMode.IF_NOT_EVALUATED);
await graph.invoke(inputs, { callbacks: [trellarAgent] });
// evaluateConfidence() has already run automatically if no node called it.
import { ObservabilityMode } from "trellar";
import { trellarStrandsAgent } from "trellar/strands";
const trellarAgent = trellarStrandsAgent("research-agent", ObservabilityMode.IF_NOT_EVALUATED);
// ...register `trellarAgent` via `plugins: [trellarAgent]` on the Graph...
await graph.invoke("Find me something to report on");
// evaluateConfidence() has already run automatically if no node called it.
Available modes
Never auto-call ("none"). Identical to not passing an ObservabilityMode at all — call evaluateConfidence() yourself wherever you need a score.
Always call evaluateConfidence() when the run finishes ("always") — in addition to any manual calls already made during the run.
Call evaluateConfidence() when the run finishes ("if_not_evaluated"), but only if it wasn't already called successfully earlier in the run (e.g. from a gate node).
NetworkHaltedError — is caught and logged instead of propagating out of graph.invoke(). A manual call to evaluateConfidence() still throws normally.
observability_call: true in the payload sent to the backend (false for a normal, manually-invoked call), so it can tell automatic pings apart from explicit ones.
trellarStrandsSingleCall never supports a manual evaluateConfidence() call. It defaults to ObservabilityMode.ALWAYS — pass ObservabilityMode.NONE to only record — and you read the outcome off trellarAgent.trellarEvaluateResult (an AgentLoopResult, or null if not yet evaluated) and trellarAgent.trellarEvaluateError (the error, if the auto-triggered call failed).
trellarLangchainSingleCall never supports a manual evaluateConfidence() call. It defaults to ObservabilityMode.ALWAYS — pass ObservabilityMode.NONE to only record — and you read the outcome off trellarAgent.trellarEvaluateResult (an AgentLoopResult, or null if not yet evaluated) and trellarAgent.trellarEvaluateError (the error, if the auto-triggered call failed).
Usage Patterns
Call evaluate_confidence() from a graph node, at the point in the run you want scored — it must run while the graph is still in progress, since the guard is released as soon as the root run ends. There are three common patterns:
The Strands snippets below use the ConfidenceNode shape from the Quick Start — a non-LLM node whose invoke_async awaits asyncio.to_thread(evaluate_confidence). as_agent_result(prompt) stands for the AgentResult(...) construction shown there, which passes the previous step's output to the next node.
Call evaluateConfidence() from a graph node (or, in Strands, a tool), at the point in the run you want scored — it must run while the graph is still in progress, since the Trellar agent is released as soon as the root run ends. The payload is the events captured so far; later nodes are not included. There are three common patterns:
Gate
Validate before the graph continues — halt or branch based on result.score.
def confidence_gate(state):
result = evaluate_confidence()
if result.score < 7:
return {**state, "halt": True, "reason": result.explanation}
return {**state, "halt": False}
class LowConfidenceError(Exception):
pass
# Same node shape as the Quick Start. Raising stops the
# graph, so the next node never runs.
class ConfidenceGate:
name = "confidence_gate"
async def invoke_async(self, prompt=None, **kwargs):
result = await asyncio.to_thread(evaluate_confidence)
if result.score < 7:
raise LowConfidenceError(result.explanation)
return as_agent_result(prompt) # wrap prompt in an AgentResult
Observe
Record a score without restricting the graph — the run continues either way.
def report_confidence(state):
result = evaluate_confidence()
return {**state,
"confidence_score": result.score,
"confidence_explanation": result.explanation}
class ConfidenceObserver:
name = "report_confidence"
async def invoke_async(self, prompt=None, **kwargs):
result = await asyncio.to_thread(evaluate_confidence)
# log it, but don't branch on it
print(result.score, result.explanation)
return as_agent_result(prompt)
Safety net
Gate on the runs that matter, and let IF_NOT_EVALUATED catch everything else — a score is always on record.
guard = get_agent_guard(
"research-agent",
ObservabilityMode.IF_NOT_EVALUATED,
)
graph.invoke(inputs, config={"callbacks": [guard]})
# a manual gate node may or may not have called
# evaluate_confidence() already — either way, a score
# is now on record for this run.
guard = get_strands_guard(
"research-agent",
ObservabilityMode.IF_NOT_EVALUATED,
)
# guard on every Agent + set_hook_providers([guard])
graph("Find me something to report on")
# a manual gate node may or may not have called
# evaluate_confidence() already — either way, a score
# is now on record for this run.
The Strands snippets below put the manual call in a tool the agent invokes right before the risky step. Make sure that step's agent is instructed to call it first — a tool is only a gate if it actually runs.
Gate
Validate before the graph continues — halt or branch based on result.score.
async function confidenceGate(state: State) {
const result = await evaluateConfidence();
if (result.score < 7) return { halt: true, reason: result.explanation };
return { halt: false };
}
import { tool } from "@strands-agents/sdk";
import { z } from "zod";
import { evaluateConfidence } from "trellar";
const confidenceGate = tool({
name: "confidence_gate",
description: "Check confidence in the run so far. Call before executing a deal.",
inputSchema: z.object({}),
callback: async () => {
const result = await evaluateConfidence();
if (result.score < 7) return `HALT: ${result.explanation}`;
return "OK";
},
});
Observe
Record a score without restricting the graph — the run continues either way.
async function reportConfidence(state: State) {
const result = await evaluateConfidence();
return {
...state,
confidenceScore: result.score,
confidenceExplanation: result.explanation,
};
}
const reportConfidence = tool({
name: "report_confidence",
description: "Record confidence in the run so far.",
inputSchema: z.object({}),
callback: async () => {
const result = await evaluateConfidence();
// log it, but don't branch on it
console.log(result.score, result.explanation);
return "recorded";
},
});
No node needed if you only want every run scored — set ObservabilityMode.ALWAYS instead.
Safety net
Gate on the runs that matter, and let IF_NOT_EVALUATED catch everything else — a score is always on record.
const trellarAgent = trellarLangchainAgent(
"research-agent",
ObservabilityMode.IF_NOT_EVALUATED,
);
await graph.invoke(inputs, { callbacks: [trellarAgent] });
// a manual gate node may or may not have called
// evaluateConfidence() already — either way, a score
// is now on record for this run.
const trellarAgent = trellarStrandsAgent(
"research-agent",
ObservabilityMode.IF_NOT_EVALUATED,
);
// plugins: [trellarAgent] on the Graph
await graph.invoke("Find me something to report on");
// a manual gate tool may or may not have called
// evaluateConfidence() already — either way, a score
// is now on record for this run.
Parallel runs
Several runs can execute in parallel in the same process. Give each its own Trellar agent, and each evaluateConfidence() call evaluates its own run.
await Promise.all([
graph.invoke(inputA, { callbacks: [trellarLangchainAgent("agent-a")] }),
graph.invoke(inputB, { callbacks: [trellarLangchainAgent("agent-b")] }),
]);
await Promise.all([
graphA.invoke(inputA), // graphA registered with trellarStrandsAgent("agent-a")
graphB.invoke(inputB), // graphB registered with trellarStrandsAgent("agent-b")
]);
If you call evaluateConfidence() from code you started yourself (for example a background task spawned while several runs are open), wrap it so Trellar knows which run you mean:
import { evaluateConfidence, runWithTrellarAgent } from "trellar";
await runWithTrellarAgent(trellarAgent, () => evaluateConfidence());
AI Setup Prompt
Don't want to wire this up by hand? Copy this prompt into Cursor, Claude Code, or any other coding agent — paste it into a chat in your repo and it will read your code and connect it to Trellar for you.
Connect to Trellar
Paste into your coding agent for your first integration. It checks how you use LangChain / LangGraph, wires up get_agent_guard() in observability mode, and sets your API key. No big code changes.
I want to connect this codebase to trellar.io. Trellar adds protection and readability to my multi-agent graph executions.
Libraries:
- Trellar (Python): https://pypi.org/project/trellar/
- LangChain: https://pypi.org/project/langchain/
- LangGraph: https://pypi.org/project/langgraph/
Keep the changes small - just connect my existing agents to Trellar, don't restructure my code. For now use observability mode IF_NOT_EVALUATED and don't add any manual evaluate_confidence() calls.
1. First, look at how this project uses LangChain / LangGraph and tell me in a line or two: does it run a graph (StateGraph, .compile(), create_react_agent), or only bare chat-model calls with no graph? It may have both.
2. Install: pip install "trellar[langchain]" (Python 3.9+), using my project's existing package manager / lockfile.
3. For each graph: create guard = get_agent_guard("name-here", ObservabilityMode.IF_NOT_EVALUATED) and pass config={"callbacks": [guard]} on every invoke/stream call of that graph (merge with any callbacks already passed).
For each bare chat-model call with no graph: use get_single_call_guard("name-here", ObservabilityMode.IF_NOT_EVALUATED) the same way. Both come from `from trellar import ...`, along with ObservabilityMode.
4. Names: give every Trellar name a stable, unique, descriptive value that says what that graph does (e.g. "customer-support-triage", not "agent1"). Use kebab-case, and keep it constant across runs and deploys - Trellar uses it to track that graph over time.
5. API key: tell me to create a Trellar API key at https://trellar.io/agent-gateway/apikeys/ (copy it right away, it's shown once). Then add it as an environment variable named TRELLAR_API_KEY wherever this project's container / deployment env vars are defined (Dockerfile, docker-compose, Kubernetes manifest, ECS task definition, .env, CI secrets - whatever this repo uses). Ask me for the value, never hard-code it.
6. Finish with a short summary: the files you changed and the name you gave each graph.
Connect to Trellar
Paste into your coding agent for your first integration. It checks how you use Strands Agents, wires up get_strands_guard() in observability mode, and sets your API key. No big code changes.
I want to connect this codebase to trellar.io. Trellar adds protection and readability to my multi-agent graph executions.
Libraries:
- Trellar (Python): https://pypi.org/project/trellar/
- Strands Agents: https://pypi.org/project/strands-agents/
Keep the changes small - just connect my existing agents to Trellar, don't restructure my code. For now use observability mode IF_NOT_EVALUATED and don't add any manual evaluate_confidence() calls.
1. First, look at how this project uses Strands Agents and tell me in a line or two: does it build a multi-agent graph (GraphBuilder, builder.build()), or only single Agent(...) calls with no graph? It may have both.
2. Install: pip install "trellar[strands]" (Python 3.10+), using my project's existing package manager / lockfile.
3. For each graph: create ONE guard = get_strands_guard("name-here", ObservabilityMode.IF_NOT_EVALUATED), pass hooks=[guard] to EVERY Agent(...) in that graph, and also call builder.set_hook_providers([guard]) on its GraphBuilder (merge with any hooks already passed). Give every Agent a stable name= too.
For each single Agent called directly with no graph: use get_strands_single_call_guard("name-here", ObservabilityMode.IF_NOT_EVALUATED) and pass it as Agent(..., hooks=[guard]). Both come from `from trellar import ...`, along with ObservabilityMode.
4. Names: give every Trellar name a stable, unique, descriptive value that says what that graph does (e.g. "customer-support-triage", not "agent1"). Use kebab-case, and keep it constant across runs and deploys - Trellar uses it to track that graph over time.
5. API key: tell me to create a Trellar API key at https://trellar.io/agent-gateway/apikeys/ (copy it right away, it's shown once). Then add it as an environment variable named TRELLAR_API_KEY wherever this project's container / deployment env vars are defined (Dockerfile, docker-compose, Kubernetes manifest, ECS task definition, .env, CI secrets - whatever this repo uses). Ask me for the value, never hard-code it.
6. Finish with a short summary: the files you changed and the name you gave each graph.
Connect to Trellar
Paste into your coding agent for your first integration. It checks how you use LangChain / LangGraph, wires up trellarLangchainAgent() in observability mode, and sets your API key. No big code changes.
I want to connect this codebase to trellar.io. Trellar adds protection and readability to my multi-agent graph executions.
Libraries:
- Trellar (npm): https://www.npmjs.com/package/trellar
- LangChain: https://www.npmjs.com/package/@langchain/core
- LangGraph: https://www.npmjs.com/package/@langchain/langgraph
Keep the changes small - just connect my existing agents to Trellar, don't restructure my code. For now use observability mode IF_NOT_EVALUATED and don't add any manual evaluateConfidence() calls.
1. First, look at how this project uses LangChain / LangGraph and tell me in a line or two: does it run a graph (StateGraph, createAgent, .compile()), or only bare chat-model calls with no graph? It may have both.
2. Install: npm install trellar @langchain/core @langchain/langgraph (Node.js 20+), using my project's existing package manager / lockfile and skipping what's already installed.
3. For each graph: create const trellarAgent = trellarLangchainAgent("name-here", ObservabilityMode.IF_NOT_EVALUATED) and pass it as { callbacks: [trellarAgent] } in the config of every invoke/stream call of that graph (merge with any callbacks already passed). Forward the node config into nested runnables (model.invoke(msgs, config)) so child runs are reported.
For each bare chat-model call with no graph: use trellarLangchainSingleCall("name-here", ObservabilityMode.IF_NOT_EVALUATED) the same way. Import trellarLangchainAgent / trellarLangchainSingleCall from "trellar/langchain" and ObservabilityMode from "trellar".
4. Names: give every Trellar name a stable, unique, descriptive value that says what that graph does (e.g. "customer-support-triage", not "agent1"). Use kebab-case, and keep it constant across runs and deploys - Trellar uses it to track that graph over time.
5. API key: tell me to create a Trellar API key at https://trellar.io/agent-gateway/apikeys/ (copy it right away, it's shown once). Then add it as an environment variable named TRELLAR_API_KEY wherever this project's container / deployment env vars are defined (Dockerfile, docker-compose, Kubernetes manifest, ECS task definition, .env, CI secrets - whatever this repo uses). Ask me for the value, never hard-code it.
6. Finish with a short summary: the files you changed and the name you gave each graph.
Connect to Trellar
Paste into your coding agent for your first integration. It checks how you use Strands Agents, wires up trellarStrandsAgent() in observability mode, and sets your API key. No big code changes.
I want to connect this codebase to trellar.io. Trellar adds protection and readability to my multi-agent graph executions.
Libraries:
- Trellar (npm): https://www.npmjs.com/package/trellar
- Strands Agents: https://www.npmjs.com/package/@strands-agents/sdk
Keep the changes small - just connect my existing agents to Trellar, don't restructure my code. For now use observability mode IF_NOT_EVALUATED and don't add any manual evaluateConfidence() calls.
1. First, look at how this project uses Strands Agents and tell me in a line or two: does it run a multi-agent Graph / Swarm, or only single Agent calls with no Graph? It may have both.
2. Install: npm install trellar @strands-agents/sdk (Node.js 22+), using my project's existing package manager / lockfile and skipping what's already installed.
3. For each Graph / Swarm: create ONE const trellarAgent = trellarStrandsAgent("name-here", ObservabilityMode.IF_NOT_EVALUATED) and register it as plugins: [trellarAgent] on the Graph / Swarm itself (merge with any plugins already passed). Give every Agent a stable name and a unique id.
For each single Agent called directly with no Graph: use trellarStrandsSingleCall("name-here", ObservabilityMode.IF_NOT_EVALUATED) and pass it as new Agent({ ..., plugins: [trellarAgent] }). Import trellarStrandsAgent / trellarStrandsSingleCall from "trellar/strands" and ObservabilityMode from "trellar".
4. Names: give every Trellar name a stable, unique, descriptive value that says what that graph does (e.g. "customer-support-triage", not "agent1"). Use kebab-case, and keep it constant across runs and deploys - Trellar uses it to track that graph over time.
5. API key: tell me to create a Trellar API key at https://trellar.io/agent-gateway/apikeys/ (copy it right away, it's shown once). Then add it as an environment variable named TRELLAR_API_KEY wherever this project's container / deployment env vars are defined (Dockerfile, docker-compose, Kubernetes manifest, ECS task definition, .env, CI secrets - whatever this repo uses). Ask me for the value, never hard-code it.
6. Finish with a short summary: the files you changed and the name you gave each graph.