SDK Reference
Core Concepts
Four building blocks underpin everything in Niitaka: Sessions, Events, Policies, and Runtime Config. Understanding these makes the rest of the SDK and dashboard obvious — plus two sets of finer-grained controls that build on Sessions and Events directly: cross-process sessions and logging control.
Session
A session is one complete run of your agent — everything from start to finish, scoped to your organisation.
- Created by start_session() and closed when the context manager exits.
- Tracks total cost, total tokens, error count, duration, and status.
- "error" is a real, reachable terminal status (running → done/error/stopped/stale) — but it's not authoritative for error detection. error_count isn't a stored column either; it's derived at query time from events where type="error". Detect errors by counting those events, not by filtering on session status.
- Every event inside the session is linked to it via session_id.
- create_session() is a fire-and-forget alternative to start_session() for when you want a session row to exist (e.g. for a downstream process to attach_session() into) without wrapping your own code in its context manager.
python
with niitaka.start_session(
goal="Analyse customer churn",
agent_id="churn-analyser",
) as session:
# your agent code here
pass
# Access session ID if needed
print(session.session_id)Cross-Process Sessions
attach_session() lets a downstream service log into a session an upstream caller already opened, instead of creating an unrelated top-level session per call.
- Redirects where auto-instrumented events log for the duration of the block — no session created, no policy fetch, no heartbeat.
- Pair with policy_scope() to also carry enforcement (cost_limit/step_limit/retry/fallback) across the same process boundary — attach_session() alone only controls visibility.
- attach_session(starting_sequence=...) keeps event ordering correct on a long-running, multi-threaded process that attaches to many sessions over its lifetime.
- See the Guardrails guide for the full cross-process scope-aggregation pattern (export_state() / imported_state=).
python
# Upstream service already has a real session open for this request.
# A downstream service/process can log into that SAME session instead of
# creating its own unrelated top-level one:
with niitaka.attach_session(session_id):
# auto-instrumented calls here log as typed events under session_id
call_llm(...)
# attach_session(None) is a true no-op — safe to pass an optional/absent header straight through.
# Controls WHERE events log, not policy enforcement — combine with
# policy_scope() for retry/fallback/cost_limit across the same boundary.
# See the Guardrails guide for the full cross-process scope-aggregation pattern.Event
Events are the structured log entries that make up a session — one per LLM call, tool invocation, or decision.
- Captured automatically by instrumentation — you never write event logs by hand.
- Events can be nested: a tool call event can have child LLM call events.
- Real latency lives in data.meta.latency (seconds). The duration_ms field is always 0 — ignore it.
- Event types: llm, tool, decision, error.
python
# Events are captured automatically — you don't log them manually.
# Every instrumented LLM call becomes an event like:
{
"type": "llm",
"data": {
"model": "gpt-4o",
"prompt_tokens": 512,
"completion_tokens": 128,
"cost_usd": 0.00384,
"meta": {
"latency": 1.43 # seconds — always read from here
}
}
}Logging Control
Finer control over what gets logged and where, for proxy/gateway and multi-client setups.
- suppress_logging() — enforce policy without recording anything. Cost/step accounting and spike signals still run, so later calls in the same scope stay correctly gated.
- log_independent_event() — a fire-and-forget event POST that touches none of the active session's sequence counter or scope accounting. For a second NiitakaClient instance logging to its own session on a thread another instance is already using.
- patch_event_cost() — retroactively corrects an event's cost once it's known (a streaming response's real usage is only available after the stream exhausts). Session total_cost adjusts by the delta.
python
# suppress_logging() — enforce policy without recording content
# (e.g. the end user opted out of observability for this one request):
with niitaka.suppress_logging():
call_llm(...) # cost_limit/step_limit/retry/fallback still run; nothing is logged
# log_independent_event() — a second NiitakaClient instance logging into its
# own session on a thread where another instance already has one active,
# without touching that other session's sequence counter or cost/step accounting:
other_client.log_independent_event(session_id=obs_session_id, event_type="tool", data={...})
# patch_event_cost() — correct a streaming call's cost after the fact, once
# usage is known (raw provider chunks carry no usage until reconstructed):
niitaka.patch_event_cost(event_id, cost=0.00412)Policy
Policies are rules the Niitaka backend evaluates in real time as events arrive — without any code changes.
- Configured in the dashboard (or via YAML file), versioned, and audited.
- Policy types: cost_limit, step_limit, retry, fallback.
- Actions: warn, abort, retry, fallback (switch to a different model).
- Policies apply per-agent and execute in priority order.
python
# Policies are configured in the Niitaka dashboard, not in code.
# Example policy shape (YAML):
- type: cost_limit
condition:
threshold_usd: 0.50
action: abort
priority: 1
- type: retry
condition:
on_errors: ["rate_limit_error", "timeout"]
action:
max_retries: 3
backoff_seconds: 2
priority: 2Runtime Config
Runtime config lets the backend push model, prompt, and guardrail settings to your agent at session start — no redeploy needed.
- Schema: { llm: { model, temperature, max_tokens, system_prompt }, guardrails: { cost_limit_usd, max_steps, ... } }
- Three sources in priority order: active experiment variant, pinned agent version, live policies.
- Fetched automatically by start_session(), or explicitly via get_runtime_config().
- Lets you change model or prompt for all running agents from the dashboard instantly.
python
# SDK auto-fetches config at session start — or call explicitly:
config = niitaka.get_runtime_config(agent_id="my-agent")
model = config["llm"]["model"]
temperature = config["llm"]["temperature"]
system_prompt= config["llm"]["system_prompt"]
cost_limit = config["guardrails"]["cost_limit_usd"]Was this page helpful?