SDK Reference

Runtime Config

Runtime Config is the mechanism that lets the Niitaka backend push model selection, prompt content, and guardrail thresholds to your agent at session start — without a code change or redeployment. When your agent callsget_runtime_config(), it receives the configuration that is currently active for that agent, from whichever source has the highest priority.

Why use Runtime Config?

Ship model changes instantly — switch from GPT-4o to Gemini Flash for all running agents from the dashboard. No PR, no deploy, no downtime.
Experiment without code branches — variant configs for A/B tests are injected at session start. Your agent code stays identical across variants.
Centralise prompt management — store system prompts in the dashboard rather than scattered across agent repositories. Edit in one place, propagate everywhere.
Instant rollback — if a new config causes issues, revert from the dashboard. Takes effect on the next session with no deployment required.

Config schema

The runtime config object has two top-level keys: llm for model settings and guardrails for safety limits.

json

# Full runtime config schema
{
  "llm": {
    "model":         "gpt-4o-mini",   # model identifier
    "temperature":   0.3,             # float 0–2
    "max_tokens":    512,             # int
    "system_prompt": "You are ..."    # string
  },
  "guardrails": {
    "cost_limit_usd": 0.25,           # abort threshold in USD
    "max_steps":      50,             # maximum event count
    "retry":          true,           # whether retry policy is active
    "fallback":       "gpt-4o-mini"   # fallback model, if any
  }
}

Config resolution order

When get_runtime_config() is called, Niitaka resolves the config from three potential sources, in this priority order:

ExperimentVariant

Active when session is in a running experiment

highest priority

AgentVersion

Active pinned version for the agent

Policies table

Base guardrail defaults

fallback

ExperimentVariantHighest priority. When the session is part of a running experiment, the variant assigned to this session provides its config. This is how A/B test variants deliver different models and prompts to different sessions.

AgentVersionIf no experiment is active, the pinned active version of the agent provides the config. Update the active version in the Dashboard to change the config for all sessions not in an experiment.

Policies tableFallback. Guardrail values (cost_limit_usd, max_steps) are always read from the Policies table when not overridden by a variant or version. LLM settings not set in the active version also fall through to policy defaults.

Note:Fields are merged across sources — you do not need to specify every field at every level. A variant can override only model and temperature while the agent version continues to provide system_prompt.

Using runtime config in your agent

Before and after

Without runtime config, every setting is hardcoded — changing anything requires a deploy:

python

# Before: config hardcoded in agent source
import openai

client = openai.OpenAI()

with niitaka.start_session(goal="...", agent_id="report-summariser") as session:
    response = client.chat.completions.create(
        model="gpt-4o",                     # hardcoded — requires redeploy to change
        messages=[
            {"role": "system", "content": "You are a concise summariser."},
            {"role": "user",   "content": document},
        ],
    )

With runtime config, the agent reads settings from Niitaka at session start:

python

# After: config resolved from Niitaka at session start
import openai

client = openai.OpenAI()

with niitaka.start_session(goal="...", agent_id="report-summariser") as session:
    config = niitaka.get_runtime_config(agent_id="report-summariser")

    model         = config["llm"]["model"]
    temperature   = config["llm"]["temperature"]
    system_prompt = config["llm"]["system_prompt"]
    cost_limit    = config["guardrails"]["cost_limit_usd"]

    response = client.chat.completions.create(
        model=model,
        temperature=temperature,
        messages=[
            {"role": "system", "content": system_prompt},
            {"role": "user",   "content": document},
        ],
    )

Runtime config and experiments

When a session belongs to an experiment, the variant's config is returned automatically. Your agent code does not need to know which variant is active — it just calls get_runtime_config() and uses what it gets back.

python

# When a session is part of an experiment, the assigned variant's
# config is returned automatically — no extra code needed.
with niitaka.start_session(
    goal="...",
    agent_id="report-summariser",
    experiment_id="exp_a1b2c3",
) as session:
    config = niitaka.get_runtime_config(agent_id="report-summariser")
    # config now reflects the variant this session was assigned to
    model = config["llm"]["model"]

Prompt slots

Prompt slots let you define multiple system prompts in a single config and inject the right one based on what part of your agent is running — without any conditional logic in your code. Define named slots in the variant or version config, then tag call sites with niitaka.prompt_slot().

json

# In an experiment variant or agent version config, define named slots:
{
  "prompt_slots": {
    "__default__": { "system_prompt": "You are a general assistant." },
    "planner":     { "system_prompt": "You are a strategic planner. Think step by step." },
    "executor":    { "system_prompt": "You are a precise executor. Follow instructions exactly." }
  }
}

Use the context manager or decorator form to activate a slot around each LLM call. Instrumentation reads the active slot and substitutes the matching system prompt automatically — your agent code stays unchanged across variants.

python

import niitaka

# Context manager — tags the LLM call so the right slot is injected
with niitaka.start_session(goal="...", agent_id="my-agent"):
    with niitaka.prompt_slot("planner"):
        plan = llm.chat(...)          # receives the "planner" system prompt

    with niitaka.prompt_slot("executor"):
        result = llm.chat(...)        # receives the "executor" system prompt

# Decorator — applies the slot to every call inside the function
@niitaka.prompt_slot("planner")
def run_planning_step(context):
    return llm.chat(...)

# Slot resolution order (highest priority first):
#   1. prompt_slots[active_slot]  — exact match
#   2. prompt_slots[__default__]  — fallback if no matching slot
#   3. system_prompt (top-level)  — legacy / no-slot config
#   4. No override                — model uses its own default

Tip:The __default__ slot acts as a catch-all. Any call not wrapped in a named slot, or wrapped in a slot name that isn't defined in the config, falls back to __default__. This means you can add named slots incrementally without breaking existing calls.

Updating the config

Via the dashboard

Go to Agents → [your agent].
Click Runtime Config tab.
Edit any field and save. The change takes effect on the next session start.

Via the API

python

import requests

requests.patch(
    "https://api.niitaka.ai/agents/report-summariser/config",
    json={
        "llm": {
            "model":         "gpt-4o-mini",
            "temperature":   0.3,
            "system_prompt": "You are a concise summariser. Reply in under 3 sentences.",
        },
        "guardrails": {
            "cost_limit_usd": 0.15,
        },
    },
    headers={"Authorization": f"Bearer {JWT}"},
)

Tip:Partial updates are supported — you only need to include the fields you want to change. Omitted fields retain their current values.

Rollback

If a config change causes problems in production, rolling back is the same operation in reverse: update the config to the previous values via the dashboard or API. Because the config is fetched at session start, the rollback takes effect on the next session with no deployment.

For a versioned rollback, pin the agent to a previous AgentVersion from the Agents dashboard. This restores all LLM settings that were active at that version in one step.

Next steps

Shipping a winner — how experiment results promote a variant's config to the active AgentVersion.
Policies — set the guardrail defaults that runtime config falls back to.
Cost Management — use runtime config to roll out cheaper models validated by experiments.

Providers

Environments

Was this page helpful?

Need help? Contact Support Questions? Contact Sales LLM? Read llms.txt