LLM Providers

Octo ships four LLM provider connectors, llm-anthropic, llm-openai, llm-gemini, and llm-openrouter, that all satisfy the same client interface. Every AI block references a provider by connector name, so switching providers is a connector-level change. The first three talk to one vendor each; llm-openrouter talks to OpenRouter, which fronts hundreds of models from many vendors behind a single key.

Configuring a provider

connectors:
  - name: claude
    type: llm-anthropic
    settings:
      apiKey: ${ANTHROPIC_API_KEY}
      # model: claude-sonnet-4-6      # default
      # maxTokens: 4096               # default response cap
      # baseURL: https://...          # for proxies or testing

All four connectors take the same four settings:

Each also takes settings of its own: reasoning and thinking effort, and for OpenRouter the two attribution headers. See the connector reference. A connector validates its settings at startup, so a missing apiKey fails the service immediately rather than on the first request.

One interface, any provider

The LLM connectors register under category llm, and every AI block (ai-agent, ai-router, ai-mapping, ai-retry) binds to one through the shared client interface, by name:

      - type: ai-mapping
        settings:
          connector: claude    # any llm-* connector name works here
          prompt: "..."

Repoint claude at a different type, or configure several providers side by side. Referencing a connector that is not an LLM provider is a startup error.

OpenAI-compatible and local endpoints

The llm-openai connector's baseURL targets any server that speaks the OpenAI Responses API: a corporate proxy, Azure OpenAI, or a local runtime such as Ollama or vLLM:

connectors:
  - name: local
    type: llm-openai
    settings:
      apiKey: ${LOCAL_LLM_KEY}     # many local servers accept any non-empty key
      model: llama3.1
      baseURL: http://localhost:11434/v1

Every AI block then runs against the local model with no other changes. The baseURL on llm-anthropic, llm-gemini and llm-openrouter serves the same purpose for proxies and testing.

Key hygiene

Never put a literal API key in a flow file. Declare the variable in the env section and reference it with ${...} substitution:

env:
  - name: ANTHROPIC_API_KEY
    required: true

connectors:
  - name: claude
    type: llm-anthropic
    settings:
      apiKey: ${ANTHROPIC_API_KEY}

required: true makes a missing key a startup failure with a clear message. The flow file stays committable and the connectors never log the key.

Choosing models

The model setting is per connector, and a flow can define several connectors against the same provider, so match the model to each block's job. High-volume, narrow tasks (routing, mapping, formatting) favor a smaller model; the Slack agent capstone runs its whole pipeline on claude-haiku-4-5 with a 1200 token cap. Agents reason across multiple tool calls, so a more capable model can pay for itself in fewer iterations. maxTokens is a cost and latency guard: set it near the size of the output you expect (a router's decision is tiny; a summary is not).

Model ids pass through to the provider as-is, so new models work as soon as your account has access: update the setting and restart.