LLM Providers

Octo ships three LLM provider connectors — llm-openai, llm-anthropic, and llm-gemini. All three register under the connector category llm and expose the same provider-agnostic completion capability, so the AI blocks (ai-agent, ai-router, ai-mapping, ai-retry) reference any of them interchangeably through their connector setting.

Provides a source: no — all three are service connectors. Blocks that bind to them: the AI blocks, which accept any connector in the llm category. See AI in Octo for usage.

Each connector concentrates provider policy — API key, model, and the default response token cap — and validates its configuration at startup, so a missing key fails fast rather than on first request. An AI block's own maxTokens overrides the connector default per request.

llm-openai connector

OpenAI — a configured Chat Completions client.

llm-anthropic connector

Anthropic — a configured Messages API client.

llm-gemini connector

Gemini — a configured Google Gemini client.

Example

Declare one or more providers and point an AI block at whichever you like — swapping providers is a one-line change:

service:
  name: ai-demo

env:
  - name: ANTHROPIC_API_KEY
    required: true

connectors:
  - name: claude
    type: llm-anthropic
    settings:
      apiKey: ${ANTHROPIC_API_KEY}
      model: claude-sonnet-4-6
      maxTokens: 2048

flows:
  - name: summarize
    process:
      - type: ai-mapping
        settings:
          connector: claude          # any llm-category connector works here
          prompt: Summarize the incoming order into a one-line description.