Composing Flows

As an integration grows, one long process list gets hard to read and impossible to reuse. This guide splits the work into named flows and calls them with flow-ref, synchronously when you need the result and one-way when you don't, then shows when to compose over HTTP instead.

Sourceless flows

A flow without a source: block gets an implicit source and becomes callable by name. It never binds a port or fires a schedule, and only runs when something invokes it. The smallest example is samples/hello-invoke.yaml:

flows:
  - name: greet
    process:
      - type: set-payload
        name: build-greeting
        settings:
          value: '{"greeting": "hello, " + body.name + "!"}'

Call it directly from the CLI:

octo invoke --config samples/hello-invoke.yaml --flow greet --data '{"name":"Ada"}'
# -> {"event_id":"5c12…","body":{"greeting":"hello, Ada!"}}

In invoke mode no sources start; only the requested flow runs. Sourceless flows are the unit of composition: other flows call them with flow-ref, and you test each one in isolation with octo invoke.

Calling flows with flow-ref

samples/http-orders.yaml puts both invocation styles in one API. The HTTP-triggered flow delegates to two sourceless flows:

process:
  # Fire-and-forget: the caller does not wait, the result is ignored.
  - type: flow-ref
    name: audit-async
    settings:
      flow: audit
      oneWay: true

  # Synchronous (the default): wait for the called flow, then fold its
  # body and variables back into this message.
  - type: flow-ref
    name: enrich-sync
    settings:
      flow: enrich-order

Run it and post an order:

octo run --config samples/http-orders.yaml
curl -s -X POST localhost:8080/api/v1/orders/42 \
  -H 'X-Tenant: acme' -H 'X-Request-Id: req-1' \
  -d '{"item":"widget","amount":1500}'

The response comes from enrich-order (called synchronously), while audit logs its line in the background without delaying the request.

What the called flow sees

flow-ref sends the target flow a clone of the current message (same body, same variables, same correlation ID) under a fresh event ID, so the sub-invocation correlates independently in logs. That is why enrich-order can read vars.id, vars["X-Tenant"], and body.amount without being passed anything explicitly:

- name: enrich-order
  process:
    - type: set-payload
      name: normalize-order
      settings:
        value: >
          {
            "orderId":   vars.id,
            "tenant":    vars["X-Tenant"],
            "item":      body.item,
            "amount":    body.amount,
            "currency":  vars.currency,
            "requestId": correlationID
          }

Sync vs one-way

With oneWay omitted or false the call is synchronous: the caller waits for the called flow to finish, the result's body replaces the caller's body, and every variable the called flow set is merged into the caller's variables. If the called flow drops the message (a filter said no), the caller's message continues unchanged.

With oneWay: true the caller hands off the clone and moves on immediately, and the called flow's result is ignored. Use it for side effects: audit trails, notifications, metrics.

Composing over HTTP

When the flows belong to separate deployables (different configs, different lifecycles, maybe different teams), compose over HTTP instead: expose the callee as an endpoint and call it with the rest block. samples/flow-to-flow-http.yaml simulates this inside one config: a cron-driven caller POSTs to a greet-service flow served by the same process.

connectors:
  - name: internal
    type: http-client
    settings:
      baseURL: http://127.0.0.1:${HTTP_PORT}${HTTP_BASE_PATH}
      timeout: 5s

flows:
  - name: caller
    source:
      type: cron
      settings:
        schedule: "@every 5s"
        payload: '{"name": "world", "at": string(now)}'
    process:
      - type: rest
        name: call-flow-b
        settings:
          connector: internal
          method: POST
          path: /greet
          body: body
          headers:
            X-Request-Id: eventID
      - type: log
        name: report-reply
        settings:
          logger: out
          message: '"flow B replied: " + body.greeting'

Run it and watch the caller log the greeting the service returned every few seconds, or call the service yourself:

octo run --config samples/flow-to-flow-http.yaml
curl -s -X POST localhost:8080/api/v1/greet -d '{"name":"ada"}'

Propagating correlation IDs

In-process, flow-ref carries the correlation ID for free. Over HTTP you thread it through a header. The caller sends X-Request-Id, here the caller's eventID (forward correlationID instead to preserve an ID that arrived from upstream), and the callee's source names that header with correlationIdHeader:

- name: greet-service
  source:
    connector: api
    type: http
    settings:
      path: /greet
      correlationIdHeader: X-Request-Id
  process:
    - type: set-payload
      settings:
        value: '{"greeting": "Hello, " + body.name + "!", "requestId": correlationID}'

Both services then log the same ID for one logical request, and the callee can echo it back in its response.

Choosing between flow-ref and HTTP

Prefer flow-ref until you genuinely need a deployment boundary; you can lift a sourceless flow into its own service later by giving it an HTTP source.

Where to go next