Performance Tuning
What workers, buffer, pool and the connector pools govern, and how to pick their values.
A flow's throughput is set by a handful of concurrency settings and the connection pools behind them. This page says what each one governs, gives the one rule that predicts when they bind, and walks you through sizing them for a flow that waits on something.
The settings
| Setting | Where | Default | What it governs |
|---|---|---|---|
workers | root flow | max(8, GOMAXPROCS × 4) | The most messages the flow processes at once. Each worker runs one message through the whole chain. |
buffer | root flow | max(64, workers) | The depth of the channel between the source and the workers. It absorbs bursts; it adds no capacity. |
pool | root flow | 8 | The workers of the shared pool that fork branches and resumed split continuations run on, in front of a fixed queue of 64 tasks. Every worker of the flow submits into this one pool. |
listeners | queue source | 8 | Concurrent handlers for a queue subscription, per replica. A handler holds its slot until the flow finishes, so this, not workers, caps a queue-fed flow. |
maxOpenConns, maxIdleConns | database connector | 16 (postgres), 1 (sqlite) | How many connections the flows bound to the connector can use at once. See database. |
pool.maxIdleConnsPerHost | http-client connector | 100 | Idle outbound connections kept for reuse. See http. |
The runtime logs each flow's resolved workers at startup, marked derived when it came from the default and configured when you set it.
It also warns when a flow's workers exceed what something the flow drives can serve at once: the maxOpenConns of a database connector, the maxPoolSize of a mongodb connector, or the listeners of its queue source. The warning names the flow, the connector, both numbers and the setting to change:
WARN flow workers exceed connector capacity flow=orders connector=db workers=32 workersOrigin=derived maxOpenConns=16 fix="raise maxOpenConns to 32, or set the flow's workers to 16"The warning changes nothing; the extra workers still wait on the pool. Several flows sharing one connector are each checked on their own, so size the pool for their combined load too.
The processing pipeline explains how the workers, the buffer and the pool fit together.
The rule
A worker that waits on a database or an upstream API is doing nothing else for that time. So a flow can sustain at most workers ÷ seconds blocked per message requests a second, and to reach a target you need:
workers ≥ target_rps × seconds_blocked_per_messageTwo measurements, both at workers: 8, show how closely it predicts:
| Flow | Blocked per message | Predicted | Measured |
|---|---|---|---|
| Postgres CRUD, three round trips | ≈ 4.5 ms | ≈ 1,780 req/s | 1,664 req/s |
| HTTP proxy to a 70 ms backend | 70 ms | ≈ 114 req/s | 108 req/s |
Size a flow
- Check whether the flow blocks. Look for
sql,rest,queue-dispatchwithawaitReply: true, an LLM call, or any other outbound call. If there is none, stop: tuning will not change anything, and that is a result, not a failure. - Measure the blocked time per message. Add up the round trips, or run the flow at low load with
--metricsand readocto_flow_duration_seconds. At low load nothing queues, so the duration is the time spent working and waiting. - Compute
workersfrom the rule, then round up to the next power of two for headroom. For a flow fed by aqueuesource, setlistenersto the same number: the source hands the flow no more thanlistenersmessages at a time, however many workers wait. - Size the connector pools to match. Thirty-two workers sharing a
databaseconnector withmaxOpenConns: 4queue for connections instead of for workers; the bottleneck moves, it does not go away. Give the pool at least as many connections as the busiest flow using it has workers, within what the database server allows across all replicas. The startup warning above flags a pool smaller than a flow'sworkers. - Leave
bufferalone unless arrivals come in bursts. A deeper buffer turns rejected work into waiting work; it does not process anything faster. - Raise
poolwithworkersif the flow forks. Every worker submits its message's branches into the one shared pool, which holdspoolrunning tasks plus 64 queued ones and panics when that overflows, rather than risk a deadlock. So it must holdworkers × branches per message: at 32 workers, a three-branchforkneeds room for 96 tasks, which the defaultpool: 8does not have. - Verify with
octo_flow_in_flight. If it sits at the worker count (or, for a queue-fed flow, atlisteners) while throughput stays flat, every slot is still waiting and the limit is still too low.octo_flow_queue_depthsays by how much: a backlog that keeps growing means messages arrive faster than the workers drain them, and one held atocto_flow_queue_capacitymeans the source is blocked.
The symptom points the wrong way
A flow with too few workers looks like a slow dependency. Latency climbs into seconds at moderate load, CPU is idle, and there are no errors and no log lines. In the Postgres measurement above, CPU peaked at 136% of one core and then fell to 42% while latency kept rising.
An idle process that is slow sends you to look at the database or the network, and both are fine. Check octo_flow_in_flight first: pinned at the worker count means the messages are waiting for a worker, not for the dependency.
When tuning does nothing
A flow that never waits (transforms, routing, CEL, in-memory lookups) is limited by CPU, and the default workers already keeps every core busy. Raising it adds goroutines that compete for the same cores. workers is not a general performance dial: raise it for flows that block, and only as far as the rule says.