Test File

A test suite is a YAML file describing one flow and the cases that exercise it. It lives beside the flows it tests and is named for them — orders.yaml is tested by orders_test.yaml, the way orders.go is tested by orders_test.go. octo skips _test.yaml when it loads a directory, so a suite sits next to its flows without ever being mistaken for one.

This page is the canonical reference for every key. For the narrative version, with worked examples, read writing test cases.

A case is a debug config with assertions: the input, the mocks and the spied blocks are the same ones octo invoke --run-debug-config takes.

Top-level keys

flow:     # the flow under test
inputs:   # named messages the cases share
mocks:    # blocks to stand in for, in every case
env:      # environment variables every case runs with
timeout:  # how long any one case may take
cases:    # the scenarios

A file-level mocks: block is where "no test in this file ever reaches the payment API" is said once.

inputs

Named messages, so the body that means a vip order is written once and referred to by name. A case may also write its input inline.

inputs:
  small:
    data: { amount: 10 }
  a vip order:
    data: { orderId: 42, amount: 250 }
    vars: { x-api-key: dev-key }

cases

Each case is one octo invoke, in its own process, run with the mocks and spies it asks for.

cases:
  - name: a declined card takes the flow error path and answers 502
    input: large
    expect:
      vars: { httpStatus: 502 }
      that: ['body.error.contains("card declined")']

Two merge rules, and they differ on purpose. A case's mocks replace the file's per address, whole — so a case's mock says exactly what that block will do, without the reader having to hold the file's version in their head too. A case's env overrides the file's per variable, because a variable is a scalar: a case that needs one value different should not have to restate the others to get it.

skip requires a reason. A skip with no reason is a test nobody will ever come back to.

expect

What the flow should have done. Omitting expect entirely still asserts something: that the flow completed and did not fail.

A flow either produces a message, drops it, or fails — so dropped and error are exclusive with each other and with the message fields. A flow that dropped or failed returned no message to check, and dolphin rejects a case that asks for two.

Why body is exact but vars is a subset. The body is the flow's answer, and a test earns its keep by failing when a field appears in it that nobody expected. Variables are scratch space the engine and the blocks add to, so pinning the whole map would break on every unrelated change. Reach for that: ['vars.size() == 2'] when you do want it exact.

The error match is a substring: a case expecting card declined that passed on any failure would prove the wrong thing.

The CEL in that

The same CEL a mock's when and every block in the flow uses, over the same variables: body, vars, eventID, correlationID, now.

An expression that evaluates to something other than true or false is rejected rather than read as truthy — a non-boolean is a mistake, and reporting it as a false would hide it.

mocks

How one block is stood in for. Keyed by block address, which works unquoted as a YAML key.

At least one of the two is required.

mocks:
  charge-flowlevel.call-charge:
    cases:
      - when: 'body.amount > 100'
        error: card declined
    default:
      body: { id: ch_1, status: captured }

A mock case

Exactly one of body, error and drop per case. when is the only expression a mock carries: the outcome is a literal, because a mock is a canned response and the dispatch already happened in when.

spies

What one watched block should have seen. Keyed by block address.

A block inside a fork branch or a foreach body is crossed once per branch or item, and every crossing is recorded — so the count is the iteration count.

count: 0 is an assertion, not an absence: it says the block never ran, which is how a case proves a branch was not taken or an error path not reached. It is kept distinct from asserting nothing at all.

Records are positional: the first entry is the first crossing. Asserting on fewer crossings than happened is fine; naming more than happened is a failure.

spies:
  demo.each-order[body].classify-order:
    count: 2
    records:
      - input: { that: ['vars.order.id == 1'] }
      - input: { that: ['vars.order.id == 2'] }

A crossing

output, dropped and error are exclusive — a block either returns a message, drops it, or fails. input is always available, even for a block that failed: it is captured before the block runs, so a failure still shows what the block was carrying.

input and output each take body (exact), vars (subset) and that (CEL), the same three fields expect uses for a message.

Unknown keys are rejected

dolphin decodes a suite with unknown fields disallowed, and refuses the whole file over one. This is deliberate, and it is the single most valuable piece of strictness in the format: a misspelled spys: would otherwise watch nothing, assert nothing, and go green — a test that passes by doing nothing at all.

The editor's Testing tab reports the same thing before you run, and disables the form rather than re-serializing a file it cannot fully read.

Generating the schema

To have your editor complete a suite and a validator check it:

dolphin schema --format yaml    # or --format json, the default
dolphin schema --out test.schema.json

See also

• Writing test cases — the same format, explained through worked examples.
• Running tests — dolphin test, its flags, and the exit codes.
• CLI reference — the block-address grammar mocks and spies are keyed by.
• CEL reference — the expression language that and when use.