Test File

A test suite is a YAML file describing one flow and the cases that exercise it. It lives beside the flows it tests: orders.yaml is tested by orders_test.yaml, the way orders.go is tested by orders_test.go. octo skips _test.yaml when it loads a directory.

This page is the canonical reference for every key. For worked examples, read writing test cases.

A case is a debug config with assertions: the input, the mocks and the spied blocks are the same ones octo invoke --run-debug-config takes.

Top-level keys

flow:     # the flow under test
inputs:   # named messages the cases share
mocks:    # blocks to stand in for, in every case
env:      # environment variables every case runs with
timeout:  # how long any one case may take
cases:    # the scenarios

inputs

Named messages, so the body that means a vip order is written once and referred to by name. A case may also write its input inline.

inputs:
  small:
    data: { amount: 10 }
  a vip order:
    data: { orderId: 42, amount: 250 }
    vars: { x-api-key: dev-key }

cases

Each case is one octo invoke, in its own process, run with the mocks and spies it asks for.

cases:
  - name: a declined card takes the flow error path and answers 502
    input: large
    expect:
      vars: { httpStatus: 502 }
      that: ['body.error.contains("card declined")']

The two merge rules differ on purpose: a case's mocks replace the file's per address, whole, so the case states exactly what that block will do, while a case's env overrides per variable because a variable is a scalar.

expect

What the flow should have done. Omitting expect entirely still asserts something: that the flow completed and did not fail.

A flow either produces a message, drops it, or fails, so dropped and error are exclusive with each other and with the message fields; dolphin rejects a case that asks for two.

body is exact because it is the flow's answer. vars is a subset because variables are scratch space the engine and the blocks add to. Use that: ['vars.size() == 2'] when you do want them exact.

The CEL in that

The same CEL a mock's when and every block in the flow uses, over the same variables: body, vars, eventID, correlationID, now. An expression that evaluates to something other than true or false is rejected rather than read as truthy.

mocks

How one block is stood in for. Keyed by block address, which works unquoted as a YAML key.

At least one of the two is required.

mocks:
  charge-flowlevel.call-charge:
    cases:
      - when: 'body.amount > 100'
        error: card declined
    default:
      body: { id: ch_1, status: captured }

A mock case

Exactly one of body, error and drop per case. when is the only expression a mock carries; the outcome is a literal.

Un-mocking a block for one case

A case can lift the file's mock, so that one case runs the real block. Write the address with a null:

mocks:
  progress.accepted: { default: { body: {} } }   # every case, except…

cases:
  - name: the flow still reaches the emit point
    expect:
      body: { percent: 100 }

  - name: without a live stream the block fails rather than pretending it wrote
    mocks:
      progress.accepted: null                    # …this one, which runs the real block
    expect:
      error: sse-event

It is for the one case whose point is what the real block does, such as asserting that it fails loudly. Without it, the file-level mocks: would have to be repeated on every other case.

Only an address the file actually mocks may be nulled; dolphin refuses the file otherwise, naming the address. The misspelled progres.accepted: null would otherwise read as this case runs the real block while the mock it meant to lift stayed on.

spies

What one watched block should have seen. Keyed by block address.

A block inside a fork branch or a foreach body is crossed once per branch or item, and every crossing is recorded, so the count is the iteration count. count: 0 is an assertion, not an absence: it says the block never ran.

Records are positional: the first entry is the first crossing. Asserting on fewer crossings than happened is fine; naming more than happened is a failure.

spies:
  demo.each-order[body].classify-order:
    count: 2
    records:
      - input: { that: ['vars.order.id == 1'] }
      - input: { that: ['vars.order.id == 2'] }

A crossing

output, dropped and error are exclusive. input is always available, even for a block that failed, because it is captured before the block runs.

input and output each take body (exact), vars (subset) and that (CEL), the same three fields expect uses for a message.

Unknown keys are rejected

dolphin decodes a suite with unknown fields disallowed, and refuses the whole file over one. A misspelled spys: would otherwise watch nothing, assert nothing, and go green. The editor's Testing tab reports the same thing before you run, and disables the form rather than re-serializing a file it cannot fully read.

Generating the schema

To have your editor complete a suite and a validator check it:

dolphin schema --format yaml    # or --format json, the default
dolphin schema --out test.schema.json

See also

• Writing test cases: the same format, explained through worked examples.
• Running tests: dolphin test, its flags, and the exit codes.
• CLI reference: the block-address grammar mocks and spies are keyed by.
• CEL reference: the expression language that and when use.