Testing

A flow that calls a payment API can only be run where that API is. So it gets run once, by hand, on the day it is written — and after that nobody dares touch it.

dolphin is octo's test runner, and it removes the reason not to touch it. A test is a debug config with assertions: the same input, the same mocks, the same spies you would pass to octo invoke by hand, plus what should have happened. The mocks are what make the flow runnable anywhere — the HTTP call, the LLM, the database write never happen — and the assertions are what make running it mean something.

One file, four surfaces

A suite is a YAML file that lives beside the flows it tests: orders.yaml is tested by orders_test.yaml. Nothing about it is private to a tool.

All four give the same verdict, because all four shell out to the same binary. The editor does not reimplement the assertions; it runs dolphin and reads its report. That is what makes a green tab mean a green pipeline.

The shape of a suite

flow: orders                       # the flow every case calls

mocks:                             # stand in for the blocks that call the world
  orders.charge-card:
    cases:
      - when: 'body.amount > 100'
        error: card declined
    default:
      body: { id: ch_1, status: captured }

inputs:                            # named messages the cases share
  small: { data: { amount: 10 } }
  large: { data: { amount: 500 } }

cases:
  - name: a small charge is captured
    input: small
    expect:
      body: { id: ch_1, status: captured }

  - name: a large charge is declined and answers 502
    input: large
    expect:
      vars: { httpStatus: 502 }
      that: ['body.error.contains("card declined")']
    spies:                         # and prove the block was actually reached
      orders.charge-card:
        count: 1

Start here

What dolphin is not

• Not an assertion DSL. expect.that and a mock's when are CEL — the same expression language every block in the flow already uses, over the same variables. There is nothing extra to learn to write an assertion.
• Not an in-process harness. One case is one octo process. The debug seam is a config rewrite, so a service can only serve the case it was built for. That buys full isolation between cases, and it is why a failing case can hand you a runnable command that reproduces it.
• Not a mock of octo. The engine under test is the real one. A mocked block failure is indistinguishable from the real block failing, so an error path behaves in a test exactly as it will in production.