Testing in the Editor
Write a flow's dolphin tests in the editor, run the battery, and commit the file CI will run.
Debugging a flow answers one question once. The mocks you placed and the input you typed live in the editor's own scratch file, and the answer lives in your head — so the next person to touch the flow starts over.
The Testing tab is where that becomes a test. It writes a real
<flow>_test.yaml beside your flows: the same input, the same mocks, plus what
should have happened. That file is committed, dolphin test runs it from a
terminal, and CI runs it too. The tab is a front-end for
that file — not a private feature of the editor.
The tab appears only when the host stores suites for you (the standalone editor and
the platform both do). Running them needs a second binary, dolphin, alongside
octo — see DOLPHIN_BIN_PATH. Without it
you can still author: only the Run tests button goes dead, and it says why.
The tab
Pick Testing in the view switcher. The left rail lists your flows; a flow with no suite yet offers to start one, scaffolded with comments explaining each section.
The middle column is the suite's cases; the right pane is the case you selected.
The rail tells you where you stand before you click anything: a flow with a suite shows how many cases it has, and a flow without one shows a dimmed flask. Pick the one with no suite and it offers to write it:
What it writes is not an empty file. The scaffold carries a comment above every section a suite can hold, so the first thing you read is what the file is for:
A case, field by field
Every field is on one pane, and the two that are easiest to confuse label themselves — Body is exact, Variables is a subset.
Input is what the flow is called with — a body and the variables a source would
have set. It is either written inline or pointed at one of the file's shared
inputs:, which is what you want once three cases send the same order.
Mocks stand in for blocks, so the flow can run anywhere. Addresses come from a picker over the blocks in the flow, never typed by hand. A mock listed at the file level applies to every case; a case can override one — and the override replaces the file's mock for that address whole, rather than merging with it, so a case reads as a complete statement of the world it runs in.
A spec is a list of When… cases over the message the block received, and an Otherwise… default. Without the default, a message matching no case fails the block — a mock replaces the block, so there is no real one left to fall through to, and a mocked call that quietly reached the network would be the exact bug mocking exists to prevent.
Expect is a three-way switch, because a flow does exactly one of three things: produce a message, drop it, or fail. It is not three checkboxes — dolphin rejects a case that asks for two.
expect.body is an exact deep-equal of the whole body. expect.vars is a
subset — only the variables you name are checked. The asymmetry is deliberate:
the body is the flow's answer and a test earns its keep by failing when a field
appears in it, while variables are scratch space the engine adds to. The form labels
both inline so you never have to remember which is which.
That holds CEL expressions over the message, each of which must be true — the
tool for a body carrying a timestamp or a generated id, which an exact body could
never match. Add a row per expression; the placeholder shows the shape
(body.total > 0). It is the same CEL every block in the flow
uses, over the same variables — with one exception worth knowing before you write
one: env is not bound, because dolphin never loads your config. Assert on what
the flow put in the body or the variables instead.
An expression that returns something other than a boolean is reported by dolphin as
the mistake it is rather than counted as a false, because it fails the case either
way. To try an expression against a real message before you assert on it, run the
flow and use the expression tester on the canvas.
Spies assert what a watched block saw: how many messages crossed it, and what
each crossing carried. count: 0 is a real assertion — "this block never ran" — and
is kept distinct from asserting nothing at all.
Suite settings holds the file-level inputs:, mocks:, env: and timeout:.
The env: block is not a nicety: a config's environment is resolved when it loads,
before any block runs, so a flow whose connector reads ${SOME_KEY} cannot be built
without one — and mocking the block that uses it does not help. These values are fake
by construction, which is what makes them safe to commit.
Form or YAML
Each suite has a Form / YAML toggle. The YAML view saves the file byte for byte; the form re-renders it from the model.
Editing in the form drops the file's comments. A suite is a file people read, and the scaffold ships full of explanation. The tab warns when the open suite has comments and offers to switch you to YAML; nothing is lost until you actually edit through the form.
When dolphin would refuse the file
Anything that would stop dolphin loading the suite is reported above the editor, and Run tests goes dead until it is fixed — there is no point spending a run on a file the runner will reject:
Issues are reported per case, so a file with six cases tells you which one is wrong rather than that something, somewhere, is. The rail carries the same count, so a broken suite is visible from the flow list without opening it.
One kind of problem goes further and disables the form as well: a file holding
something the editor cannot represent, an unknown key, most often a typo.
Re-serializing it would delete what it could not read, so the YAML view is forced and
the problem reported instead. dolphin refuses the whole file over that key anyway,
which is the point — a misspelled spys: watches nothing, asserts nothing, and would
otherwise go green.
Running
Run tests runs the open flow's suite and reports on the console's Tests tab, beside the logs and the output — every kind of run reports in the same place. The tab carries a badge counting the cases that failed or errored, so a red verdict is visible without opening it.
Each case shows its verdict; a failing one shows what dolphin printed and the message the flow actually produced, side by side. The tally distinguishes two things worth keeping apart:
- failed — the flow ran and did not do what the case says. The flow is wrong.
- errored — the case never ran: an address that resolves to nothing, an input the file does not declare. The test is wrong.
A failing case opens by itself, because a verdict you have to click on is a verdict you will skim past:
The config under test is rendered from the document in front of you, not from what was last saved. The tab exists to tell you whether the flow you are looking at does what you said.
The dev .env is deliberately not injected into a test run, unlike a debug run.
Injecting a resource the config never declared would make the tab and dolphin test
disagree — and would let a "test" quietly authenticate with your real credentials. A
suite says what environment it needs in its own env:.
Running every suite
The toolbar's Run tests runs the suite you have open. The one in the header runs them all.
That is what the RUN control becomes on the Testing tab. Running the whole set is the ordinary thing to want there; starting the integration is not, because nothing on that tab is about a live runner — so a button that started one would be a button nobody meant to press.
Hovering it names the suites it is about to run, and any it will leave out, so you know the scope of the press before you make it.
The results are grouped one block per suite, each with its own tally, so a red suite is findable in a run of a dozen:
What gets held back, and why
Not every stored suite is sent. dolphin loads and validates every file it was named before it runs any case, so a single file it refuses aborts the whole run and reports nothing — including for the five suites that were fine. A run-everything that forwarded a suite the editor already knows is broken would turn one bad file into zero results, with an error naming none of the good ones.
Three reasons a suite is held back:
| Reason | What it means |
|---|---|
there is no flow by that name in this document | A suite left behind by a rename or a deletion. The Testing tab lists flows, so the file is invisible there — but it is still on disk, and dolphin test and CI still run it. This is usually the first you hear of it. |
has no cases yet | The ordinary starting state: a scaffold nobody has filled in. |
dolphin would refuse it: … | The file has a problem the suite issues panel is already reporting. |
What is left out is reported, not dropped — named in the tooltip before the run and again above the results after it. A green tally that quietly covered three suites which never ran is worse than a red one.
Held back here is not the same as excluded from CI. dolphin test takes the files you
point it at, so an orphaned suite that this button skips still runs — and still fails —
in a terminal. The skip list is the place that tells you it exists; see
running tests.
The two bridges
The Testing tab and the canvas are not separate worlds.
Scenarios. A flow's ▶ menu lists its test cases under Scenarios. Pick one and the flow runs on the canvas with that case's input and mocks — the setup you would otherwise rebuild by hand every time you came back to it. The run does not check the case's assertions; that is what Run tests is for, and the menu says so.
Save as test case. Every result on the console's Output tab carries Save as
test case, which turns that run into a case in the flow's suite: the input it used,
the mocks that were active, and what came back. It previews the exact YAML before
writing, and warns that a body carrying a generated id will fail on its next run —
pointing you at that: instead.
A run stopped at a breakpoint cannot be promoted: it reports the message at that block, not the flow's result, so the case would assert something no flow produces. The button says so rather than disappearing.
Where the file lands
One suite per flow, named <flow>_test.yaml. In the standalone editor it sits in
your flows directory, next to the flows; on the platform it is stored with the
integration and never pulled by a deployed runtime.
dolphin's own convention names a suite after the config file it accompanies, and a
suite declares exactly one flow — so a config holding three flows cannot have three
companions. We name suites after the flow, which is what makes one suite per flow
possible. The two coincide in the common case, but a config orders.yaml whose flow is
called checkout produces checkout_test.yaml, and dolphin will not pair them up.
Naming the suite alone does not settle it either: with no companion config, dolphin falls back to the containing directory, which merges every config in it. In a workspace holding several, that means the suite runs against all of them at once — and fails on the first environment variable an unrelated flow declares. Name the config:
dolphin test .octo-flows/checkout_test.yaml --config .octo-flows/orders.yamlThe Testing tab has no such ambiguity: it stages the one config the suite is for.
See also
- Your first test — the same file, written from scratch in a terminal, in about ten minutes.
- Writing test cases — the suite format in full.
- Test file reference — every key, field by field.
- Running tests —
dolphin test, exit codes, and CI. - Debugging flows — the mocks, spies and breakpoints a test is built out of.
- Testing with an agent — the same suites, written and run over MCP.