Raw Content and Streaming
Serve and accept non-JSON payloads with explicit content types, and why Octo materializes every message instead of streaming.
Octo is JSON-first: every message body is decoded JSON, and an HTTP flow answers
with application/json unless you say otherwise. Raw content is the opt-out
that lets a flow carry and serve a typed, non-JSON payload such as HTML, XML,
CSV, Markdown, or a urlencoded form.
The raw-content model
A message is in one of two modes. By default it is JSON: body holds decoded
JSON (numbers are floats, objects are maps, arrays are lists). In
raw-content mode, body instead takes the fixed shape:
{ "contentType": "text/html; charset=utf-8", "rawData": "<h1>Hi</h1>" }rawData is the payload as a UTF-8 string and contentType is the MIME type
to serve it with. A raw-aware sink (today, the HTTP source's response writer)
writes rawData verbatim with that Content-Type instead of JSON-encoding the
body.
A multipart/form-data payload carries a third key, parts, holding its decoded
parts by name alongside contentType and rawData, which stay exactly what
was received. See Multipart bodies.
Raw mode is a flag on the message, not a shape you can fake. A plain object
with contentType and rawData keys from an ordinary set-payload (without
rawBody: true) stays in JSON mode and is served as application/json. Use
one of the producers below.
Producing raw content
Six blocks and connectors put a message into raw mode:
| Producer | How |
|---|---|
template-resource | Set rawBody: true and a contentType; the rendered template becomes the raw body. target must be empty. |
set-payload | Set rawBody: true and a contentType; the value must evaluate to a string, which becomes rawData. |
rest | A non-JSON HTTP-client response folds to raw content automatically, carrying the upstream Content-Type. |
notion-page-to-markdown | Emits the page as text/markdown raw content. |
file-read | Reads a file as raw content, typed from its extension. Leave resultVar empty. |
HTTP source (rawBody: true) | Inbound: the request enters the flow as raw content instead of being parsed as JSON (see below). |
From a template
Best for real markup. The template file lives under resources.templates and
can embed {{ CEL }} placeholders:
- type: template-resource
settings:
id: page # a declared template alias
rawBody: true
contentType: text/html; charset=utf-8Inline, from a string
Best for a few lines of HTML or a small typed blob. value must produce a
string: single-quoted YAML wrapping a double-quoted CEL string.
- type: set-payload
settings:
rawBody: true
contentType: text/html; charset=utf-8
value: '"<!doctype html><h1>Oops</h1><p>Try again shortly.</p>"'Both set the response Content-Type from contentType; pair either with
set-variable on httpStatus to control the status code.
Accepting raw requests
Set rawBody: true on the HTTP source and the request body enters the flow
as {contentType, rawData}, using the request's own Content-Type, instead of
being parsed as JSON. Forms, XML, and signed webhook payloads pass through
unchanged:
source:
connector: api
type: http
settings:
path: /submit
rawBody: true
process:
- type: set-payload
settings:
value: 'fromFormData(body.rawData)' # parse the raw string into JSONA multipart/form-data request needs no setting: it is decoded whatever
rawBody says, its parts are reachable at body.parts, and rawData keeps
every byte. See Multipart bodies.
To keep parsing JSON but also capture the exact bytes (for an HMAC signature,
say), use rawBodyVar instead of rawBody: it stores the untouched request
string in a variable while body stays JSON. This is how
slack-verify-request and
notion-verify-request check signatures.
rawData is a UTF-8 string, not arbitrary bytes, so raw content is for text
payloads: HTML, XML, CSV, Markdown, JSON-as-text, form encodings. As a string
it survives cloning and cross-node hops, so it works the same in a fork
branch or across a queue. A multipart file part is base64 because JSON-encoding
the message on a queue hop or into a trace would replace invalid UTF-8 with
U+FFFD; the file connector has
encoding: base64 for the same reason.
Streaming
Messages do not stream. The HTTP source reads the entire request body into
memory (capped by maxBodyBytes, default 1 MiB, returning 413 when exceeded)
before the flow starts, and an ordinary route writes the complete response after
the flow finishes. There is no chunked transfer of a message body, and an
ai-mapping block returns its complete result. Orchestration needs the whole
payload: CEL expressions read across the entire body, validate checks it as a
unit, multi-transform and enrich reshape it, ai-mapping validates against
a schema, routers branch on its contents, and cache-scope keys on the finished
result. Raw content covers what you serve and with which type, always as a
complete payload.
Streaming to the caller is a separate feature beside the message. Setting
sse.enabled on an http source keeps the connection open as a
server-sent event stream, and
sse-event blocks push frames to it from anywhere in the run. An ai-agent
with stream: true feeds the model's output to its events path as it is
produced. The flow's own message is still finished as one unit.
To move a very large object, keep it out of the message: write it to the object store or external storage and pass a reference through the flow.