Object Storage and Caching
Persist state in the object store and memoize with cache scopes.
Flows are stateless by default. This guide adds state: object-read/object-write
for values that persist between runs, and cache-scope to memoize expensive work
such as an LLM call.
Read and write objects
object-write persists a value under a key; object-read loads it back into a
variable.
samples/object-store.yaml
reads a key before and after writing it, showing a miss and a hit:
process:
# Cold read: the key is absent, so the default is folded in and
# profileExists is false.
- type: object-read
name: load-before
settings:
key: '"profile:" + body.id'
as: profile
default: '{"plan": "free"}'
existsVar: profileExists
- type: object-write
name: save-profile
settings:
key: '"profile:" + body.id'
value: '{"plan": body.plan}'
# Warm read: the key now exists, so the stored value wins over the
# default and profileExists is true.
- type: object-read
name: load-after
settings:
key: '"profile:" + body.id'
as: profile
default: '{"plan": "free"}'
existsVar: profileExistsRun it:
octo invoke --config samples/object-store.yaml --flow profile \
--data '{"id":"A-1","plan":"pro"}'
# -> logs "before write: A-1 exists=false plan=free" (miss -> default)
# then "after write: A-1 exists=true plan=pro" (hit)key is a CEL expression on both blocks, so per-entity keys like
'"profile:" + body.id' work. as names the variable the value lands in
(vars.profile), leaving the body untouched. default supplies a fallback
expression on a miss, and existsVar writes a boolean that tells a genuine hit
from a served default.
Memoize with cache-scope
cache-scope wraps a sub-flow and memoizes the body it produces under a CEL
key with a TTL; on a hit, the wrapped blocks never run.
samples/runtime-services.yaml
generates a "report" every 5 seconds, but the expensive part runs once per
minute:
- type: cache-scope
name: cached-report
key: '"report"'
ttl: 1m
body:
process:
- type: log
name: compute
settings:
logger: out
message: '"cache MISS — generating report"'
- type: set-payload
settings:
value: '{"report": "generated", "event": eventID}'key, ttl, and body sit directly on the block, and body holds a nested
process list. Evict an entry with invalidate-cache and the same key
expression:
- name: reset
process:
- type: invalidate-cache
name: bust-report
settings:
key: '"report"'octo run --config samples/runtime-services.yaml
# "cache MISS — generating report" appears once, then ticks serve the cache.
# In another terminal, bust it and the next tick recomputes:
octo invoke --config samples/runtime-services.yaml --flow resetOnly the message body is cached. Variables the wrapped flow sets are not restored on a hit, so keep anything a later block needs inside the body.
ttl is a Go duration (30s, 5m, 1h). Omit it for the default of 60
seconds; "0" means the entry never expires (until invalidated). The evaluated
key is hashed, so long or exotic keys are fine. Storing is best-effort: if
another worker cached first, your result is still correct, just uncached.
Caching an LLM call
samples/ai-quote-cache.yaml
puts an ai-mapping call to Gemini behind a cache scope on an HTTP endpoint:
process:
# Memoize the slow/paid Gemini call under a constant key. The first
# request is a MISS; requests within the 5m TTL replay the cached quote
# with no model call.
- type: cache-scope
name: cached-quote
key: '"seneca:quote"'
ttl: 5m
body:
process:
- type: log
name: cache-miss
settings:
logger: log
message: '"cache MISS — calling Gemini"'
- type: ai-mapping
name: seneca
settings:
connector: gemini
prompt: >
Ignore the input entirely. Produce a single, genuine quotation
from the Stoic philosopher Seneca (Lucius Annaeus Seneca the
Younger). Return only the mapped object.
outputExample: |
{ "quote": "We suffer more often in imagination than in reality.", "author": "Seneca" }export GEMINI_API_KEY=...
octo run --config samples/ai-quote-cache.yaml
curl -s localhost:8080/quote # first call: MISS, ~seconds (model call)
curl -s localhost:8080/quote # within 5m: HIT, sub-millisecond, same quotePut the paid call inside the scope, key it by whatever makes responses reusable
(a constant, vars.query.topic, a user ID), and do per-request derivation after
the scope so it runs on hits too, as the sample's multi-transform does. See
An AI Web App End to End for a full application.
Where state lives
Both ride on the runtime's KV service, so the same YAML runs everywhere.
Standalone, octo run serializes objects to --storage-dir (default
./octo-store), so they survive a restart, while octo invoke keeps them in
memory so a probe is reproducible. On the platform the store is durable, scoped
to your deployment and shared by all replicas. See
KV and Storage for scoping, quotas, and inspection.
Either way a namespace picks a
durability tier. Cache entries are
always volatile (losing one costs a recompute), while objects default to
persistent and take a volatile setting when you want the cheaper tier for a
value you can afford to lose.