Refactor EventLog to use service and layer composition instead of a session option

Problem

Fold currently has an EventLog Effect service and concrete layers, but the public session API does not compose them as services. Instead, startSession / resumeSession accept a FoldEventLog descriptor as the log option. StartSession.ts converts that value back into a layer internally via eventLogLayerFor.

That creates an unnecessary value-level bridge around an existing Effect service:

const eventLog = yield* eventLogs.open
const log = eventLogSource(Effect.succeed(eventLog))

return yield* startSession({ agent, log })

The current API name is startSession / resumeSession (rather than CreateAgent), but the underlying concern is unchanged: EventLog should be an ambient Effect dependency, and application composition roots should select its implementation with normal Layer composition.

Current implementation

packages/fold-core/src/EventLog/EventLogService.ts defines the EventLog service (append, entries, and subscribe).
packages/fold-core/src/EventLog/EventLogLayerMemory.ts already exposes in-memory EventLog layers.
packages/fold-agent/src/EventLog/JsonlLayer.ts already exposes JSONL EventLog layers.
packages/fold-core/src/Api/EventLogDescriptor.ts adds the FoldEventLog, memoryEventLog, and eventLogSource descriptor indirection.
packages/fold-core/src/Api/StartSession.ts accepts that descriptor through StartSessionOptions.log / ResumeSessionOptions.log, then lowers it back into an EventLog layer.
packages/fold-agent/src/EventLog/JsonlDescriptor.ts builds a JSONL layer, extracts its service, and wraps it in eventLogSource.
examples/alchemy-cloudflare/src/DurableObjectEventLog.ts exposes an open effect that returns an EventLogService; ChatSession.ts then wraps that service in an event-log descriptor before starting or resuming a session.

Proposed direction

Make EventLog a real dependency of session construction and move backend selection to the host's composition root.

The intended custom-backend usage should look like ordinary Effect composition rather than passing a layer or service as an option:

startSession({ agent, cwd }).pipe(
  Effect.provide(layerJsonl(logPath)),
  Effect.scoped,
)

Do not replace log with a Layer-valued option. The dependency should stay in the Effect environment.

Preserve the ergonomic fresh-session default: startSession({ agent }) should still create a fresh, isolated in-memory log when no EventLog service is supplied. resumeSession must adopt an existing supplied log; it must not silently resume from a new empty in-memory log.

For Cloudflare, change the SQLite/Durable Object implementation to provide EventLog directly from a layer. Compose that layer once at the Durable Object boundary and yield the same EventLog service for:

deciding whether the log is empty;
startSession / resumeSession;
the Durable Object's entries() RPC.

This removes the current open -> EventLogService -> eventLogSource -> FoldEventLog -> EventLog layer round trip.

Scope

`fold-core`

Refactor startSession / resumeSession and their internal session graph to consume EventLog from the Effect environment.
Keep the default in-memory path for new sessions.
Remove the internal eventLogLayerFor descriptor lowering.
Remove or deprecate FoldEventLog, memoryEventLog, and eventLogSource, including their root exports.
Update comments and docs that currently state that no public API uses layer composition.

`fold-agent`

Use the existing layerJsonl, layerJsonlWithIds, and layerJsonlNode APIs directly.
Remove or deprecate jsonlEventLog and its descriptor wrapper.
Update SessionLayout, launcher code, examples, and tests to compose the JSONL layer at the runtime/session boundary.

Alchemy Cloudflare example

Refactor DurableObjectEventLog into an EventLog-producing layer instead of a separate service whose open method returns an EventLogService.
Compose the backend with the other Durable Object layers in ChatSession.ts.
Ensure the backend is opened once per object/session and remains alive for the Fold session's scope.
Update the example README and API tests to demonstrate the layer-native integration.

Documentation

Update the root README's JSONL example.
Update the repository's Effect-program-design guidance, which currently explicitly recommends the descriptor seam and says callers should not compose event-log layers.

Behavioral constraints

The refactor must preserve the existing event-log contract:

One log instance represents one Fold session.
Separate default in-memory sessions do not share entries, sequence numbers, IDs, or subscriptions.
resumeSession reuses the existing backend, recovers identity from the existing session_started row, and does not append another start row.
entries remains a finite snapshot/replay; subscribe remains replay plus gap-free live events.
Sequence numbers remain contiguous and start at zero.
JSONL and SQLite persistence happens before publication.
Existing versioned and legacy unversioned-v1 JSONL rows continue to decode.
Event-log resources live for the whole session scope.
Existing error behavior is preserved unless this issue explicitly chooses to expose layer-construction errors instead of treating them as defects.

Layer cardinality needs particular care: reusing a memoized EventLog layer across two session starts can accidentally share one log. The implementation should make the ownership rule explicit and retain the current session-isolation tests.

Acceptance criteria

Custom event logs are supplied through the Effect environment with Layer composition; no EventLog, EventLogService, or Layer is passed as a session option.
startSession({ agent }) still uses a fresh in-memory log by default.
resumeSession requires and reuses an existing event-log backend.
Two default in-memory sessions created in the same Effect program remain isolated.
JSONL-backed CLI/example sessions start and resume through the layer-native API.
The Alchemy Cloudflare example provides its SQLite-backed implementation as EventLog and shares that service across Fold and its RPC methods.
Descriptor APIs are either removed as a documented breaking change or retained only as clearly deprecated compatibility shims with a follow-up removal plan.
README and Effect-program-design documentation show the new composition pattern.
fold-core, fold-agent, and examples/alchemy-cloudflare tests and typechecks pass.

Relevant tests

At minimum, update/retain coverage in:

packages/fold-core/test/Api/StartSession.vi.test.ts
packages/fold-core/test/Api/ResumeSession.vi.test.ts
packages/fold-core/test/Api/SessionIsolation.vi.test.ts
packages/fold-core/test/EventLog/EventLogMemory.vi.test.ts
packages/fold-agent/test/EventLog/EventLogJsonl.vi.test.ts
packages/fold-agent/test/Session/SessionLayout.vi.test.ts
examples/alchemy-cloudflare/test/Api.test.ts

Related layer-composition reference: https://alchemy.run/infrastructure-as-effects/layers/

hmm ok btw we probably need to be careful of effect layer memoization

@humanlayer-channels-app

@humanlayer-channels-app please take a look and let me know what you think - any important open questions to answer here?

I think this is a good direction: the descriptor round trip isn’t adding much policy, and EventLog already has the right service seam.

A few decisions are worth making explicit before implementation:

Optional dependency semantics for startSession. Keeping startSession({ agent }) dependency-free means EventLog is an optional ambient override, not a required dependency. I’d use the supplied service when present and allocate a fresh memory log only when absent. resumeSession should require EventLog in its Effect environment. Is that asymmetry the intended public contract?
Ownership/cardinality of a supplied log. Yes, the memoization concern matters. StartSession.ts already uses a fresh memo map per session to prevent module-level layers from being shared. That protection cannot isolate an EventLog the host has already built and supplied.
- Default memory logs: fresh per start, regardless of the ambient memo map.
- Supplied logs: adopt the exact instance; don’t rebuild or automatically freshen its layer.
- Hosts: supply one backend per session, deliberately sharing it only among that session’s consumers.
What should a second startSession against the same supplied log do? I’d reject a nonempty log rather than append another session start. Layer.fresh alone isn’t a solution for persistent backends: two fresh instances pointing at the same JSONL file still target one log.
Scope and the Durable Object construction boundary. The supplied backend must outlive every use of the returned session handle. The example with Effect.scoped is safe only if the session is also used inside that scope—not returned for later use. For Cloudflare, the code explicitly distinguishes an outer, once-per-class Effect from the returned, once-per-object Effect. We should confirm where Alchemy builds/memoizes the new layer so SQLite initialization happens per object activation, not at class construction, and the same instance reaches emptiness checks, Fold, and RPCs.
Error and compatibility policy. Directly providing layerJsonl naturally exposes its typed construction errors; the current descriptor path converts those to defects. I’d choose typed acquisition errors and document that change. Also choose removal versus deprecation now—my preference is removal if a breaking change is acceptable, rather than maintaining two composition models.

I’d add tests for two default starts under an ambient provided layer, reuse of one supplied log, and backend lifetime across later send/RPC calls. The rest of the issue’s persistence and replay constraints look sufficiently specified.

@humanlayer-channels-app

@humanlayer-channels-app

re: 1 - i'm open to ambient event log but i think the linter might not like optional services you might have to think about that one or we might have to tweak the oxlint config

re: 2 - wait so we have separate session start vs. resume semantics? hmm if that's the cas ei'm open to one of two things - either error with an EventLogIsNonEmptyError or someting like that, or just tread it as a resume - ideally though callers shouldn't ahve to think about this, do we know why they do? is there a good reason / how is the wiring and machinery different here

re: 3 - i don't really undrestand what you're saying here can you take a look and point me at relevant context or explain more clearly?

re: 4 -idk i don't quite follow here but you an check the effect docs and make sure that we're following effect best practices here though

Plan

✅ Check optional-service lint rules and start/resume behavior: Confirmed production serviceOption ban; start and resume share graph but differ in identity/log initialization and reconciliation.
✅ Verify Alchemy lifecycle and Effect layer/error semantics: Checked pinned Effect 4.0.0 and Alchemy beta.80 sources. Alchemy evaluates outer init during planning with mock storage and runs both effects per activation at runtime; provide/scoped lifetimes require explicit object-scope ownership.
✅ Explain findings and recommend API decisions: Prepared recommendations for a narrow lint exception, automatic open semantics with identity safeguards, per-activation backend ownership, and typed acquisition errors. No code changed.

I checked the code and the source/API docs shipped with our pinned Effect 4.0.0 and Alchemy beta.80. One correction to my earlier explanation below.

1. Optional service: you’re right about the linter

.oxlintrc.jsonc enables automation/no-service-option, and the rule explicitly bans production Effect.serviceOption.

Effect itself supports it: serviceOption(EventLog) returns an optional service without adding EventLog to the required environment.

Recommendation: keep the internal session graph strictly dependent on EventLog, with one optional lookup at the public fresh-session boundary. Add a narrowly documented lint exception there—not a global relaxation, and not a workaround using manual context inspection. That boundary would implement the intentional policy “use the supplied log, otherwise allocate an isolated memory log.”

2. Start versus resume: different initialization, same wiring

Both functions call the same assembleSessionGraph. The difference is what happens afterward:

Fresh start	Resume
Mints session/root-agent identity	Recovers identity from `session_started`
Writes `session_started` and initial root-agent rows	Adopts identity without repeating those rows
Initializes model/prompt/tool configuration	Projects existing state and writes a configuration transition only if needed

Relevant code: public start/resume and internal start/adopt.

So there is a good reason for two internal paths, but no architectural requirement that callers choose between them. Explicit start/resume lets a caller assert “this must be new” or “this must already exist.”

I’m open to an automatic openSession operation: empty log → initialize; existing valid session → adopt/reconcile. We could retain strict start/resume for callers wanting those assertions, or intentionally make startSession mean open.

Two safeguards matter:

Nonempty but invalid history must fail, not become a fresh session.
Existing identity must win. If the caller supplies a conflicting sessionId, reject it; cwd/initial metadata should not silently rewrite history.

Auto-open also doesn’t make two simultaneously active handles over one log safe. Backend sharing and session-handle ownership still need an explicit rule.

Cloudflare will still have some host-side branching: its fresh path clones repositories, whereas its resume path discovers the existing workspace. That’s separate from Fold’s initialization machinery.

3. Cloudflare: the concrete concern is when storage is accessed and how long the instance survives

The current sequence is:

Construct DurableObjectEventLog, but defer SQLite work.
Run eventLogs.open inside the returned Effect.
Capture that instance for session construction and entries() RPC.
Keep the Fold session in an explicit object-lifetime scope.

See ChatSession.ts.

Correction: I relied on the comment saying the outer Effect runs once per class. Alchemy’s pinned implementation shows:

The outer Effect also runs during planning with mock storage.
At runtime, its bridge runs the outer Effect and then the returned Effect per object activation.

Sources: Alchemy construction/planning and runtime bridge.

Therefore, simply replacing the current outer DurableObjectEventLog.layer with a layer that immediately queries SQLite would access mock storage during planning.

Recommendation: build the real EventLog layer inside the returned, runtime-only Effect, once per activation, with its resources owned by the object/session scope. Capture that same built context for later session creation and RPCs. Don’t rebuild it per RPC.

4. Errors: let normal Effect composition preserve them

The Effect 4 API confirms that Effect.provide adds the layer’s construction errors to the resulting Effect’s error channel.

Thus corrupt JSONL, unsupported versions, or unavailable storage should remain typed acquisition failures. Today the session graph’s Effect.orDie converts backend-construction failures into defects.

Recommendation: let layer errors propagate normally; let the CLI/Cloudflare boundary decide how to report them or terminate. Don’t broaden this refactor into changing every existing session-operation error.

Also, Effect.provide(layer) owns a layer-build scope, and Effect.scoped closes its scope when the enclosed workflow returns. Examples should therefore enclose both session construction and use, not return a supposedly live handle after closing its resources.

No code changed. The checkout may be behind because the initial pull was rate-limited.

@humanlayer-channels-app

@humanlayer-channels-app

Yeah, for the first thing, I'm fine with adding an exception. For number two, instead of like open session or start session versus resume, or like start versus adopt, could you maybe, what if we just had a session.run and it just started like adopts an empty event log or resumes an existing session? I think that the safeguards, I agree with those. The third one, yeah, accessing, like querying a mock SQLite during planning is fine. I don't care if Alchemy uses a mock storage API during planning.

Technically speaking, corrupt JSONL, unsupported versions, or unavailable storage, those are intended to be treated as defects with Effect.orDie, since there's no world in which they're recoverable. I don't understand what you're saying about Effect.provide and building a layer.

A layer build scope. Let's clarify this. Also, let's try and be more concise, please.

Yes—Session.run({ agent, ... }) could be the single entry point: initialize an empty log, or recover identity and resume an existing session. Same returned handle, same internal graph. Keep the safeguards we discussed and the fresh-memory default. No need to expose start/adopt as separate public concepts.

Agreed on the optional-service lint exception.

Errors: understood. Keep Effect.orDie for corrupt/unsupported/unavailable storage. I was recommending a different error policy, not identifying an Effect requirement. With direct layer composition, put that policy on the backend layer, e.g. layerJsonl(path).pipe(Layer.orDie).

Scope clarification: Effect.provide(layer) acquires the layer’s resources for the duration of the Effect it wraps. If that Effect only constructs and returns a session handle, the layer’s resource ownership ends before subsequent calls on that handle.

The safe shape for a bounded workflow is:

Effect.gen(function* () {
  const session = yield* Session.run({ agent })
  yield* session.send("hello")
}).pipe(
  Effect.provide(layerJsonl(path).pipe(Layer.orDie)),
  Effect.scoped,
)

For a Durable Object, construction returns RPC methods that run later. Build the backend once into the explicit object-lifetime scope already used in ChatSession.ts, then make that context available to both session construction and RPCs. That’s the lifetime distinction I meant—not an objection to layers.

Planning: querying a working SQLite mock would be fine. The pinned Alchemy planning path actually supplies { storage: {} }, so raw.storage.sql.exec is absent. My concern was that concrete call failing, not a rule against storage access during planning. We just need to avoid that call there or provide a functioning mock.

@humanlayer-channels-app

@humanlayer-channels-app i think i prefer Session.open({agent}) and then session.send to session.run and then session.send

generally LGTM as long as you have a clean handle on how to do this, can you get this done and create a draft pr?

Plan

✅ Inspect current branch and affected session/backend call sites
✅ Implement Session.open and layer-native event log composition: Migrated public callers, removed descriptor APIs, and built SQLite EventLog once in the Durable Object runtime scope.
✅ Update documentation and run focused tests/typechecks: 635 tests passed across the three packages (3 live tests skipped), full workspace typecheck passed, lint passed, and migration/ownership guidance updated.
❌ Commit, push, and open a draft pull request: Committed and pushed bf88869; opened PR Draft: Session.open with layer-native EventLog composition #61 with draft-review title/body. The supplied PR tool cannot set or convert GitHub's native draft flag; confirmed draft=false. A maintainer must use Convert to draft.

Implemented and pushed: PR #61.

Session.open({ agent }) initializes or resumes the ambient log.
Fresh memory logs remain isolated; JSONL and Cloudflare use layer composition.
Identity/history safeguards and storage-as-defect behavior are preserved.
Added lifetime, resume, and subscription-isolation tests; updated migration docs.

Verified: 635 tests passed, 3 live tests skipped; workspace typechecks and lint passed.

Draft-status limitation: the PR tool cannot set GitHub’s native draft flag. I marked the title/body as draft, but please use Convert to draft on the PR.

Navigation Menu

Repository navigation

Refactor EventLog to use service and layer composition instead of a session option #60

Problem

Current implementation

Proposed direction

Scope

`fold-core`

`fold-agent`

Alchemy Cloudflare example

Documentation

Behavioral constraints

Acceptance criteria

Relevant tests

1. Optional service: you’re right about the linter

2. Start versus resume: different initialization, same wiring

3. Cloudflare: the concrete concern is when storage is accessed and how long the instance survives

4. Errors: let normal Effect composition preserve them

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Refactor EventLog to use service and layer composition instead of a session option #60

Description

Problem

Current implementation

Proposed direction

Scope

fold-core

fold-agent

Alchemy Cloudflare example

Documentation

Behavioral constraints

Acceptance criteria

Relevant tests

Activity

K-Mistele commented on Oct 9, 2026

K-Mistele commented on Oct 9, 2026

humanlayer-channels-app commented on Oct 9, 2026

K-Mistele commented on Oct 9, 2026

humanlayer-channels-app commented on Oct 9, 2026

humanlayer-channels-app commented on Oct 9, 2026

1. Optional service: you’re right about the linter

2. Start versus resume: different initialization, same wiring

3. Cloudflare: the concrete concern is when storage is accessed and how long the instance survives

4. Errors: let normal Effect composition preserve them

K-Mistele commented on Oct 9, 2026

humanlayer-channels-app commented on Oct 9, 2026

K-Mistele commented on Oct 9, 2026

humanlayer-channels-app commented on Oct 9, 2026

humanlayer-channels-app commented on Oct 9, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions

`fold-core`

`fold-agent`