feat(codegen/go): PoC for opt-in iter.Seq2 companions on :many queries

[Tilzen]

Why

Revives the long-standing request for streaming/lazy :many query results (#720). Today, sqlc always materializes :many into []T, which is fine for small result sets but costly for exports, sync jobs, and backfills.

Go 1.23+ shipped native iterators (iter.Seq2). Kyle noted that this unblocks native iterator generation; PR #3631 explored an implementation but closed without merge after API design remained open.

This PR is not a merge-ready feature. It is a proof of concept (PoC) posted for design feedback before investing in a production implementation. See the discussion proposal in #720 and related #4464.

PoC reference (fork): https://github.com/Tilzen/sqlc/tree/feat/emit-iterators-poc
PRD in PoC branch: docs/emit-iterators-prd.md

What Changed (PoC scope)

Opt-in config (zero breaking changes to default :many → []T):

gen:
  go:
    emit_iterators: true
    iterator_scope: global         # or explicit_only
    iterator_method_prefix: "Iter"
    iterator_style: seq2           # MVP: seq2 only
    iterator_start: lazy

Generates a lazy companion alongside existing slice methods:

func (q *Queries) ListAuthors(ctx context.Context) ([]Author, error) { /* unchanged */ }

func (q *Queries) IterAuthors(ctx context.Context) iter.Seq2[Author, error] {
    return func(yield func(Author, error) bool) {
        rows, err := q.db.QueryContext(ctx, listAuthors)
        // defer rows.Close(), yield rows one at a time
    }
}

Also supports query annotations for explicit_only scope:

• -- name: StreamAuthors :stream
• -- name: ListAuthors :many:stream

PoC includes: stdlib + pgx/v5 templates, metadata parsing, config wiring, PRD doc.

Explicitly out of PoC scope: callback/rows styles, eager mode, emit_interface iter methods, Python/Kotlin generators, full driver matrix.

Open Design Questions (feedback welcome)

1. Method prefix: Iter vs Stream?
2. Default lazy start — agree?
3. Ship both emit_iterators: global and :stream annotations?
4. Should :stream remain a separate query kind or only :many:stream?
5. Min Go version bump when enabled?

Test Plan

PoC test coverage (all passing on the fork branch):

• Unit: iterator naming, scope logic, stream annotation parsing, opts validation
• Codegen matrix: global, explicit_only (:stream, :many:stream), emit_iterators: false regression, parameterized :many, pgx/v5
• Golden replay: TestReplay/emit_iterators* fixtures
• Runtime (SQLite): iter/slice parity, lazy start (error only on first range), early break + connection reuse, parameterized queries

go test ./internal/codegen/golang/... ./internal/metadata/... \
  ./internal/endtoend/ -run 'EmitIterators|TestReplay/emit_iterators|ValidateOptsEmitIterators|Iterator'

cd internal/endtoend/testdata
go test ./emit_iterators/... ./emit_iterators_params/... -v

References

• Proposal comment: #720#issuecomment-4702731093
• Related: #4464, #4108
• Prior art (closed): #3631
• PoC fork branch: https://github.com/Tilzen/sqlc/tree/feat/emit-iterators-poc

Notes

• This is a PoC for discussion, not a request to merge as-is. Happy to close this PR and iterate on design in Ability to return an iterator on a "many" query #720 if that is preferred.
• If maintainers sign off on the config surface + default API (seq2 + lazy + Iter prefix), I can refine the PoC toward a production-ready PR in follow-up commits.

[benjaco]

Thanks for working on this. One limitation I think is worth considering is allocation control. The iterator avoids retaining the full result set, but each row is still scanned into a fresh result value. Dynamic fields such as strings, byte slices, or driver-specific values may require additional allocations, which are only reclaimed when the GC runs. The iterator therefore removes the growing []T, but does not by itself guarantee a constant memory footprint or eliminate allocation and GC pressure.

It would be valuable for the streaming API to give callers some way to control reuse of the scan destination. A synchronous consumer could reuse a single destination, while a consumer with multiple results in flight could obtain destinations from a sync.Pool. The important part is that ownership and lifetime remain explicit, so a destination is not reused while the caller may still retain it.

If that can be supported cleanly by the iterator API, it may also cover the allocation-control use case discussed in #4108, without requiring separate low-level query and scan helpers. If it cannot be supported safely within the iterator abstraction, then the lower-level approach from #4108 would still be useful alongside the iterator.

I also noticed that :many:stream is implemented by adding a dedicated Stream boolean and threading it through the parser and metadata system. Since stream is effectively a modifier on :many, would it make sense to represent modifiers generically, for example as a map or set, instead of adding a new field for each one? That would allow future modifiers to reuse the existing :many path without requiring another hard-coded command variant and boolean throughout the system.