Repository navigation
Introspection: the connection, six transports, and zsql introspect - #31
Open
ajoabraham wants to merge 65 commits into
Open
ajoabraham wants to merge 65 commits into
ajoabraham wants to merge 65 commits into
Conversation
…che Ossie docs/execution-engine.md surveys what could run the planner's SQL on a remote warehouse, measured against a static musl binary: the Rust drivers per dialect, ADBC and the other abstraction layers, how peer products do it, the OAuth landscape, and clean-build costs of the candidate dependency sets. A second pass widens it to a data plane with local blends and cubes, is attacked by a fresh reviewer, and lands on numeric goldens first, the customer's own federator for blends, a Python client as the Python equivalent, and no hosted data plane without customer pull. Paused in favour of the second document. docs/ossie.md compares 0sql's model with Apache Ossie (incubating, spec 0.1.1; no v1.0 and no Apache release yet). 0sql is a superset on planning semantics and a subset on interchange metadata. A four-phase plan follows: fill 0sql's own gaps first, then import and export with a loss report, then Ossie as a direct model source and a converter upstream. Composite keys and table-less metrics were considered and rejected; agent-facing context is marked unsettled; tables defined by a query get a design note recommending a separate `sql` key. Nothing in either document is implemented. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> (cherry picked from commit ba7b6db)
docs/exec.md takes the brief (the same request sql takes, run against the datasource, rows back) and writes down what it requires: a connection registry separate from the deployed model, execution as its own verb, a job rather than a response, four layers of bound on every execution, Arrow inside and JSON at the edge, three distinct caches with a security rule per credential mode, DuckDB for merges, egress policy, and numeric goldens first. The survey it rests on, docs/execution-engine.md, is cherry-picked onto this branch so the references resolve. It also records a bug found while investigating: LIMIT is appended in three places with no dialect indirection, and SQL Server has no LIMIT. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The branch narrows to the feature wanted now. docs/introspection.md plans reading a warehouse catalogue, generating and updating model YAML from it, and writing samples/values.jsonl.gz, with the execution plane left to docs/exec.md as a survey rather than a plan. The principle that 0sql never connects is struck, and replaced with a narrower one: it connects to read, from the machine you run it on, and the hosted service holds no warehouse credentials. Two things already in the tree carry most of this. datasources.yml already documents host, port, database, username and schema as keys "for the tools that run the SQL on your side"; secrets already live in .zsql and ZSQL_DS_<KEY>_PASSWORD and are already stripped from a deploy. And samples/values.jsonl.gz already has the shape, the consumers and the rules; what is missing is the thing that writes it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QStyZVXmJhAboYZBPGMoMw
The plan takes the model changes: disable_listing becomes disable_sampling, value_list_size becomes sample_size (default 1000), the values: sample|full|none key goes away, and a dimension may carry sample_sql for a query the author writes. Completeness is inferred from the recorded sample_size and the column's total distinct count rather than declared. Sampling is on by default for string alone, and for an absent data_type it is off. Ranges and format inference are out. The generated query goes through the planner's own SQL layer so the expression, the quoting and the dialect are not a format!, applies the table's partition predicate, counts with COUNT(*) and orders by it descending. Two mechanical consequences the rename carries: the golden universes hold the old key names and are never edited, so serde aliases are mandatory; and pruner.rs names value_list_size as a literal. The dialect-rendered row limit comes first, because sampling is the first feature that runs generated SQL and SQL Server has no LIMIT. catalogue.json is committed; the sample file stays the author's choice. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QStyZVXmJhAboYZBPGMoMw
A subagent review of the plan found four claims that were false and several omissions. The plan now says what the code says. False, and corrected: sql/expr.rs is not an expression renderer but three string rewrites, and the real renderer is private behind a Ctx holding a planned query graph, so sampling can reuse qualify and quote and nothing more, which puts calculated, lookup and array dimensions out of scope and gives sample_sql its real justification. The planner never emits a partition predicate, and Partition.filter_value is an authoring default of a historical window, so rendering it would silently bias every sample on a partitioned table; such a table now needs an explicit window or it is skipped. An unknown YAML key is a warning, not a refusal, so dropping the old spellings would lose the author's setting quietly, which is worse. And no golden covers a limit, so the scoreboard was a vacuous gate for that commit. Added: FORMAT must go to 2, since the snapshot's own comment names a renamed or removed field as the reason to raise it and a rollback otherwise reads a snapshot it cannot parse. The record is a tested public contract through parity_tests, so mode leaves and range stays unwritten and only sample_size is added, distinct being the total already. sample_size needs a ceiling, because an oversized file is a warning that costs every dimension its values. hidden joins the exclusions. sample_sql is not stripped from the archive but is never executed from one. COUNT(*) counts the dimension table's own rows. The datasources preamble's promise that those keys are ignored becomes false. check must fold identifiers the way each dialect folds. Phase A splits in two so the limit fix and the format bump land before anything writes a snapshot or runs a statement. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QStyZVXmJhAboYZBPGMoMw
Three places appended "LIMIT n" to a finished statement with no dialect indirection, and dialects/sqlserver.yml has no limit template because SQL Server has none: a spec carrying a limit against a sqlserver datasource planned to a statement that would not parse. Nothing caught it, because the goldens compare plan.sql and nothing ran it. base.yml gains limit_style and limit_clause; sqlserver overrides them with top and TOP (@n), which needs no ORDER BY where OFFSET/FETCH would. Dialect::apply_limit either appends the clause or splices it into the SELECT, and takes the offset where the statement begins so a limit lands on the statement a plan ends with and never on a common table expression in front of it. Plan carries that offset as root_at, which final_query reports because it is the one place that knows. The third site was dead: Builder's limit and offset were never assigned, and they are gone rather than fixed. The dialect crate now walks every adapter and asserts it takes a limit, that SQL Server takes TOP and no suffix, that the CTE offset is honoured, and that a statement the splice cannot land in is named rather than mangled. The golden suite plans every case it can and asserts the offset a plan reports is the offset its statement has, and that the limit appears exactly once. Scoreboard unchanged at 385. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QStyZVXmJhAboYZBPGMoMw
An isolated step, so the removal that follows is a code change and not a fixture change. Every field in the 82 golden universes carried display_type, secure, disable_listing and value_list_size — 24,034 of each, one per field — and Field is deny_unknown_fields, so the keys had to leave the fixtures before they could leave the struct. The universe hash is only a filename and is never checked against the content, so the files are rewritten in place and every case's reference still resolves. manifest.json holds counts, not hashes. display_type and value_list_size gain a serde default, which is what lets a stripped universe deserialize while the struct still names them; the other four already defaulted. import_goldens.py drops the same keys on the way in, since strata still writes them and a re-import would otherwise put them back. The display_type left in 17 case files is a different key: it sits in spec.projections, where a caller sends it and nothing reads it. It is not the model's, and it stays. Scoreboard unchanged at 385. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QStyZVXmJhAboYZBPGMoMw
display_type, secure, disable_listing, disable_value_listing, value_list_size and values are gone. None of them was read by anything: the planner never consulted one, values_of consulted neither listing key, and fields() never put display_type or secure in an answer, so this removes a surface rather than a behaviour. display_type's only teeth were a validation refusing a value nothing went on to use. In their place a dimension carries enable_sampling, off unless the modeller turns it on; sample_size, 1000 by default and at most 20000; and sample_sql, a query of the modeller's own with the value in the first column. There is no third state and no type rule in the loader: enable_sampling: true is obeyed whatever the field is, because the modeller knows the data and the loader does not. Where that judgement is written down is AGENTS.md, which zsql init now generates into a project beside project.yml. It says to enable sampling only on short string dimensions under 255 characters, and why long text is the mistake to avoid: prose in the catalogue matches every request a little and nothing well, which is worse than no samples at all. The sampler will warn when what it collected looks like long text. It is a raw string, because the \n\ continuation the other templates use eats the leading whitespace a markdown list needs. The ceiling is a load-time refusal naming the field and the bound. ValueCatalog::read refuses a file past its inflated limit and the loader turns that into a warning with no values at all, so one greedy dimension would cost every other one its values. The snapshot format goes to 2, as its own comment says to when a field is renamed or removed, so a stale snapshot is told to deploy again instead of failing on a serde error. DimensionValues loses mode, whose source was the values key. Scoreboard unchanged at 385. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QStyZVXmJhAboYZBPGMoMw
Three things in it were overtaken. enable_sampling replaced disable_sampling and the Option<bool> third state, since the modeller decides and the loader does not guess. The six keys are dropped rather than renamed, and the serde aliases and kept spellings the plan argued for are unnecessary: there is no production use, so the goldens were rewritten instead. Truncation lengths are out, replaced by a warning and the modeller's own sample_sql. Phases A1 to A3 are ticked, with what each actually did. The two verification items the work settled are replaced by two it raised: whether range should leave the record, and whether the long-text warning should read the declared column width once introspection can see it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QStyZVXmJhAboYZBPGMoMw
The adapter split was wrong. Introspecting and sampling ask one statement each, and INFORMATION_SCHEMA answers it on twelve of the thirteen, so what varies is how you connect: one HTTP client with a bearer or SigV4 header reaches eight, three pure-Rust wire crates reach four, two are in-process libraries. Three units of work for thirteen adapters, not thirteen. What splits them is verifiability, not difficulty. Postgres, MySQL, SQL Server, ClickHouse and Trino run in containers and SQLite and DuckDB need no server, and those eight exercise all three transports — HTTP included, through ClickHouse and Trino, which an earlier draft wrongly assumed needed a cloud account. So B builds every transport at once against containers, and the cloud five become phase E for want of credentials to verify them against rather than for want of code. B splits in two: B1 settles the connection, the Catalogue type and the Source trait on tokio-postgres, where a credential, a timeout, a row cap and a TLS configuration get their shape, and notes that tokio enters the slim build there. B2 brings the compose file and the CI job, the HTTP transport proved against ClickHouse and Trino, the remaining wire crates and SQLite, with the gate that the same catalogue assertions pass against every container so a thirteenth adapter is a file and not a project. Comments are called out as the one field that is not uniform, and the container risks — SQL Server's image and licence, Trino's catalog configuration — are on the verify list. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QStyZVXmJhAboYZBPGMoMw
B1. zsql-source is where reaching a datasource lives, and keeping it to one crate is what makes "what can open a socket" answerable by reading one directory. Nothing depends on it yet, so the binary is unchanged at 13.8 MB stripped; the growth arrives in B2 with the introspect command. Connection is the datasources.yml keys the model loader reads past — host, port, database, schema, username, file — so nothing it carries can reach the server, which keeps only the name, the adapter and the tier. The loader has no key allowlist for a datasource, so these were already accepted and dropped; now something reads them. Secret resolves at connect time from ZSQL_DS_<UID>_PASSWORD first, then .zsql nested under the datasource key as the template promises, then flat as <uid>_password, then ~/.zsql/config, with .strata read only when .zsql is absent. The environment winning lets a shell override a checked-out project without editing it, and a blank variable means unset, as it does for every other ZSQL_ variable. Secret's Debug prints Secret(…), so it cannot reach a log by accident. Source is one read-only statement in and Rows out, text cells throughout, refusing past the row cap rather than truncating so no caller averages a partial answer. read_only is a second pair of eyes rather than a sandbox — the protection that matters is a credential with no write grant — and it works on a skeleton of the statement with comments removed and string literals blanked, so a leading block comment cannot hide what a statement starts with and a semicolon inside a value is not read as a second statement. Writing the test found that last one. Postgres is the first transport, synchronous outside on a current-thread runtime, with the statement timeout set on the session from query_timeout, a connect timeout, and TLS on the platform roots with no way to turn verification off. sslmode is Prefer so a container on loopback is reachable; a warehouse over the open internet wants Require, and that key is B2's. The transport itself needs a server, so its test is #[ignore]d and driven by ZSQL_TEST_PG_* until B2's containers arrive. Licences of all 59 new crates checked against deny.toml's allow list. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QStyZVXmJhAboYZBPGMoMw
…s proven The wire transport is now proven against a real server: the ignored test ran against the local Postgres 18.6 and read, saw a null, refused a write before sending it, and refused past the row cap rather than trimming. The HTTP transport arrives through ClickHouse, on the ureq the CLI already uses for its own calls to the server: synchronous like the rest of this crate, TLS on the platform roots with no switch to turn verification off. A statement is POSTed with FORMAT JSONCompact and rows come back as arrays under their column names. Connection gains url, for an endpoint that host and port alone cannot name. ClickHouse can enforce what this crate otherwise only promises — readonly refuses a write, max_result_rows with result_overflow_mode=throw refuses past the cap — but readonly=1, the strictest mode and the one a locked-down user is given, forbids changing any setting in a request, the cap and readonly itself included. The first run against the public playground found exactly that: the whole request refused with code 164. So a connection asks once on open what mode its session is in and sends only what that mode permits; under 1 the cap is checked here instead. Proven against play.clickhouse.com as user play, which is readonly=1. SQLite is the adapter whose read-only promise is the file's: opened with the read-only flag, a write is refused by SQLite before the guard sees it. It has no statement timeout, so a progress handler watches the clock. Its test needs no server and runs with the ordinary suite. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QStyZVXmJhAboYZBPGMoMw
harness/sources/compose.yml brings up one warehouse per transport and then some — Postgres, MySQL and SQL Server on the wire, ClickHouse and Trino over HTTP — each with a health check so `up --wait` means ready. SQLite needs no server. Every password is for a throwaway container on loopback and is the same word, so the test variables are easy to type. The CI job is its own, so a slow image never holds up the planner's tests. It brings the containers up, runs the ignored tests in zsql-source — the ones that need a server, which here they have — and takes the containers down whatever happened. Two images are the known risks. SQL Server's is large and licensed for development; Trino's ships with the tpch and memory catalogs so no configuration needs mounting, but it is slow to start. If either makes the job slow it moves to a nightly run and its transport stays proven by the other container on it. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QStyZVXmJhAboYZBPGMoMw
Catalogue::read fills one shape from per-adapter statements. The two reads every SQL adapter shares — a row per table, a row per column — are joined here by schema and name, so each adapter supplies the statements and the assembly is written once, with the column names they must answer with fixed. A warehouse with more tables or columns than the caps is refused rather than half-read, so a generated model is never silently missing a schema. Postgres takes information_schema for the shape and pg_catalog for what it lacks: the planner's row estimate from reltuples and the comments from obj_description and col_description. ClickHouse takes system.tables and system.columns, which carry comments and a count that INFORMATION_SCHEMA there does not; it has no nullable flag, the type says Nullable(…), and FixedString(N) is the only bounded string. SQLite is sqlite_master and a PRAGMA per table, with no schema, no comments and no estimate, because it keeps none. The warehouse's own type spelling is kept: mapping it to 0sql's data_type is generation's job, and keeping the original is what lets a mapping be corrected later without reading the warehouse again. A scope narrows the read to one schema; without one, everything but the system schemas is read. Proven against the local Postgres on pg_catalog, the playground on blogs, and SQLite with the ordinary suite. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QStyZVXmJhAboYZBPGMoMw
Local only, as every connection is: the project's datasources.yml names the datasource, .zsql or ZSQL_DS_<UID>_PASSWORD supplies the secret, and the catalogue lands in catalogue/<datasource>.json — one file per datasource rather than the single file first planned, so re-reading one never rewrites another — committed rather than gitignored, because it is schema and a diff in review is where drift wants to be seen. With one datasource it needs no argument; with several it says which and refuses to guess. --schema narrows the read, else the connection's own schema does, else everything but the system schemas. This is the commit that links zsql-source into zsql, so it is where the measurements are taken. Three transports in: 15.2 MB stripped against 13.8 MB before, a growth of 1.4 MB, which is under half the survey's estimate for the whole wire-and-HTTP set. The slim build, which CI checks with --no-default-features, compiles with tokio now in it. The clean release build is recorded in the plan. Run against the local Postgres, pg_catalog came back as 64 tables, 80 views and 1,423 columns in 30 ms, with pg_class carrying its row estimate and its typed columns in ordinal order; SQLite answered in 1 ms. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QStyZVXmJhAboYZBPGMoMw
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QStyZVXmJhAboYZBPGMoMw
Six of the thirteen adapters can now be opened, and the three transports are each proven by at least one of them. These three are written against the containers in harness/sources and wait for them: their tests are ignored, the CI job's environment names every one, and nothing here has run against a server yet. MySQL is the synchronous mysql crate, with the server enforcing the statement timeout through max_execution_time on the session. Trino is the client protocol: POST to /v1/statement, follow nextUri until there is none, with the catalog and schema sent as headers so that information_schema alone means this catalog's; past the row cap it tells the coordinator to stop before refusing. SQL Server is tiberius on the same current-thread runtime Postgres uses, with TDS 7.3 rather than 8.0. TLS is off on all three until the sslmode key exists, and that is a decision rather than an omission. tiberius's rustls feature pulls rustls's default crypto provider, aws-lc, in beside the tree's ring — a second provider, C and assembly in the build, and a run-time panic unless a process default is installed — and 8.0 refuses to build without a TLS feature because it is TLS-first by definition. mysql's rustls feature drags in the old webpki and the unmaintained rustls-pemfile, which cargo-deny refuses under RUSTSEC-2025-0134. With both features off the lock holds ring alone, cargo deny passes, and the sslmode change can take those decisions once, with a measurement. Each adapter has its catalogue read: information_schema for MySQL, which keeps everything including both comments and a row estimate; sys and INFORMATION_SCHEMA for SQL Server, with the row count from the partitions and the comment from the MS_Description extended property; and the named catalog's information_schema for Trino with the table comment from system.metadata. SYSTEM_SCHEMAS grows mysql, performance_schema, sys and guest. cargo-deny is now installed locally, so the check CI runs can be run here. The binary with six adapters linked is recorded below the gates. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QStyZVXmJhAboYZBPGMoMw
…aits Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QStyZVXmJhAboYZBPGMoMw
…er-only The first run of the sources job on a real runner. Every container came up, including the two flagged as risks; Postgres, ClickHouse and Trino passed, Trino for the first time against a real coordinator and ClickHouse's readonly=0 path for the first time at all. MySQL and SQL Server connected, read, refused a write and refused past the cap, and then failed at the catalogue, each for its own reason. MySQL answers an unaliased information_schema column under its defined, upper-case name, so Rows::get found no table_schema. Every column in its two statements is now aliased explicitly. SQL Server's sys.objects lists user objects only; the catalogue views are not rows in it, which is right for a model and wrong for a test that scoped to sys and expected to find them. The test reads master.dbo instead, where setup always leaves the spt_ tables and the spt_values view. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QStyZVXmJhAboYZBPGMoMw
The second run of the sources job lost Trino at the three-minute mark — "application not healthy after 3m0s" — while the first run had made it. A JVM on a two-core runner takes two to four minutes to report started, and two limits conspired: the job's --wait-timeout of 180 s, and the healthcheck's own 60 retries at 3 s, after which Docker marks the container unhealthy for good whatever the wait. A start period lets the early failed checks go uncounted, so a slow boot is waited for rather than given up on; Trino gets four minutes, SQL Server ninety seconds, and the job waits nine. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QStyZVXmJhAboYZBPGMoMw
Trino twice failed to report healthy — once at three minutes, once at nine — while the other four came up both times, and the first run had all five. A log that ends in "not healthy" cannot tell a slow JVM from a starved one from a moved latest tag, and the containers are torn down before anyone can look. So on failure the job now prints the compose status, free memory, and each container's last sixty lines with its exit code, OOM flag and health state, before the teardown runs. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QStyZVXmJhAboYZBPGMoMw
The diagnostics said what three failed runs could not: the Trino server started in 13.68 seconds with every catalog loaded, exit code zero, not killed, and Docker called it unhealthy for nine minutes regardless. The probe was the fault — a curl against /v1/info that one run's image answered and later ones did not — not the JVM, not the runner, which had twelve gigabytes free. The image ships /usr/lib/trino/bin/health-check and uses it for its own HEALTHCHECK; the compose file now calls that, which holds whatever the base image carries. The start period comes down to two minutes, since the server needs a quarter of one. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QStyZVXmJhAboYZBPGMoMw
The gate held — the same catalogue assertions pass against every container — on the fourth run of the sources job, after three were lost to a Trino probe that a curl against /v1/info satisfied once and never again while the server itself was up in fourteen seconds each time. The plan records that, and the diagnostics step that answered it, for whoever adds the seventh adapter. Image pinning joins the verify list: the tags are latest or a major, which is how the probe came to break between one run and the next. The sslmode key stays the one deferred piece of B2, with its three reasons. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QStyZVXmJhAboYZBPGMoMw
ajoabraham
marked this pull request as ready for review
October 21, 2026 19:23
Hosted minutes are finite, and strata already has the answer: a local runner over the same steps as the workflow, and `gh signoff` posting a green status to the commit when all of them pass, which is what branch protection reads. bin/ci is that, in bash, with strata's shape: every step runs, a summary says what happened, and the sign-off is a claim that everything passed — so there is no flag to skip a step, and the sources step fails plainly when the docker socket is absent and says the two ways to get it. --no-signoff runs the same and only reports. The macOS compile is the one step a Linux machine cannot run, and CI's macos job keeps it. ci.yml gains workflow_dispatch so that when the minutes run short the flip to local-only is one line, as strata's is. It also gains the docs step: bin/ci ran rustdoc with warnings denied, which CI never had, and the first local run caught an unclosed tag in a doc comment the runner had passed four times. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QStyZVXmJhAboYZBPGMoMw
The one deferred piece of B2, taken against the standards rather than from instinct. libpq, MySQL and Microsoft all separate encrypted from verified and all keep a named rung for encrypted-and-unverified, so the ladder is libpq's five under libpq's names and meanings: disable, prefer, require, verify-ca, verify-full. require encrypts and does not verify, as it does in libpq — the rung a self-signed development server needs, named rather than hidden behind a flag. The default is verify-full, which is where Microsoft moved its drivers' default in 2022, and what the PostgreSQL documentation recommends over its own prefer, which it calls a default that "makes no sense from a security point of view". prefer is accepted, never defaulted to, and a connection that ends plaintext under it says so in its warnings, which introspect prints; against the local Postgres, which offers no TLS, that is exactly what happens, and under the default the refusal names its cause. The trust store is the platform's, with the bundle when the platform has none, and ca_file on top — also read as sslrootcert and ssl_ca. Over HTTP the ladder collapses to whether plain http is allowed and whether the certificate is checked, since the scheme already encrypts. What it cost, measured as the plan asked. tiberius's rustls brings rustls's default provider, aws-lc, in beside ring through a tokio-rustls dependency with no switch for it: 2.7 MB of binary, deny-clean, and ring installed as the process default once so the two coexist. MySQL moves to mysql_async, whose tree is ring and the bundle alone; the synchronous mysql crate's TLS feature dragged in the old webpki and the unmaintained rustls-pemfile. Errors now carry their cause chain, because a driver's outer message is "error performing TLS handshake" while the reason sits one source below. The Postgres container now offers TLS with a committed self-signed pair, so require, verify-full and ca_file are proven rather than parsed; MySQL and SQL Server make themselves such certificates, so the same rungs are proven on each. Every existing container test says disable explicitly, since the default now insists. The Postgres container publishes on 5466, from the local CI run, since a developer's own Postgres holds 5432. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QStyZVXmJhAboYZBPGMoMw
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QStyZVXmJhAboYZBPGMoMw
Phase C's groundwork, and the one part of it that needed no decision. A Table now carries its primary key's columns in order and its declared foreign keys, each naming what it references, and a third shared read fills them: one row per key column, grouped here by table, kind and constraint name. One query per adapter, since this is where the warehouses differ most. Postgres reads pg_constraint, unnesting conkey with its ordinality so each row is one column in order and, for a foreign key, the referenced column at the same ordinal. MySQL reads key_column_usage, which carries the referenced side directly. SQL Server reads sys.key_constraints and sys.foreign_key_columns. SQLite reads table_info's pk column and foreign_key_list. ClickHouse gives its sorting key under the name it calls it, primary key, with a note that it is no uniqueness promise. Trino's information_schema has no constraints, so it gives none. Proven on real keys rather than on an empty answer: the Postgres and MySQL containers now run a small shop schema on first start, with a primary key, a foreign key, a view and comments on a table and a column, and the tests assert all of it, including character_maximum_length. SQLite's test adds a composite key and a foreign key. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QStyZVXmJhAboYZBPGMoMw
The keys fixture for MySQL was written and the test asserted it, but the mount that puts it where the entrypoint looks was missing from the service, so the entrypoint created the database and moved on without a word, and the catalogue read an empty schema. Resolved compose output now shows the volume; the Postgres service already had its two. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QStyZVXmJhAboYZBPGMoMw
…ss marked Phase C's generator, on four decisions. An existing model is never overwritten: a table is matched by physical_name, and when it has a model the columns the catalogue has and the model lacks are listed with the exact command to add them — --columns for some, --new for all — which appends field entries to the fields: block and touches nothing else, or prints them to paste when the file is not shaped for that. Flags and a listing rather than a prompt, since agents and CI run it. Every guess is a trailing comment on its line, which never-overwrite is what makes safe: comments survive because nothing re-emits the file. A numeric column whose name looks additive becomes a sum measure, keys stay dimensions, a string declared 255 or narrower gets enable_sampling with its width on the line, and the warehouse's own type spelling rides on every data_type line. Known types map per adapter; the text-shaped unknowns — uuid, json, enums, inet — become strings and say so; anything else is skipped and named in the file's header and the output, so a hole is never silent. Joins come from foreign keys into one rel file per schema, cost is bucketed from the row estimate. Proven end to end on SQLite in the ordinary suite: two drafts and a join the real loader accepts, a sum across both tables planned by the planner on the warehouse's own foreign key, a second run that rewrites nothing, two new columns listed and then added one by name and the rest with --new, and --dry-run writing nothing. The Postgres variant on the shop fixture asserts what SQLite cannot show — sampling from a declared width, comments as descriptions, a schema-qualified join — and the sources job runs it. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QStyZVXmJhAboYZBPGMoMw
…d be The TLS test's last two rungs fell at the handshake with the pair as the trusted CA: CaUsedAsEndEntity. openssl req -x509 mints a self-signed certificate with CA:TRUE, and rustls rightly refuses a CA certificate presented as a server's. The fixture is now the shape an enterprise warehouse actually has — a private CA, and a server certificate it signed with CA:FALSE and the localhost name — and ca_file names the CA. The README says why one self-signed file will not do. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QStyZVXmJhAboYZBPGMoMw
… one Generated from the real fixture, the orders model was the design to the letter — the warehouse's spelling beside every type, total inferred as a measure, the key marked — and the one miss was the test's expectation of a cost from a row estimate. Postgres reports reltuples as -1 for a table never analysed, which the catalogue rightly reads as no estimate. The fixture now holds a few rows and is analysed, so the path from estimate to cost is proven rather than passed over. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QStyZVXmJhAboYZBPGMoMw
…fted The last proof phase D asked for. A model drafted by generate from the real fixture turns sampling on for the 80-wide name and not for the text notes; sample reads name's values most frequent first, ties by value, with their distinct total and the limit it ran with; turning notes on by hand draws the warning the catalogue's declared type raises. The sources job runs it. The plan records what is proven, and the detour the TLS test took before it was: a vacuous pass, a real transport bug, and a fixture that was a CA certificate presented as a server's. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QStyZVXmJhAboYZBPGMoMw
Phase D's last item. The sample file's own documentation no longer says that 0sql never runs SQL to make it: zsql sample writes it, for every dimension whose model asks, never a hidden one and never one a policy names, and it ships with the model on deploy. It is customer data, gitignored by default; committing it is a decision about where those values may go, and collecting it for a project's own model is one consent while pooling it beyond that project is a separate one, which nothing here does. The command says the first half once, when it writes. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QStyZVXmJhAboYZBPGMoMw
The generated sampling query is no longer written by hand against one table. The command writes an expression line - the dimension, calc(count(*), integer) descending, the limit, lookup tables as hints - and runs what the planner returns on the datasource it chose. Which table answers is the planner's decision: a partitioned table is a more recent cache, not a reason to skip, and calculated, array and lookup dimensions are the planner's to render, so none of them is skipped any more. Only the model's own rules skip: hidden, or named by a policy. The distinct total is the same plan without the order or limit, counted as a derived table behind the plan's root offset. The planner lays dimensions before calculations, so the count cannot lead the order and ties at the cut are the warehouse's; the file is sorted by count then value so the record is stable. The plan of record says all of this and the placeholder paragraph is gone. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QStyZVXmJhAboYZBPGMoMw
Read before building phase E: dwh's Snowflake, Databricks, Athena and Redshift adapters, strata's datasource model and token store, and the strata CLI's credential prompts. Redshift is a Postgres subclass there, so the plan moves it onto the Postgres transport and leaves SigV4 to Athena alone. Credentials are per adapter by name under the datasource key, which .zsql already is. OAuth's authorization-code flow is a web app's shape and stays out of the CLI. BigQuery has no prior art in either. The phase E entry now carries the proposed shape per adapter and the five decisions it rests on. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QStyZVXmJhAboYZBPGMoMw
The first of the cloud five, on the shape the plan agreed: a statement POSTed to /api/v2/statements, 202 and a handle polled with a backoff until the statement timeout cancels it, the result paged by partition, and the cap judged from the total before any partition is fetched. Every cell is text because the request pins the output formats. A personal access token is sent as it is; a key pair signs an RS256 JWT per hour with the fingerprint Snowflake registers, on the ring already in the tree, with a signer BigQuery's service-account grant will share. No browser flow: that is a web app's shape. Secrets replaces the single password: every field .zsql may hold, resolved once, so an adapter takes the ones it understands and names everything it is missing in one error, which is what strata's model does. The connection reads account, warehouse, role and a private key path. The catalogue reads INFORMATION_SCHEMA with every alias quoted, since an unquoted one comes back upper-cased, and the keys from SHOW PRIMARY KEYS and SHOW IMPORTED KEYS mapped into the shared shape. Tests run against a fake of the endpoint on a loopback socket and a throwaway key whose fingerprint openssl computed; the signature is checked with the public key. The template shows the Snowflake entry. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QStyZVXmJhAboYZBPGMoMw
The transport is the Postgres one: port 5439, and the statement timeout and search path SET on the session, since Redshift takes no options at startup. prefer cannot be confirmed there, having no pg_stat_ssl, and the warning says to use require instead. The catalogue is Redshift's own. reltuples is not maintained, so the row count is svv_table_info's; information_schema.columns does not see a late-binding view, so the columns come from pg_attribute; and with no unnest WITH ORDINALITY, a constraint's attribute numbers arrive as the array's text and are expanded against the columns already read, paired by index for a foreign key. Two of the three statements are valid Postgres, so the fixture proves them on CI; the expansion is proven on arrays with gaps. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QStyZVXmJhAboYZBPGMoMw
Databricks over the Statement Execution API: row_limit one past the cap so the manifest says whether the result was cut, chunks fetched one by one, a token as given or a service principal's minted with the client-credentials grant at the workspace's own token endpoint. Unity Catalog's information_schema with the standard four key views. BigQuery over jobs.query and getQueryResults, paged by token and polled until the job completes. Credentials are files: a service account's key signs a grant for the read-only scope with the signer Snowflake uses, or the application-default file's refresh token is traded. The catalogue reads one dataset with __TABLES__ for rows, or a whole region without them; a STRING(n) carries its width. Athena signed with SigV4 written on ring, reproducing the signing documentation's worked example. The credential chain stops at three links: explicit keys, the AWS variables, a profile file, with the region from the same places. The header row Athena repeats as data is dropped when it matches the column names. One http::call now serves every HTTP adapter, a Source may hand back its connection for a catalogue read that needs more than its name, and the catalogue's clock shares the signer's civil-date arithmetic. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QStyZVXmJhAboYZBPGMoMw
…gates An older snapshot is told to deploy again rather than failing on the field that left, which is what the plan promised. AGENTS.md's field search is spelled as the command takes it. The TLS test no longer passes when its variable is unset. bin/ci refuses to sign off a dirty tree, since what ran was not HEAD. A sample_size that is not a number is refused rather than defaulted. The datasources.yml preamble no longer says 0sql never reads it: the command does, read-only, and the server still never connects. The plan's early sections now say what landed where they still said what was proposed, the surveys call themselves surveys, and the sampled dimension is projected quoted so a name holding a keyword reads as one field. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QStyZVXmJhAboYZBPGMoMw
The HTTP call's own limit now sits a minute past the statement timeout, since every API holds a request open while a statement runs and the poll loops own the budget; Databricks' in-request wait follows the statement timeout within what the API allows. The read guard refuses a write that rides inside a read: a data-modifying CTE, SELECT INTO, INSERT behind a WITH. Response bodies may be a catalogue's size, not ureq's ten megabytes. A SQL Server statement that times out abandons its stream and the next query reconnects, rather than reading the unread rows as its own answer. A token lifetime the answer states absurdly is an hour. The Postgres schema is held to an identifier's characters and sent quoted, Redshift gets no empty startup options and two SETs rather than one line. Half an explicit AWS credential is named, not walked past. A Snowflake account may only hold what a host name holds. The SigV4 canonical form collapses spaces in header values. A MySQL DATE prints as a date by its column type, not by its value being midnight. One token helper, one reason helper, one cell helper, one plain-url check serve every HTTP adapter, and each reports the prefer-over-http note. Trino retries a 503 on nextUri as the protocol asks. The timeout path is proven on all four polling APIs, and the guard's new refusals and Databricks' truncated flag have their cases. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QStyZVXmJhAboYZBPGMoMw
A catalogue records the schema it was narrowed to, so check counts a table outside it as unchecked rather than missing and drift compares only what both reads saw. A physical name resolves by its last two parts, so a database or project in front of them is not read as the schema, and a schema part means nothing against a catalogue that has none. generate matches an existing model by those same parts and by its datasource, appends at the indent the file's own list uses, and writes a second key to the same table as the role it needs, since the model holds one join per pair of tables. A foreign key whose referenced column could not be named is left out rather than misaligned. Postgres columns come from pg_attribute, so a materialized view has them, and a width spelled into a type (varchar(80), STRING(255)) counts on every warehouse that reports none. A MySQL view's comment is not the word VIEW. The sampler reads the value from the first column and the count from the last, whatever the dimension is called, takes a null group back out of the distinct total, leaves an author's query uncut and unsized, and matches --only on a table's uid. The samples file is written beside itself and renamed over. The Postgres tests default to the fixture's port. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QStyZVXmJhAboYZBPGMoMw
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QStyZVXmJhAboYZBPGMoMw
Two inputs: a design from first principles against this repository alone, and a reading of how strata executes queries. Where they agree the matter is settled; strata's key on SQL and datasource, its timeout precedence and its streaming threshold are taken; its job queue, its unjoined thread kill and its CSV are left. Three decisions are the owner's: where results live, truncate-and-flag or refuse past the cap, and the default TTL. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QStyZVXmJhAboYZBPGMoMw
Parquet in place of JSON lines, at a measured 48 s of clean build; an identity kept narrow and a lineage block kept wide; table versions recorded for a probe that beats a TTL; a store that is a layout a bucket can hold, with a lease rather than a pid; an index rebuildable from the files; types decided before the first row. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QStyZVXmJhAboYZBPGMoMw
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QStyZVXmJhAboYZBPGMoMw
The first phase of docs/run.md. A new crate, zsql-exec, owns the result store and the cache over it: a result is a directory named by its key under ~/.zsql/results/<project>, holding the rows as gzipped JSON lines and a manifest written last as the commit marker, so a copy of the directory is a copy of the store. The key is the exact question on the exact data for the exact principal: datasource, adapter, connection identity, the SQL as sent, the context's digest. Nothing in the model is in it, so an edit that changes the statement changes the key and one that does not keeps its cache. zsql run plans as zsql sql does, answers from the store within the datasource's TTL and says so every time, and otherwise runs on the datasource the planner chose with the cap pushed one past into the statement, so the warehouse never ships more than is kept plus one and the footer can say "of more". The four formats are explicit and never change with where stdout goes. zsql results lists, shows, exports, prints the statement, removes and purges. The manifest carries the lineage block the second pass asked for, in its own format number, and the rows format, so a converter later knows what it is reading. Proven end to end on the SQLite fixture. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QStyZVXmJhAboYZBPGMoMw
The engine takes the project and the model rather than loading its own, so the repl's session hands in the model it already holds. The remote side refuses both commands by name: the credentials are on this machine and the server holds none. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QStyZVXmJhAboYZBPGMoMw
By decision, with the cost known: the engine compiles in with the crate, about four and a half minutes on a clean build here. The file is opened in the engine's own read-only mode, so a write fails inside it before the guard is even the question. Cells are rendered by Arrow's formatter, so a decimal keeps its scale, a timestamp prints as one and a list prints as DuckDB prints it. A statement past the timeout is interrupted through the engine's handle. The catalogue reads DuckDB's own functions: tables, views, columns, comments, row estimates, and the constraints with their column lists paired by index. Both Linux release targets now build in cross's container, which carries the musl C++ toolchain the runner's musl-tools lacks; that is unproven until a release runs. The sample projects, which name DuckDB, can now be introspected, sampled and run. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QStyZVXmJhAboYZBPGMoMw
…LTO'd tiny-keccak, under const-random in DuckDB's tree, is dedicated to the public domain under CC0-1.0, which the licence list did not name. The workspace never had a release profile, so what shipped carried its symbols: stripped and thinly link-time optimised, the binary with DuckDB measures 76 MB against 97. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QStyZVXmJhAboYZBPGMoMw
The command that reads a warehouse's tables is `zsql catalog`, with `introspect` kept as an alias; its file is catalog/<uid>.json; the type is Catalog and the modules are named for it. The same pass brought the prose to American forms (optimize, license, color, behavior, serialize, analyze, center, judgment), leaving vendor constants and the frozen evaluation copies as they are. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QStyZVXmJhAboYZBPGMoMw
…hat re-reads it The file records the schemas read whole and the tables read by name, each with when. zsql catalog with nothing else re-reads every one of those and replaces what each read covers; --schema adds a scope to the union; a read of every schema subsumes the rest. Drift is computed over what a read covers, so one schema's re-read cannot report another's tables as dropped. A table a successful read asked for and did not find is dropped; a read that fails keeps what the file held and exits non-zero. Nothing is removed on a failure, only on an answer. zsql catalog search looks in the file, or in the warehouse with --live (-r keeps its one meaning, the server). zsql catalog add reads named tables and folds them in. check reads the provenance: a table outside every scope and not added is unchecked, with the command that would read it, and a catalog older than a week says so. The connection's schema is its search path, not the catalog's scope, so the default that narrowed a read to it is gone. A schema on SQLite is refused by name. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QStyZVXmJhAboYZBPGMoMw
…ers with flag forms A fresh init loads. generate drafts what is named, tables, a schema or --all, and with nothing named offers a searchable picker on a terminal or refuses with the three forms without one; a named table the catalog lacks is read live on request; a short string that is a key or looks like a person is left unsampled with the reason. sample prints its plan and asks, --yes for scripts, a hundred queries unless a budget says. -r and its kin exist only on the commands that can ask a server; a read that cannot learn the branch is told to name one. -d names the datasource everywhere; the line is positional on sql, explain, explore and run. datasource list, add and test; catalog bare, read, sync and search; deployments and undeploy; server health; completions. One rule for prompts, every prompt with a flag form, a left picker exiting 130. The help lists the commands in the order of the flow and ends with a project start to finish; every argument has a sentence; a next line follows each step. AGENTS.md carries the non-interactive flow. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QStyZVXmJhAboYZBPGMoMw
…auth, one way to make a model file --agent is one flag with one promise, JSON wherever a command has a shape and never a prompt. zsql catalog show prints one table's columns, keys and comment from the file, searching every datasource's catalog when -d is unsaid, showing a lone match and listing several. catalog search does the same across files. zsql datasource auth sets or replaces a secret after the fact. generate is the one command that writes a model file: --empty table and --empty relation take over from new, and test --new writes a test from the model's own first fields. deploy loads and tests the model here before sending, --skip-check to send anyway. check says it connects to nothing. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QStyZVXmJhAboYZBPGMoMw
… unmaintained customers.name is varchar(80) and a person's name: the draft writes its sampling line commented out with the reason, and the sample test turns it on by hand first. fxhash sits under inquire's picker filter; unmaintained, not a vulnerability, allowed with the reason beside serde_yaml's. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QStyZVXmJhAboYZBPGMoMw
…s not read as HTML Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QStyZVXmJhAboYZBPGMoMw
Two defects a real grocery-store model found, both in the drafting. A column named for a word the dialect reserves was written bare, so the dimension's expression parsed to no column at all and the loader refused the file: "Column refs dimensions should reference a table column" for a column called DAY. The draft now writes a column the way the warehouse reads it, quoted when it is a reserved word or holds anything a bare name may not, from the dialect's own keyword list and quote template. Join clauses in the rel file go the same way. A column name on two tables is one field in the model, which is how a conformed dimension works, and the loader keeps the first declaration it reads. Two tables drafted independently could therefore disagree about the type, whichever file loaded first would win, and check would report the other table as a mismatch. The type is now decided once for every table of a run, seeded by what the models already declare: the wider of the two where one is wider, with the reason on the line, and the first kept with the disagreement named where neither is. check was strict in the same place: a field over a TINYINT on one table and a BIGINT on another can only declare one type, so a declared type that covers what the warehouse holds is accepted, and only a narrower one or a different kind of thing is reported. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QStyZVXmJhAboYZBPGMoMw
The first fix for this accepted any declaration wider than the column at hand, which let a field claim bigint over nothing but tinyints and said nothing about what to declare instead. The rule is now the one the model actually needs: a field's declared type must be the widest of the columns it maps to, since that one covers them all and a narrower one claims less than the data holds. One finding per field rather than one per table, naming the type to declare, or saying the columns are different kinds of thing when no declaration describes them all. A field some of whose tables are outside the catalog keeps a wider declaration without comment, since an unread table may be its reason. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QStyZVXmJhAboYZBPGMoMw
…es and a yank RUSTSEC-2026-0385: ctutils generated its masks in non-constant time, under digest and hmac in the Postgres protocol. RUSTSEC-2026-0386: tokio's AsyncFd constructors took any AsRawFd, so safe code could build one from a closed descriptor. Both are upgrades, not allowances. clipboard-win 5.4.1, under rustyline, was yanked. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QStyZVXmJhAboYZBPGMoMw
…d drafts into a directory you pick A model with no declared foreign keys had no way to a rel file: the catalog-driven one had nothing to write, and nothing else wrote one. `generate --relationships <domain>/<name>` now writes models/<domain>/rel.<name>.yml from a template with the forms worked through, and `--infer` fills it in from the tbl.*.yml files in that directory with no query run. A column joins when its name mentions another table there and that table holds the column as a key: declared one, key-shaped by name, or that table's first column, which a drafted dimension's key nearly always is. So household_key, PRODUCT_ID and COUPON_UPC each find their dimension, a key pointing at a table that is not there is reported, and a column two tables share that keys neither is left alone with the table that still needs a join named. An inferred join is many_to_one and left, since a guess should drop no rows, and each line says what it came from. What was inferred sits above the forms, and the file is never rewritten. generate also takes the directory to draft into, as `generate shopping/` or `--dir shopping`, rather than always one directory per warehouse schema: the model's directories are its subject areas and the warehouse's shape is not the model's. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QStyZVXmJhAboYZBPGMoMw
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The branch that lets 0sql read a warehouse. The planner still never connects;
zsql-sourceis the one crate that can, andzsql introspectis the first command that does.Plan of record:
docs/introspection.md. The execution-plane survey (docs/exec.md,docs/execution-engine.md) rides along as the survey it rests on.What is in it, in commit order
LIMIT nwith no dialect indirection and SQL Server has none. Now the dialect's to place (TOP (@n)for SQL Server, spliced at the root statement's offset so a CTE never takes it), with every adapter asserted.display_type,secure,disable_listing,disable_value_listing,value_list_size,values), none read by anything; the goldens stripped of them in their own commit, as an authorised one-off.enable_sampling,sample_size(default 1000, ceiling 20000, refused at load) andsample_sql.AGENTS.mdis generated byzsql initand carries the sampling guidance; the loader enforces none of it.zsql-source:Connectionfrom thedatasources.ymlkeys the loader reads past,SecretfromZSQL_DS_<UID>_PASSWORDor.zsql, aSourcetrait with one read-only statement, a read-only guard that strips comments and blanks literals, and aCataloguetype.harness/sources/compose.ymland are what this PR'ssourcesjob exists to prove.zsql introspect, writingcatalogue/<datasource>.json, committed rather than gitignored.Measured: six adapters linked, 16.5 MB stripped (+2.7 MB); clean release build 77 s (+8 s); slim build compiles;
cargo denypasses; ring is the only crypto provider.Deliberately deferred: TLS on MySQL and SQL Server, behind an
sslmodekey, because tiberius's rustls pulls aws-lc in beside ring and mysql's pulls the unmaintainedrustls-pemfile. One decision, with a measurement, rather than three accidents.Draft until the
sourcesjob has run the three unproven adapters.🤖 Generated with Claude Code
https://claude.ai/code/session_01QStyZVXmJhAboYZBPGMoMw