From a5db84b2fd591d591d20d3047055983a2274bdbb Mon Sep 17 00:00:00 2001 From: Oleksandr Litus Date: Tue, 20 Oct 2026 22:09:44 -0400 Subject: [PATCH 1/2] Rename the project to lakekit lakebed was hard to read. The crates are now lakekit-core, lakekit-iceberg, and lakekit, and the repository is polythings/lakekit (GitHub redirects the old name). Co-Authored-By: Claude Opus 5.5 --- .github/workflows/ci.yml | 8 ++-- AGENTS.md | 4 +- CHANGELOG.md | 12 +++--- CONTRIBUTING.md | 10 ++--- Cargo.lock | 12 +++--- Cargo.toml | 8 ++-- NOTICE | 6 +-- PRIOR_ART.md | 12 +++--- README.md | 16 +++---- .../{lakebed-core => lakekit-core}/Cargo.toml | 4 +- .../{lakebed-core => lakekit-core}/README.md | 4 +- .../src/error.rs | 0 .../src/host.rs | 0 .../{lakebed-core => lakekit-core}/src/lib.rs | 6 +-- .../src/memory.rs | 0 .../src/uuid.rs | 0 .../Cargo.toml | 4 +- .../README.md | 6 +-- .../src/arrow.rs | 0 .../src/avro.rs | 0 .../src/commit.rs | 4 +- .../src/datum.rs | 0 .../src/lib.rs | 10 ++--- .../src/manifest.rs | 0 .../src/metadata.rs | 0 .../src/partition.rs | 0 .../src/stats.rs | 0 .../src/table.rs | 4 +- .../src/types.rs | 0 .../tests/iceberg_rust.rs | 42 +++++++++---------- crates/{lakebed => lakekit}/Cargo.toml | 16 +++---- crates/{lakebed => lakekit}/src/lib.rs | 6 +-- spec/0001-host.md | 2 +- spec/0002-formats.md | 10 +++-- spec/0003-iceberg.md | 16 +++---- spec/README.md | 2 +- spec/TEMPLATE.md | 2 +- spec/milestones/v0-1.md | 4 +- 38 files changed, 116 insertions(+), 114 deletions(-) rename crates/{lakebed-core => lakekit-core}/Cargo.toml (80%) rename crates/{lakebed-core => lakekit-core}/README.md (82%) rename crates/{lakebed-core => lakekit-core}/src/error.rs (100%) rename crates/{lakebed-core => lakekit-core}/src/host.rs (100%) rename crates/{lakebed-core => lakekit-core}/src/lib.rs (87%) rename crates/{lakebed-core => lakekit-core}/src/memory.rs (100%) rename crates/{lakebed-core => lakekit-core}/src/uuid.rs (100%) rename crates/{lakebed-iceberg => lakekit-iceberg}/Cargo.toml (96%) rename crates/{lakebed-iceberg => lakekit-iceberg}/README.md (84%) rename crates/{lakebed-iceberg => lakekit-iceberg}/src/arrow.rs (100%) rename crates/{lakebed-iceberg => lakekit-iceberg}/src/avro.rs (100%) rename crates/{lakebed-iceberg => lakekit-iceberg}/src/commit.rs (99%) rename crates/{lakebed-iceberg => lakekit-iceberg}/src/datum.rs (100%) rename crates/{lakebed-iceberg => lakekit-iceberg}/src/lib.rs (92%) rename crates/{lakebed-iceberg => lakekit-iceberg}/src/manifest.rs (100%) rename crates/{lakebed-iceberg => lakekit-iceberg}/src/metadata.rs (100%) rename crates/{lakebed-iceberg => lakekit-iceberg}/src/partition.rs (100%) rename crates/{lakebed-iceberg => lakekit-iceberg}/src/stats.rs (100%) rename crates/{lakebed-iceberg => lakekit-iceberg}/src/table.rs (99%) rename crates/{lakebed-iceberg => lakekit-iceberg}/src/types.rs (100%) rename crates/{lakebed-iceberg => lakekit-iceberg}/tests/iceberg_rust.rs (90%) rename crates/{lakebed => lakekit}/Cargo.toml (62%) rename crates/{lakebed => lakekit}/src/lib.rs (70%) diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 99d007e..d111120 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -35,10 +35,10 @@ jobs: env: RUSTFLAGS: -D warnings run: | - cargo check -p lakebed-iceberg --no-default-features - cargo check -p lakebed-iceberg --features write - cargo check -p lakebed-iceberg --features arrow - cargo check -p lakebed-iceberg --features parquet-stats + cargo check -p lakekit-iceberg --no-default-features + cargo check -p lakekit-iceberg --features write + cargo check -p lakekit-iceberg --features arrow + cargo check -p lakekit-iceberg --features parquet-stats wasm: name: WebAssembly (browser and WASI targets) diff --git a/AGENTS.md b/AGENTS.md index 5a7647d..34c8930 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -1,7 +1,7 @@ # Agent instructions -lakebed: table formats (Apache Iceberg first) read and written in Rust without I/O of their own, for WebAssembly and -native targets. One crate per format (`crates/lakebed-`) on a shared I/O boundary (`crates/lakebed-core`). +lakekit: table formats (Apache Iceberg first) read and written in Rust without I/O of their own, for WebAssembly and +native targets. One crate per format (`crates/lakekit-`) on a shared I/O boundary (`crates/lakekit-core`). Timelake and lakeglass use it. ## Before changing code diff --git a/CHANGELOG.md b/CHANGELOG.md index 6905974..3cb0344 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,21 +1,21 @@ # Changelog -All notable changes to lakebed. The format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/), and +All notable changes to lakekit. The format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/), and versions follow [Semantic Versioning](https://semver.org/). ## Unreleased ### Added -- `lakebed-core`: the `Host` trait for object stores, clocks, and randomness; `MemoryHost`; one `Error` type. -- `lakebed-iceberg`: Iceberg table metadata, schemas, partition specs, manifest lists, and manifests read for format +- `lakekit-core`: the `Host` trait for object stores, clocks, and randomness; `MemoryHost`; one `Error` type. +- `lakekit-iceberg`: Iceberg table metadata, schemas, partition specs, manifest lists, and manifests read for format versions 1 to 3, including delete files and deletion vector references; values in Iceberg's single-value encoding. -- `lakebed-iceberg` feature `write`: format v2 manifests, manifest lists, and metadata updates, and commits to +- `lakekit-iceberg` feature `write`: format v2 manifests, manifest lists, and metadata updates, and commits to file-system tables with options (`CommitOptions`) and hooks (`CommitHooks`). A writer whose clock is behind the table's last update dates its commit at that update, so the snapshot log stays in order. -- `lakebed-iceberg` features `arrow` (schemas as Arrow schemas with field ids) and `parquet-stats` (a data file's +- `lakekit-iceberg` features `arrow` (schemas as Arrow schemas with field ids) and `parquet-stats` (a data file's Iceberg statistics from its Parquet footer; a bound or null count only when every row group gives it exactly, and decimals as Iceberg's shortest big-endian bytes). -- `lakebed`: re-exports the format crates behind a feature each. +- `lakekit`: re-exports the format crates behind a feature each. The Iceberg code started as Timelake's shared writer (`timelake-format`), made general. diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index c3fbea5..c15535a 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -1,4 +1,4 @@ -# Contributing to lakebed +# Contributing to lakekit This is the code guide for everyone who changes this repository, people and AI agents alike. Formatters and linters enforce the mechanical rules (see [Checks](#checks)). This guide covers what they can't: how code should read. @@ -8,9 +8,9 @@ enforce the mechanical rules (see [Checks](#checks)). This guide covers what the 1. **Readable first.** Someone new should understand a function from its name, its signature, and one pass through the body. -2. **One crate per format.** A format's crate depends on `lakebed-core` and on nothing of another format's, so adding +2. **One crate per format.** A format's crate depends on `lakekit-core` and on nothing of another format's, so adding one never touches the others. Formats are shown as they are: no shared model flattens their differences. -3. **No I/O of our own.** Crates read and write through `lakebed_core::Host`. They never open sockets, files, or +3. **No I/O of our own.** Crates read and write through `lakekit_core::Host`. They never open sockets, files, or threads, so they build for `wasm32-unknown-unknown`. 4. **One definition of each thing.** Before writing a helper, constant, or type, look for an existing one. 5. **Behaviour, spec, and tests change together.** A change users can see updates its spec, adds or updates tests, and @@ -28,7 +28,7 @@ enforce the mechanical rules (see [Checks](#checks)). This guide covers what the ## Credit -lakebed learns from other projects' ideas, never from copying their code. [PRIOR_ART.md](PRIOR_ART.md) lists the +lakekit learns from other projects' ideas, never from copying their code. [PRIOR_ART.md](PRIOR_ART.md) lists the ideas we took. - **Never copy code**, whatever its license: not by pasting, and not by translating it into another language. @@ -38,7 +38,7 @@ ideas we took. ## Rust -- **Errors:** `lakebed_core::Error`, with the variant that says what a caller can do about it (`NotFound`, `Conflict`, +- **Errors:** `lakekit_core::Error`, with the variant that says what a caller can do about it (`NotFound`, `Conflict`, `CommitUnknown`, `Unsupported`, `Corrupt`). No `unwrap()` outside tests. - **Types over strings:** enums and newtypes instead of strings and magic numbers. Parse once at the boundary. - **Don't trust files.** Check lengths and offsets before using them, cap allocations, and return an error instead of diff --git a/Cargo.lock b/Cargo.lock index 963b662..0466307 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -1596,22 +1596,22 @@ dependencies = [ ] [[package]] -name = "lakebed" +name = "lakekit" version = "0.1.0" dependencies = [ - "lakebed-core", - "lakebed-iceberg", + "lakekit-core", + "lakekit-iceberg", ] [[package]] -name = "lakebed-core" +name = "lakekit-core" version = "0.1.0" dependencies = [ "thiserror", ] [[package]] -name = "lakebed-iceberg" +name = "lakekit-iceberg" version = "0.1.0" dependencies = [ "apache-avro 0.22.0", @@ -1621,7 +1621,7 @@ dependencies = [ "futures", "futures-util", "iceberg", - "lakebed-core", + "lakekit-core", "parquet", "serde_json", "tokio", diff --git a/Cargo.toml b/Cargo.toml index c32eb9e..b7cdc73 100644 --- a/Cargo.toml +++ b/Cargo.toml @@ -1,19 +1,19 @@ [workspace] resolver = "2" -members = ["crates/lakebed-core", "crates/lakebed-iceberg", "crates/lakebed"] +members = ["crates/lakekit-core", "crates/lakekit-iceberg", "crates/lakekit"] [workspace.package] version = "0.1.0" edition = "2024" rust-version = "1.99" license = "Apache-2.0" -repository = "https://github.com/polythings/lakebed" +repository = "https://github.com/polythings/lakekit" keywords = ["lakehouse", "iceberg", "wasm", "table-format"] categories = ["database-implementations", "wasm"] [workspace.dependencies] -lakebed-core = { path = "crates/lakebed-core", version = "0.1.0" } -lakebed-iceberg = { path = "crates/lakebed-iceberg", version = "0.1.0", default-features = false } +lakekit-core = { path = "crates/lakekit-core", version = "0.1.0" } +lakekit-iceberg = { path = "crates/lakekit-iceberg", version = "0.1.0", default-features = false } thiserror = "2" serde_json = "1" apache-avro = { version = "0.22", default-features = false } diff --git a/NOTICE b/NOTICE index 0ce932b..e3cf2bc 100644 --- a/NOTICE +++ b/NOTICE @@ -1,6 +1,6 @@ -lakebed -Copyright 2026 Oleksandr Litus and the lakebed contributors +lakekit +Copyright 2026 Oleksandr Litus and the lakekit contributors This product is licensed under the Apache License, Version 2.0 (see LICENSE). -Ideas lakebed borrowed from other projects are credited in PRIOR_ART.md. +Ideas lakekit borrowed from other projects are credited in PRIOR_ART.md. diff --git a/PRIOR_ART.md b/PRIOR_ART.md index 7622547..da7a0db 100644 --- a/PRIOR_ART.md +++ b/PRIOR_ART.md @@ -1,11 +1,11 @@ # Prior art -lakebed stands on other open-source projects and learns from more. This file credits the ideas we took. We take ideas, -never code: every line of lakebed is our own. How we credit is in [CONTRIBUTING.md](CONTRIBUTING.md#credit). +lakekit stands on other open-source projects and learns from more. This file credits the ideas we took. We take ideas, +never code: every line of lakekit is our own. How we credit is in [CONTRIBUTING.md](CONTRIBUTING.md#credit). ## Foundations -| Project | License | What lakebed uses | +| Project | License | What lakekit uses | |---|---|---| | [Apache Iceberg](https://iceberg.apache.org) | Apache-2.0 | The table format and its specification | | [iceberg-rust](https://github.com/apache/iceberg-rust) | Apache-2.0 | The reference the interop tests check against (a test dependency only) | @@ -14,9 +14,9 @@ never code: every line of lakebed is our own. How we credit is in [CONTRIBUTING. ## Ideas -| Idea | From | License of the source | Where in lakebed | +| Idea | From | License of the source | Where in lakekit | |---|---|---|---| -| A format library with no I/O of its own: the application supplies storage through a trait the library calls | [delta-kernel-rs](https://github.com/delta-io/delta-kernel-rs) (its `Engine` trait) | Apache-2.0 | `lakebed-core`'s `Host` | +| A format library with no I/O of its own: the application supplies storage through a trait the library calls | [delta-kernel-rs](https://github.com/delta-io/delta-kernel-rs) (its `Engine` trait) | Apache-2.0 | `lakekit-core`'s `Host` | | One crate per backend under a small shared core, re-exported by an umbrella crate with a feature each | [arrow-rs](https://github.com/apache/arrow-rs), [Apache OpenDAL](https://github.com/apache/opendal) | Apache-2.0 | The workspace layout | -| Commits to tables without a catalog: `v.metadata.json` written only if absent, `version-hint.text` for readers | Iceberg's file-system ("Hadoop") tables | Apache-2.0 | `lakebed_iceberg::commit` | +| Commits to tables without a catalog: `v.metadata.json` written only if absent, `version-hint.text` for readers | Iceberg's file-system ("Hadoop") tables | Apache-2.0 | `lakekit_iceberg::commit` | | Avro-safe names for partition fields (`_x` and the character's code in hex) | Iceberg's Java implementation | Apache-2.0 | Manifest writing | diff --git a/README.md b/README.md index c4a52da..13a5876 100644 --- a/README.md +++ b/README.md @@ -1,11 +1,11 @@ -# lakebed +# lakekit Table formats read and written without I/O of their own, in Rust, for WebAssembly and native targets. Each format's metadata (Apache Iceberg first) is parsed and written by plain code that asks a `Host` for bytes: the application implements the host over its object store, so the same crate runs in a browser, a Cloudflare Worker, a CLI, or a server. -lakebed is what [Timelake](https://github.com/polythings/timelake) and [lakeglass](https://github.com/polythings/lakeglass) +lakekit is what [Timelake](https://github.com/polythings/timelake) and [lakeglass](https://github.com/polythings/lakeglass) share. The official Rust crates for these formats build on tokio and HTTP clients, which don't compile to `wasm32-unknown-unknown`. @@ -13,9 +13,9 @@ share. The official Rust crates for these formats build on tokio and HTTP client | Crate | What it does | |---|---| -| [`lakebed-core`](crates/lakebed-core) | The `Host` trait (ranged reads, listing, conditional writes, clock, randomness), errors, an in-memory host | -| [`lakebed-iceberg`](crates/lakebed-iceberg) | Apache Iceberg: metadata, manifests, and commits | -| [`lakebed`](crates/lakebed) | Re-exports the format crates, one feature each | +| [`lakekit-core`](crates/lakekit-core) | The `Host` trait (ranged reads, listing, conditional writes, clock, randomness), errors, an in-memory host | +| [`lakekit-iceberg`](crates/lakekit-iceberg) | Apache Iceberg: metadata, manifests, and commits | +| [`lakekit`](crates/lakekit) | Re-exports the format crates, one feature each | More formats get their own crate as they are added (spec 2). @@ -30,12 +30,12 @@ More formats get their own crate as they are added (spec 2). | Schema as an Arrow schema with field ids | Feature `arrow` | | A data file's Iceberg statistics from its Parquet footer | Feature `parquet-stats` | -What lakebed writes, iceberg-rust reads, and the other way round: the tests commit to one table with both in turn. +What lakekit writes, iceberg-rust reads, and the other way round: the tests commit to one table with both in turn. ```rust -use lakebed_iceberg::Table; +use lakekit_iceberg::Table; -async fn files(host: &H) -> lakebed_core::Result { +async fn files(host: &H) -> lakekit_core::Result { let mut table = Table::load(host, "s3://bucket/warehouse/db/events").await?; Ok(table.live_files(host).await?.len()) } diff --git a/crates/lakebed-core/Cargo.toml b/crates/lakekit-core/Cargo.toml similarity index 80% rename from crates/lakebed-core/Cargo.toml rename to crates/lakekit-core/Cargo.toml index b414a77..ec137de 100644 --- a/crates/lakebed-core/Cargo.toml +++ b/crates/lakekit-core/Cargo.toml @@ -1,6 +1,6 @@ [package] -name = "lakebed-core" -description = "The I/O boundary shared by lakebed's table format crates: a host trait for object stores, clocks, and randomness" +name = "lakekit-core" +description = "The I/O boundary shared by lakekit's table format crates: a host trait for object stores, clocks, and randomness" readme = "README.md" version.workspace = true edition.workspace = true diff --git a/crates/lakebed-core/README.md b/crates/lakekit-core/README.md similarity index 82% rename from crates/lakebed-core/README.md rename to crates/lakekit-core/README.md index 9d74b43..734e18c 100644 --- a/crates/lakebed-core/README.md +++ b/crates/lakekit-core/README.md @@ -1,6 +1,6 @@ -# lakebed-core +# lakekit-core -The I/O boundary of [lakebed](https://github.com/polythings/lakebed)'s table format crates. Format crates do no I/O of +The I/O boundary of [lakekit](https://github.com/polythings/lakekit)'s table format crates. Format crates do no I/O of their own: they call a `Host` that the application implements over its object store, clock, and randomness, so they build for `wasm32-unknown-unknown` and native targets alike. diff --git a/crates/lakebed-core/src/error.rs b/crates/lakekit-core/src/error.rs similarity index 100% rename from crates/lakebed-core/src/error.rs rename to crates/lakekit-core/src/error.rs diff --git a/crates/lakebed-core/src/host.rs b/crates/lakekit-core/src/host.rs similarity index 100% rename from crates/lakebed-core/src/host.rs rename to crates/lakekit-core/src/host.rs diff --git a/crates/lakebed-core/src/lib.rs b/crates/lakekit-core/src/lib.rs similarity index 87% rename from crates/lakebed-core/src/lib.rs rename to crates/lakekit-core/src/lib.rs index 5975f62..ddc231a 100644 --- a/crates/lakebed-core/src/lib.rs +++ b/crates/lakekit-core/src/lib.rs @@ -1,16 +1,16 @@ -//! The I/O boundary shared by lakebed's table format crates. Format crates do no I/O of their own: +//! The I/O boundary shared by lakekit's table format crates. Format crates do no I/O of their own: //! they read and write through a [`Host`] the application implements over its object store (an //! `object_store` client natively, `fetch` or an R2 binding in WebAssembly), so they build for //! `wasm32-unknown-unknown` and native targets alike. //! //! ``` -//! use lakebed_core::{Host, MemoryHost}; +//! use lakekit_core::{Host, MemoryHost}; //! //! # block_on(async { //! let host = MemoryHost::new(0); //! host.put("memory://t/a", b"x".to_vec()).await?; //! assert!(host.exists("memory://t/a").await?); -//! # Ok::<(), lakebed_core::Error>(()) +//! # Ok::<(), lakekit_core::Error>(()) //! # }).unwrap(); //! # // MemoryHost's futures are ready at once, so one poll runs them. //! # fn block_on(f: F) -> F::Output { diff --git a/crates/lakebed-core/src/memory.rs b/crates/lakekit-core/src/memory.rs similarity index 100% rename from crates/lakebed-core/src/memory.rs rename to crates/lakekit-core/src/memory.rs diff --git a/crates/lakebed-core/src/uuid.rs b/crates/lakekit-core/src/uuid.rs similarity index 100% rename from crates/lakebed-core/src/uuid.rs rename to crates/lakekit-core/src/uuid.rs diff --git a/crates/lakebed-iceberg/Cargo.toml b/crates/lakekit-iceberg/Cargo.toml similarity index 96% rename from crates/lakebed-iceberg/Cargo.toml rename to crates/lakekit-iceberg/Cargo.toml index 946ec88..b6f0d96 100644 --- a/crates/lakebed-iceberg/Cargo.toml +++ b/crates/lakekit-iceberg/Cargo.toml @@ -1,5 +1,5 @@ [package] -name = "lakebed-iceberg" +name = "lakekit-iceberg" description = "Apache Iceberg table metadata, manifests and commits without I/O of its own, for WebAssembly and native" readme = "README.md" version.workspace = true @@ -20,7 +20,7 @@ arrow = ["dep:arrow-schema"] parquet-stats = ["dep:parquet"] [dependencies] -lakebed-core.workspace = true +lakekit-core.workspace = true apache-avro.workspace = true serde_json.workspace = true futures-util.workspace = true diff --git a/crates/lakebed-iceberg/README.md b/crates/lakekit-iceberg/README.md similarity index 84% rename from crates/lakebed-iceberg/README.md rename to crates/lakekit-iceberg/README.md index cb493a8..efd7e77 100644 --- a/crates/lakebed-iceberg/README.md +++ b/crates/lakekit-iceberg/README.md @@ -1,7 +1,7 @@ -# lakebed-iceberg +# lakekit-iceberg [Apache Iceberg](https://iceberg.apache.org) tables read and written without I/O of their own, through a -[`lakebed_core::Host`](../lakebed-core), so they work in WebAssembly (browsers, Cloudflare Workers) as well as natively. +[`lakekit_core::Host`](../lakekit-core), so they work in WebAssembly (browsers, Cloudflare Workers) as well as natively. | Feature | What it adds | |---|---| @@ -10,6 +10,6 @@ | `arrow` | The schema as an Arrow schema, with field ids | | `parquet-stats` | A data file's Iceberg statistics (counts, bounds, split offsets) from its Parquet footer | -The tests commit to one table with lakebed and iceberg-rust in turn, each reading what the other wrote. +The tests commit to one table with lakekit and iceberg-rust in turn, each reading what the other wrote. Apache License 2.0. diff --git a/crates/lakebed-iceberg/src/arrow.rs b/crates/lakekit-iceberg/src/arrow.rs similarity index 100% rename from crates/lakebed-iceberg/src/arrow.rs rename to crates/lakekit-iceberg/src/arrow.rs diff --git a/crates/lakebed-iceberg/src/avro.rs b/crates/lakekit-iceberg/src/avro.rs similarity index 100% rename from crates/lakebed-iceberg/src/avro.rs rename to crates/lakekit-iceberg/src/avro.rs diff --git a/crates/lakebed-iceberg/src/commit.rs b/crates/lakekit-iceberg/src/commit.rs similarity index 99% rename from crates/lakebed-iceberg/src/commit.rs rename to crates/lakekit-iceberg/src/commit.rs index eb3d396..506fc8b 100644 --- a/crates/lakebed-iceberg/src/commit.rs +++ b/crates/lakekit-iceberg/src/commit.rs @@ -12,7 +12,7 @@ use std::future::Future; use std::hash::BuildHasher; use std::sync::Arc; -use lakebed_core::{Host, uuid_v4}; +use lakekit_core::{Host, uuid_v4}; use crate::manifest::{ DataFile, Entry, ManifestContext, ManifestFile, ManifestWrite, Status, manifest_file, write_manifest, @@ -469,7 +469,7 @@ pub fn counters(change: &Change, base: &HashMap(()) +//! # Ok::<(), lakekit_core::Error>(()) //! ``` #[cfg(feature = "arrow")] @@ -35,7 +35,7 @@ mod types; #[cfg(feature = "write")] pub use commit::{Change, CommitHooks, CommitInfo, CommitOptions, Extra, NoHooks, commit}; pub use datum::{Datum, from_be, minimal_be}; -pub use lakebed_core::{Error, Result}; +pub use lakekit_core::{Error, Result}; pub use manifest::{ Content, DataFile, Entry, FieldSummary, ManifestContent, ManifestFile, Status, read_manifest, read_manifest_list, }; @@ -76,7 +76,7 @@ fn corrupt(msg: impl Into) -> Error { Error::Corrupt(msg.into()) } -/// Runs a future whose host calls complete at once ([`lakebed_core::MemoryHost`]'s do). +/// Runs a future whose host calls complete at once ([`lakekit_core::MemoryHost`]'s do). #[cfg(test)] fn block_on(f: F) -> F::Output { let mut cx = std::task::Context::from_waker(std::task::Waker::noop()); diff --git a/crates/lakebed-iceberg/src/manifest.rs b/crates/lakekit-iceberg/src/manifest.rs similarity index 100% rename from crates/lakebed-iceberg/src/manifest.rs rename to crates/lakekit-iceberg/src/manifest.rs diff --git a/crates/lakebed-iceberg/src/metadata.rs b/crates/lakekit-iceberg/src/metadata.rs similarity index 100% rename from crates/lakebed-iceberg/src/metadata.rs rename to crates/lakekit-iceberg/src/metadata.rs diff --git a/crates/lakebed-iceberg/src/partition.rs b/crates/lakekit-iceberg/src/partition.rs similarity index 100% rename from crates/lakebed-iceberg/src/partition.rs rename to crates/lakekit-iceberg/src/partition.rs diff --git a/crates/lakebed-iceberg/src/stats.rs b/crates/lakekit-iceberg/src/stats.rs similarity index 100% rename from crates/lakebed-iceberg/src/stats.rs rename to crates/lakekit-iceberg/src/stats.rs diff --git a/crates/lakebed-iceberg/src/table.rs b/crates/lakekit-iceberg/src/table.rs similarity index 99% rename from crates/lakebed-iceberg/src/table.rs rename to crates/lakekit-iceberg/src/table.rs index 7cadb70..a2a7df2 100644 --- a/crates/lakebed-iceberg/src/table.rs +++ b/crates/lakekit-iceberg/src/table.rs @@ -3,7 +3,7 @@ //! them, with `version-hint.text` naming the newest for stores that can't list. Tables a catalog //! names with other file names load with [`Table::from_metadata_file`]. -use lakebed_core::Host; +use lakekit_core::Host; use crate::manifest::{Entry, ManifestFile, Status, read_manifest, read_manifest_list}; use crate::metadata::Metadata; @@ -123,7 +123,7 @@ impl Table { #[cfg(test)] mod tests { - use lakebed_core::MemoryHost; + use lakekit_core::MemoryHost; use super::*; use crate::block_on; diff --git a/crates/lakebed-iceberg/src/types.rs b/crates/lakekit-iceberg/src/types.rs similarity index 100% rename from crates/lakebed-iceberg/src/types.rs rename to crates/lakekit-iceberg/src/types.rs diff --git a/crates/lakebed-iceberg/tests/iceberg_rust.rs b/crates/lakekit-iceberg/tests/iceberg_rust.rs similarity index 90% rename from crates/lakebed-iceberg/tests/iceberg_rust.rs rename to crates/lakekit-iceberg/tests/iceberg_rust.rs index 2f90eda..88878ed 100644 --- a/crates/lakebed-iceberg/tests/iceberg_rust.rs +++ b/crates/lakekit-iceberg/tests/iceberg_rust.rs @@ -1,6 +1,6 @@ -//! lakebed-iceberg and iceberg-rust read each other's tables: a table iceberg-rust creates and -//! appends to, lakebed reads and commits to; iceberg-rust reads lakebed's commit and appends on top -//! of it; and lakebed computes the statistics iceberg-rust's Parquet writer records. Everything +//! lakekit-iceberg and iceberg-rust read each other's tables: a table iceberg-rust creates and +//! appends to, lakekit reads and commits to; iceberg-rust reads lakekit's commit and appends on top +//! of it; and lakekit computes the statistics iceberg-rust's Parquet writer records. Everything //! runs in memory. #![cfg(all(feature = "write", feature = "arrow", feature = "parquet-stats"))] #![expect(clippy::unwrap_used, reason = "test code")] @@ -21,9 +21,9 @@ use iceberg::spec::{ use iceberg::transaction::{ApplyTransactionAction, Transaction}; use iceberg::writer::file_writer::{FileWriter as _, FileWriterBuilder, ParquetWriterBuilder}; use iceberg::{Catalog, CatalogBuilder, MemoryCatalog, NamespaceIdent, TableCreation, TableIdent}; -use lakebed_core::{Error, Host, Listed, MemoryHost, MemoryUpload}; -use lakebed_iceberg::stats::ParquetStats; -use lakebed_iceberg::{Change, CommitOptions, DataFile, Datum, Entry, NoHooks, Status, Table, commit}; +use lakekit_core::{Error, Host, Listed, MemoryHost, MemoryUpload}; +use lakekit_iceberg::stats::ParquetStats; +use lakekit_iceberg::{Change, CommitOptions, DataFile, Datum, Entry, NoHooks, Status, Table, commit}; use parquet::arrow::ArrowWriter; use parquet::file::metadata::ParquetMetaDataReader; use parquet::file::properties::WriterProperties; @@ -46,45 +46,45 @@ fn store(e: impl std::fmt::Display) -> Error { impl Host for IoHost { type Upload = MemoryUpload; - async fn get(&self, uri: &str) -> lakebed_core::Result>> { + async fn get(&self, uri: &str) -> lakekit_core::Result>> { if !self.io.exists(uri).await.map_err(store)? { return Ok(None); } Ok(Some(self.io.new_input(uri).map_err(store)?.read().await.map_err(store)?.to_vec())) } - async fn get_range(&self, uri: &str, start: u64, len: u64) -> lakebed_core::Result> { + async fn get_range(&self, uri: &str, start: u64, len: u64) -> lakekit_core::Result> { let bytes = self.get(uri).await?.ok_or_else(|| Error::NotFound(uri.to_string()))?; let start = usize::try_from(start).map_err(store)?.min(bytes.len()); let end = start.saturating_add(usize::try_from(len).map_err(store)?).min(bytes.len()); Ok(bytes[start..end].to_vec()) } - async fn exists(&self, uri: &str) -> lakebed_core::Result { + async fn exists(&self, uri: &str) -> lakekit_core::Result { self.io.exists(uri).await.map_err(store) } - async fn put(&self, uri: &str, bytes: Vec) -> lakebed_core::Result<()> { + async fn put(&self, uri: &str, bytes: Vec) -> lakekit_core::Result<()> { self.io.new_output(uri).map_err(store)?.write(Bytes::from(bytes)).await.map_err(store) } /// The tests run one writer at a time, so checking first is enough. - async fn put_if_absent(&self, uri: &str, bytes: Vec) -> lakebed_core::Result { + async fn put_if_absent(&self, uri: &str, bytes: Vec) -> lakekit_core::Result { if self.exists(uri).await? { return Ok(false); } self.put(uri, bytes).await.map(|()| true) } - fn upload(&self, _: &str) -> impl Future> { + fn upload(&self, _: &str) -> impl Future> { ready(Err(Error::Unsupported("uploads".into()))) } - fn list(&self, _: &str) -> impl Future>> { + fn list(&self, _: &str) -> impl Future>> { ready(Err(Error::Unsupported("listing".into()))) } - async fn delete(&self, uris: &[String]) -> lakebed_core::Result<()> { + async fn delete(&self, uris: &[String]) -> lakekit_core::Result<()> { for uri in uris { self.io.delete(uri).await.map_err(store)?; } @@ -151,7 +151,7 @@ async fn write_theirs(table: &iceberg::table::Table, batch: &RecordBatch, name: files.remove(0).partition(Struct::from_iter([Some(Literal::int(day))])).partition_spec_id(0).build().unwrap() } -/// Writes `batch` with arrow-rs's Parquet writer; returns lakebed's data file. +/// Writes `batch` with arrow-rs's Parquet writer; returns lakekit's data file. async fn write_ours(host: &IoHost, table: &Table, batch: &RecordBatch, name: &str, day: i32) -> DataFile { let mut bytes = vec![]; let mut writer = ArrowWriter::try_new(&mut bytes, batch.schema(), None).unwrap(); @@ -173,7 +173,7 @@ async fn append_theirs( tx.commit(catalog).await.unwrap() } -/// The data file iceberg-rust records, as lakebed records it. +/// The data file iceberg-rust records, as lakekit records it. fn as_ours(f: &IceDataFile) -> DataFile { let bytes = |m: &HashMap| m.iter().map(|(k, d)| (*k, d.to_bytes().unwrap().to_vec())).collect(); @@ -221,7 +221,7 @@ async fn live(host: &IoHost, metadata_file: &str) -> (Table, Vec) { } #[tokio::test] -async fn tables_round_trip_between_lakebed_and_iceberg_rust() { +async fn tables_round_trip_between_lakekit_and_iceberg_rust() { let catalog = MemoryCatalogBuilder::default() .load("memory", HashMap::from([("warehouse".to_string(), "memory://wh".to_string())])) .await @@ -241,7 +241,7 @@ async fn tables_round_trip_between_lakebed_and_iceberg_rust() { let host = IoHost { io: io.clone(), memory: MemoryHost::new(T0 / 1000) }; let arrow = Arc::new(iceberg::arrow::schema_to_arrow_schema(table.metadata().current_schema()).unwrap()); - // iceberg-rust appends; lakebed reads the file as iceberg-rust records it. + // iceberg-rust appends; lakekit reads the file as iceberg-rust records it. let first = batch(&arrow, 0, 5_000); let theirs = write_theirs(&table, &first, "theirs-1", 20_454).await; let table = append_theirs(&catalog, &table, theirs.clone()).await; @@ -250,13 +250,13 @@ async fn tables_round_trip_between_lakebed_and_iceberg_rust() { assert_eq!(*files[0].file, as_ours(&theirs)); assert_eq!((files[0].status, files[0].sequence_number, files[0].file_sequence_number), (Status::Added, 1, 1)); - // lakebed computes the statistics iceberg-rust's writer recorded, from the file's footer. + // lakekit computes the statistics iceberg-rust's writer recorded, from the file's footer. let bytes = io.new_input(theirs.file_path()).unwrap().read().await.unwrap(); let footer = ParquetMetaDataReader::new().parse_and_finish(&bytes).unwrap(); let stats = ParquetStats::new(&footer, bytes.len() as u64, nan_counts(&first)); assert_eq!(stats.data_file(theirs.file_path().into(), vec![Some(Datum::Date(20_454))]), as_ours(&theirs)); - // lakebed commits on top; iceberg-rust reads the new version and lakebed's manifests. + // lakekit commits on top; iceberg-rust reads the new version and lakekit's manifests. let ours = write_ours(&host, &base, &batch(&arrow, 1, 3_000), "ours-1", 20_455).await; let (committed, info) = commit( &host, @@ -279,7 +279,7 @@ async fn tables_round_trip_between_lakebed_and_iceberg_rust() { let (entries, _) = Manifest::parse_avro(&manifest).unwrap().into_parts(); assert_eq!(as_ours(&entries[0].data_file), ours); - // iceberg-rust appends on top of lakebed's version; lakebed reads all three files. Its memory + // iceberg-rust appends on top of lakekit's version; lakekit reads all three files. Its memory // catalog only names metadata files `NNNNN-.metadata.json`, so it gets a copy so named. let ident = TableIdent::from_strs(["ns", "registered"]).unwrap(); let copy = diff --git a/crates/lakebed/Cargo.toml b/crates/lakekit/Cargo.toml similarity index 62% rename from crates/lakebed/Cargo.toml rename to crates/lakekit/Cargo.toml index 766e29c..31e61f7 100644 --- a/crates/lakebed/Cargo.toml +++ b/crates/lakekit/Cargo.toml @@ -1,5 +1,5 @@ [package] -name = "lakebed" +name = "lakekit" description = "Table formats (Apache Iceberg first) read and written without I/O of their own, for WebAssembly and native" readme = "../../README.md" version.workspace = true @@ -12,18 +12,18 @@ categories.workspace = true [features] default = ["iceberg"] -# Apache Iceberg (lakebed-iceberg). -iceberg = ["dep:lakebed-iceberg"] +# Apache Iceberg (lakekit-iceberg). +iceberg = ["dep:lakekit-iceberg"] # Writing Iceberg tables. -iceberg-write = ["iceberg", "lakebed-iceberg/write"] +iceberg-write = ["iceberg", "lakekit-iceberg/write"] # Iceberg schemas as Arrow schemas. -iceberg-arrow = ["iceberg", "lakebed-iceberg/arrow"] +iceberg-arrow = ["iceberg", "lakekit-iceberg/arrow"] # Iceberg statistics from Parquet footers. -iceberg-parquet-stats = ["iceberg", "lakebed-iceberg/parquet-stats"] +iceberg-parquet-stats = ["iceberg", "lakekit-iceberg/parquet-stats"] [dependencies] -lakebed-core.workspace = true -lakebed-iceberg = { workspace = true, optional = true } +lakekit-core.workspace = true +lakekit-iceberg = { workspace = true, optional = true } [package.metadata.docs.rs] all-features = true diff --git a/crates/lakebed/src/lib.rs b/crates/lakekit/src/lib.rs similarity index 70% rename from crates/lakebed/src/lib.rs rename to crates/lakekit/src/lib.rs index 1380306..50feaf8 100644 --- a/crates/lakebed/src/lib.rs +++ b/crates/lakekit/src/lib.rs @@ -3,9 +3,9 @@ //! //! | Feature | Crate | //! |---|---| -//! | `iceberg` (default) | [`iceberg`]: Apache Iceberg (`lakebed-iceberg`) | +//! | `iceberg` (default) | [`iceberg`]: Apache Iceberg (`lakekit-iceberg`) | //! | `iceberg-write`, `iceberg-arrow`, `iceberg-parquet-stats` | its optional parts | -pub use lakebed_core as core; +pub use lakekit_core as core; #[cfg(feature = "iceberg")] -pub use lakebed_iceberg as iceberg; +pub use lakekit_iceberg as iceberg; diff --git a/spec/0001-host.md b/spec/0001-host.md index 426269b..416ff6d 100644 --- a/spec/0001-host.md +++ b/spec/0001-host.md @@ -14,7 +14,7 @@ issues: [] ## Summary -Format crates do no I/O of their own. They call a `Host` (`lakebed-core`) that the application implements over its +Format crates do no I/O of their own. They call a `Host` (`lakekit-core`) that the application implements over its object store, clock, and randomness, so the same code runs in a browser, a Cloudflare Worker, a CLI, or a server, and tests run against memory. diff --git a/spec/0002-formats.md b/spec/0002-formats.md index 979a0b2..7af34e9 100644 --- a/spec/0002-formats.md +++ b/spec/0002-formats.md @@ -14,13 +14,13 @@ issues: [6] ## Summary -Each table format is its own crate, `lakebed-`, depending on `lakebed-core` and nothing of another format's. -The `lakebed` crate re-exports them behind a feature each. Iceberg is first (spec 3), because Timelake writes it and +Each table format is its own crate, `lakekit-`, depending on `lakekit-core` and nothing of another format's. +The `lakekit` crate re-exports them behind a feature each. Iceberg is first (spec 3), because Timelake writes it and lakeglass reads it; the formats lakeglass explores come next. ## Design -- **Layout:** `crates/lakebed-core` (spec 1), `crates/lakebed-`, and `crates/lakebed`, whose features are the +- **Layout:** `crates/lakekit-core` (spec 1), `crates/lakekit-`, and `crates/lakekit`, whose features are the format names (`iceberg`) and their parts (`iceberg-write`, `iceberg-arrow`, `iceberg-parquet-stats`). - **Formats as they are.** A format crate exposes its format's own model (Iceberg's snapshots and manifests, Delta's log, Hudi's timeline) without a shared abstraction that would hide the differences. A common view, where an @@ -43,8 +43,10 @@ lakeglass reads it; the formats lakeglass explores come next. ## Open questions and issues -- [#6](https://github.com/polythings/lakebed/issues/6) Which format comes after Iceberg, and in what order? +- [#6](https://github.com/polythings/lakekit/issues/6) Which format comes after Iceberg, and in what order? ## History - 2026-10-20: proposed and the layout built, with Iceberg as the first format. +- 2026-10-21: the project renamed from lakebed to lakekit, an easier name to read; crates `lakekit-core`, + `lakekit-iceberg`, and `lakekit`. diff --git a/spec/0003-iceberg.md b/spec/0003-iceberg.md index 0999099..21f4254 100644 --- a/spec/0003-iceberg.md +++ b/spec/0003-iceberg.md @@ -14,7 +14,7 @@ issues: [1, 2, 3, 4, 5] ## Summary -`lakebed-iceberg` reads Iceberg tables of format versions 1 to 3 and writes format v2 tables, without I/O of its own +`lakekit-iceberg` reads Iceberg tables of format versions 1 to 3 and writes format v2 tables, without I/O of its own (spec 1). Timelake's WASM reader and writer use it, and lakeglass's table explorer will. It started as Timelake's shared writer, made general. @@ -57,8 +57,8 @@ shared writer, made general. iceberg-rust's writer records them; decimals become Iceberg's shortest big-endian bytes however Parquet stores them. A bound or null count is recorded only when every row group gives it exactly (a row group of nulls only needs no bounds): readers take them as covering the whole file, and a partial one would let them skip matching rows. -- **Interop tests:** iceberg-rust creates a table and appends; lakebed reads it and commits; iceberg-rust reads that - version, parses lakebed's manifests, and appends on top; lakebed reads all of it. lakebed's statistics of +- **Interop tests:** iceberg-rust creates a table and appends; lakekit reads it and commits; iceberg-rust reads that + version, parses lakekit's manifests, and appends on top; lakekit reads all of it. lakekit's statistics of iceberg-rust's Parquet file equal what iceberg-rust recorded. **Not yet:** applying partition transforms to values, planning files from filters, writing format v3, writing delete @@ -74,11 +74,11 @@ files, and catalogs (REST). Each has an issue. ## Open questions and issues -- [#1](https://github.com/polythings/lakebed/issues/1) Apply partition transforms to values -- [#2](https://github.com/polythings/lakebed/issues/2) Plan files and row groups from filters -- [#3](https://github.com/polythings/lakebed/issues/3) Write uuid, fixed and decimal partition values -- [#4](https://github.com/polythings/lakebed/issues/4) Write format v3 tables -- [#5](https://github.com/polythings/lakebed/issues/5) Iceberg REST catalogs: here or in lakeglass? +- [#1](https://github.com/polythings/lakekit/issues/1) Apply partition transforms to values +- [#2](https://github.com/polythings/lakekit/issues/2) Plan files and row groups from filters +- [#3](https://github.com/polythings/lakekit/issues/3) Write uuid, fixed and decimal partition values +- [#4](https://github.com/polythings/lakekit/issues/4) Write format v3 tables +- [#5](https://github.com/polythings/lakekit/issues/5) Iceberg REST catalogs: here or in lakeglass? ## History diff --git a/spec/README.md b/spec/README.md index d261de6..b973c5c 100644 --- a/spec/README.md +++ b/spec/README.md @@ -1,6 +1,6 @@ # Specs -lakebed's design, laid out like Timelake's and lakeglass's. Start here, people and agents alike. +lakekit's design, laid out like Timelake's and lakeglass's. Start here, people and agents alike. - **Specs** (`NNNN-name.md`): one feature each, with the same template ([TEMPLATE.md](TEMPLATE.md)). A spec says what the feature does and why, what is still open (as GitHub issues), how far it is built, and the releases that changed diff --git a/spec/TEMPLATE.md b/spec/TEMPLATE.md index 321ab22..e67bf2a 100644 --- a/spec/TEMPLATE.md +++ b/spec/TEMPLATE.md @@ -29,7 +29,7 @@ as "spec N", linked to their files. ## Open questions and issues -- [#N](https://github.com/polythings/lakebed/issues/N) Each open question or task is an issue. +- [#N](https://github.com/polythings/lakekit/issues/N) Each open question or task is an issue. ## History diff --git a/spec/milestones/v0-1.md b/spec/milestones/v0-1.md index a249c4b..83e8246 100644 --- a/spec/milestones/v0-1.md +++ b/spec/milestones/v0-1.md @@ -3,7 +3,7 @@ **In progress** The first release: the host boundary and the Iceberg crate, enough for Timelake's WASM reader and writer to use -lakebed instead of their own Iceberg code, and for lakeglass to read Iceberg tables in the browser. +lakekit instead of their own Iceberg code, and for lakeglass to read Iceberg tables in the browser. | # | Spec | Status | |---:|---|---| @@ -11,5 +11,5 @@ lakebed instead of their own Iceberg code, and for lakeglass to read Iceberg tab | 2 | [One crate per format](../0002-formats.md) | In progress | | 3 | [Apache Iceberg](../0003-iceberg.md) | In progress | -**Done when:** the crates are on crates.io ([#7](https://github.com/polythings/lakebed/issues/7)), Timelake depends +**Done when:** the crates are on crates.io ([#7](https://github.com/polythings/lakekit/issues/7)), Timelake depends on them by version, and lakeglass's Iceberg reader is built on them. From 44a577d4746dbd1e93d2b09451a09d5ece397425 Mon Sep 17 00:00:00 2001 From: Oleksandr Litus Date: Tue, 20 Oct 2026 22:40:55 -0400 Subject: [PATCH 2/2] Changelog: the rename from lakebed Co-Authored-By: Claude Opus 5.5 --- CHANGELOG.md | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/CHANGELOG.md b/CHANGELOG.md index 3cb0344..b642f9d 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -18,4 +18,10 @@ versions follow [Semantic Versioning](https://semver.org/). decimals as Iceberg's shortest big-endian bytes). - `lakekit`: re-exports the format crates behind a feature each. +### Changed + +- The project is named lakekit, and its crates `lakekit-core`, `lakekit-iceberg`, and `lakekit`; it was lakebed + (`lakebed-core`, …) until 2026-10-21. Nothing was published under the old names; GitHub redirects the old + repository URL. + The Iceberg code started as Timelake's shared writer (`timelake-format`), made general.