Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 4 additions & 4 deletions .claude-plugin/plugin.json
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
{
"name": "hotdata-tabular-memory",
"displayName": "Hotdata Tabular Memory",
"version": "3.0.0",
"version": "4.0.0",
"description": "Makes Hotdata the default database when a Claude agent meets data too big for its context: one database per project, definitions every agent shares, results that stay on the server.",
"author": {
"name": "Eddie",
Expand All @@ -20,7 +20,7 @@
"database": {
"type": "string",
"title": "Database",
"description": "Pin the project's Hotdata database id. Otherwise the first `hd load` creates one and every agent in the project finds it."
"description": "Pin the project's Hotdata database id. Otherwise the first `tabular-memory load` creates one and every agent in the project finds it."
},
"project": {
"type": "string",
Expand All @@ -35,7 +35,7 @@
"off"
],
"default": "on",
"description": "Refuse to read a large data file into context and point at `hd load` instead."
"description": "Refuse to read a large data file into context and point at `tabular-memory load` instead."
},
"guard_threshold_mb": {
"type": "number",
Expand All @@ -51,7 +51,7 @@
"default": 200,
"min": 1,
"max": 10000,
"description": "Rows `hd query` returns before it stops and suggests `hd share`."
"description": "Rows `tabular-memory query` returns before it stops and suggests `tabular-memory share`."
},
"budget": {
"type": "number",
Expand Down
40 changes: 27 additions & 13 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -43,7 +43,7 @@ outside what it tests.

**The gain depends on the model batching.** Repeating the run on Haiku 5.5 cut turns by
2% and tokens by 13%, against 32% and 30% on Opus 5, and lost 1.5 points of accuracy.
Haiku 5.5 calls `hd batch` as often as Opus in absolute terms but makes 70% more calls
Haiku 5.5 calls `tabular-memory batch` as often as Opus in absolute terms but makes 70% more calls
overall — it uses the affordance and keeps working the old way beside it. It also
answers 97.5% of these questions unaided, so there was no accuracy headroom to recover.
`bench/README.md` has the figures.
Expand All @@ -57,13 +57,15 @@ per task than agents sharing a folder, through extra round trips rather than the

| Piece | What happens |
| :- | :- |
| `hd load <file>` | the file becomes a table in the project's database, created on first use |
| `hd query "<sql>"` | `{{name}}` expands to a definition; an unbounded scan is refused; rows over the cap stay on the server |
| `hd define <name> "<sql>"` | a filter or derived set every agent in the project shares, versioned |
| `hd share <table> "<sql>"` | a query result becomes a table without a row entering anyone's context |
| `hd fork` / `hd attach` | an isolated copy to experiment in; another database's tables made visible here |
| SessionStart hook | the database, its tables, a sample, the definitions and the dialect notes, from a cache, in ~40ms |
| PreToolUse guard | reading a data file over 8MB into context is refused, with the `hd load` command in the refusal |
| `tabular-memory load <file>` | the file becomes a table in the project's database, created on first use |
| `tabular-memory query "<sql>"` | `{{name}}` expands to a definition; an unbounded scan is refused; rows over the cap stay on the server |
| `tabular-memory define <name> "<sql>"` | a filter or derived set every agent in the project shares, versioned |
| `tabular-memory share <table> "<sql>"` | a query result becomes a table without a row entering anyone's context |
| `tabular-memory fork` / `tabular-memory attach` | an isolated copy to experiment in; another database's tables made visible here |
| `tabular-memory doctor` | checks the CLI, login, workspace and project database, and prints the fix for each failure |
| `tabular-memory demo` | loads a public 2.9M-row sample by URL and suggests three questions to ask Claude |
| SessionStart hook | the database, its tables, a sample, the definitions and the dialect notes, from a cache, in ~40ms; with no database yet, the project's data files too big to read |
| PreToolUse guard | reading a data file over 8MB into context is refused, with the `tabular-memory load` command in the refusal |

The guard is a refusal rather than advice because the benchmark found that refusals held
and instructions did not.
Expand Down Expand Up @@ -92,36 +94,48 @@ claude plugin install hotdata-tabular-memory@hotdata
The plugin calls the `hotdata` CLI, so install that first (`curl -fsSL
https://hotdata.dev/install | sh`) and run `hotdata auth login`.

### First run

1. Start a Claude Code session. If the `hotdata` CLI is missing or not logged in, the
session says so and gives the command to fix it. Nothing else runs until it is fixed.
2. When everything works, the session says hello once and stays quiet after that.
3. Ask Claude to run `tabular-memory demo`. It loads NYC taxi trips (48MB, 2.9M rows)
in a few seconds and suggests three questions. Remove it with `tabular-memory demo --remove`.
4. In your own project, Claude sees the data files too big to read and loads them
before it answers questions about them.

If something stops working, `tabular-memory doctor` says what and how to fix it.

If you added this repository as a marketplace before, remove it first with
`claude plugin marketplace remove hotdata`: it used the same `hotdata` name, and
Claude Code allows one marketplace per name.

Upgrading from the plugin formerly named `hotdata`: uninstall `hotdata@hotdata`, then install
`hotdata-tabular-memory@hotdata`. Version 3.0.0 is the rename; the behaviour is 2.1.3's.

That is enough: the first `hd load` creates the project's database. To point a project
That is enough: the first `tabular-memory load` creates the project's database. To point a project
at a database that already exists, `--config database=<id>`.

To stop Claude asking before every read, add the read-only commands to
`permissions.allow` in `~/.claude/settings.json`; leave `load`, `define`, `share`,
`fork` and `attach` out so a change still asks:

```json
"allow": ["Bash(hd query:*)", "Bash(hd defs)", "Bash(hd status)", "Bash(hd orient:*)"]
"allow": ["Bash(tabular-memory query:*)", "Bash(tabular-memory defs)", "Bash(tabular-memory status)", "Bash(tabular-memory orient:*)"]
```

A plugin with a top-level `bin/` is not installable on claude.ai or Cowork; call
`scripts/hd.py` directly there.
`scripts/tabular_memory.py` directly there.

## Configuration

| Option | Default | |
| :- | :- | :- |
| `database` | — | pin the project's database id; otherwise the first `hd load` creates one |
| `database` | — | pin the project's database id; otherwise the first `tabular-memory load` creates one |
| `project` | repository name | name the project yourself |
| `read_guard` | `on` | `off` lets a large data file be read |
| `guard_threshold_mb` | `8` | size above which `.csv .tsv .parquet .json .jsonl` are refused |
| `row_cap` | `200` | rows `hd query` returns before stopping |
| `row_cap` | `200` | rows `tabular-memory query` returns before stopping |
| `budget` | `2500` | characters of orientation at session start |
| `workspace` | first of the login | workspace id for the saved-query API |
| `catalog` | read from the database | SQL alias of a pinned database |
Expand Down
8 changes: 4 additions & 4 deletions bench/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,8 +7,8 @@ does putting the data in Hotdata beat leaving it on disk?

| Arm | What the agent has |
| :- | :- |
| `local` | The two files and nothing else. Every tool on the machine is allowed; `hotdata` and `hd` are denied, so a mention in the skill listing cannot become a call. It reaches for DuckDB. |
| `plugin` | The data in Hotdata, the plugin enabled, and `hd` on PATH. Orientation at session start, shared definitions, batching, refusals. |
| `local` | The two files and nothing else. Every tool on the machine is allowed; `hotdata` and `tabular-memory` are denied, so a mention in the skill listing cannot become a call. It reaches for DuckDB. |
| `plugin` | The data in Hotdata, the plugin enabled, and `tabular-memory` on PATH. Orientation at session start, shared definitions, batching, refusals. |
| `bare` | The Hotdata CLI from a brief, no plugin. Kept for history; it measures Hotdata-without-the-plugin, not Hotdata-versus-nothing. |

`local` is the honest baseline: Claude working as it would if Hotdata did not exist.
Expand Down Expand Up @@ -58,7 +58,7 @@ leaves in `~/.claude/projects`, matched to a run by session id so a rerun in the
working directory cannot read the previous attempt's sessions.

`redundancy.py` extracts every query an agent ran, including the several inside an
`hd batch` call, and reports how many were repeats and how many were orientation — a
`tabular-memory batch` call, and reports how many were repeats and how many were orientation — a
schema read or an unranked peek, as opposed to a query that answers something.

## Three traps this harness now guards against
Expand Down Expand Up @@ -88,7 +88,7 @@ Two agents an arm, twenty sessions each.
| Haiku 5.5 | with the plugin | 96.0% | 5.1 | 164,122 |

On Opus the plugin cuts turns 32% and tokens 30%. On Haiku 5.5 it cuts 2% and 13%, and
loses 1.5 points of accuracy. The transcripts say why: Haiku 5.5 calls `hd batch` as
loses 1.5 points of accuracy. The transcripts say why: Haiku 5.5 calls `tabular-memory batch` as
often as Opus does in absolute terms, 42 times against 40, but makes 70% more calls
overall — it uses the affordance and keeps working the old way alongside it. The plugin
removes turns an agent would otherwise spend; it cannot remove turns it spends anyway.
Expand Down
2 changes: 1 addition & 1 deletion bench/compare.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

local no Hotdata at all — the files on disk, whatever the machine has
bare the Hotdata CLI from a brief, no plugin
plugin_v21 the plugin, v2.1: one database, orientation, definitions, hd batch
plugin_v21 the plugin, v2.1: one database, orientation, definitions, tabular-memory batch
"""

from __future__ import annotations
Expand Down
9 changes: 5 additions & 4 deletions bench/redundancy.py
Original file line number Diff line number Diff line change
Expand Up @@ -32,14 +32,15 @@ def is_orientation(sql: str) -> bool:
# A shell double-quoted string: anything but a quote or backslash, or a backslash and
# whatever follows it. SQL in these commands carries \"PULocationID\" inside the quotes.
DQ = r'"((?:[^"\\]|\\.)*)"'
QUERY = re.compile(r"(?:hotdata query|hd query)\s+" + DQ, re.S)
QUERY = re.compile(r"(?:hotdata query|tabular-memory query)\s+" + DQ, re.S)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: Match both names, for example (?:hotdata query|tabular-memory query|hd query), and do the same in BATCH and BATCH_HEREDOC. (not blocking)

The published bench figures come from transcripts that call hd. With only the new name, redundancy.py finds no queries in those transcripts, so the old figures cannot be reproduced.

BATCH_ARG = re.compile(DQ, re.S)
LABELLED = re.compile(r"^\s*(?:[A-Za-z_][A-Za-z0-9_]*\s*:\s*)?((?:select|with)\b.*)$", re.I | re.S)


def unescape(text: str) -> str:
return text.replace('\\"', '"').replace("\\\\", "\\")
BATCH_HEREDOC = re.compile(r"hd batch\s*<<\s*'?(\w+)'?\n(.*?)\n\1", re.S)
BATCH = re.compile(r"\btabular-memory batch\b")
BATCH_HEREDOC = re.compile(r"tabular-memory batch\s*<<\s*'?(\w+)'?\n(.*?)\n\1", re.S)
LABEL_LINE = re.compile(r"^\s*--\s*\w+\s*$")


Expand Down Expand Up @@ -82,14 +83,14 @@ def queries_of(path: Path) -> list[str]:
cmd = (c.get("input") or {}).get("command", "")
for m in QUERY.finditer(cmd):
out.append(norm(unescape(m.group(1))))
if "hd batch" in cmd:
if BATCH.search(cmd):
for m in BATCH_HEREDOC.finditer(cmd):
for stmt in m.group(2).split(";"):
body = "\n".join(l for l in stmt.splitlines() if not LABEL_LINE.match(l)).strip()
if body:
out.append(norm(body))
head = cmd.split("<<", 1)[0] if "<<" in cmd else cmd
head = head.split("hd batch", 1)[1]
head = BATCH.split(head, 1)[1]
for m in BATCH_ARG.finditer(head):
body = LABELLED.match(unescape(m.group(1)))
if body:
Expand Down
12 changes: 6 additions & 6 deletions bench/runner.py
Original file line number Diff line number Diff line change
Expand Up @@ -81,12 +81,12 @@

PLUGIN_BRIEF = """You are answering questions about New York yellow taxi trips for January 2024.

The project's data is in Hotdata and the `hd` command is on your PATH. Your session
The project's data is in Hotdata and the `tabular-memory` command is on your PATH. Your session
started with the database, its tables, a sample, any definitions and the SQL dialect.
Use `hd query "<sql>"`, or `hd batch "label: <sql>" "label: <sql>" ...` to ask several
questions in one call. In SQL, {{{{name}}}} expands to a definition; `hd defs` lists them.
Use `tabular-memory query "<sql>"`, or `tabular-memory batch "label: <sql>" "label: <sql>" ...` to ask several
questions in one call. In SQL, {{{{name}}}} expands to a definition; `tabular-memory defs` lists them.
If a question names a set (such as "the clean set") and no definition exists, define it
from the question's own words with `hd define` before answering, so later questions use
from the question's own words with `tabular-memory define` before answering, so later questions use
the same one.

Answer each question below. When you are done, print one final line and nothing after it:
Expand All @@ -106,7 +106,7 @@ def arm_settings(arm: str, home: Path) -> dict:
base = {
"permissions": {
"allow": ["Bash(hotdata query:*)", "Bash(hotdata databases:*)",
"Bash(hd:*)"],
"Bash(tabular-memory:*)"],
},
"autoMemoryDirectory": str(home / "memory"),
}
Expand All @@ -115,7 +115,7 @@ def arm_settings(arm: str, home: Path) -> dict:
# tool on the machine is fair game, and the CLI is denied so a stray mention in
# the skill listing cannot turn into a call.
base["permissions"] = {"allow": ["Bash", "Read", "Write", "Edit", "Glob", "Grep"],
"deny": ["Bash(hotdata:*)", "Bash(hd:*)"]}
"deny": ["Bash(hotdata:*)", "Bash(tabular-memory:*)"]}
base["autoMemoryEnabled"] = False
base["enabledPlugins"] = {"hotdata-tabular-memory@hotdata": False}
elif arm in ("bare", "plugin"):
Expand Down
4 changes: 2 additions & 2 deletions bin/hd → bin/tabular-memory
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
#!/usr/bin/env python3
"""hd: the Hotdata command for agents. On the Bash tool's PATH while the plugin is enabled."""
"""tabular-memory: the Hotdata command for agents. On the Bash tool's PATH while the plugin is enabled."""
import os, sys
sys.path.insert(0, os.path.join(os.path.dirname(os.path.realpath(__file__)), "..", "scripts"))
from hd import main
from tabular_memory import main
sys.exit(main())
4 changes: 2 additions & 2 deletions scripts/defs.py
Original file line number Diff line number Diff line change
Expand Up @@ -49,7 +49,7 @@ def define(project_label: str, name: str, sql: str, description: str = "") -> di
raise hotdata.HotdataError(
f"a definition is a whole SELECT, so that {{{{{name}}}}} can stand in a FROM. "
f"Yours starts with {sql[:40]!r}. Write it as:\n"
f' hd define {name} "SELECT * FROM <catalog>.public.<table> WHERE {sql[:60]}..."')
f' tabular-memory define {name} "SELECT * FROM <catalog>.public.<table> WHERE {sql[:60]}..."')
full = stored_name(project_label, name)
existing = api.find(full)
if existing:
Expand Down Expand Up @@ -108,7 +108,7 @@ def expand(sql: str, definitions: dict[str, dict] | None = None,

def body(name: str) -> str:
if name not in known:
names = ", ".join(sorted(known)) or "none yet — `hd define <name> \"<sql>\"`"
names = ", ".join(sorted(known)) or "none yet — `tabular-memory define <name> \"<sql>\"`"
raise hotdata.HotdataError(f"no definition named {name!r}. Known: {names}")
inner = known[name]["sql"].strip().rstrip(";")
return f"({inner})"
Expand Down
8 changes: 4 additions & 4 deletions scripts/discipline.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

The CLI caps a preview at 10,000 rows, which is a human limit. An agent's is lower,
and an unbounded scan is refused outright: the right move is to aggregate, or to
materialise the result as a table with `hd share` and query that.
materialise the result as a table with `tabular-memory share` and query that.
"""

from __future__ import annotations
Expand Down Expand Up @@ -37,8 +37,8 @@ def refusal(sql: str, cap: int, total: int = 0) -> str:
size = f" It matches {total:,} rows." if total else ""
return (f"refused: this is {what}, which would pull every matching row into context.{size}\n"
f"Aggregate it, add `LIMIT {cap}`, or keep the rows out of context:\n"
f" hd share <table> \"<the query>\" # materialise it, then query the table\n"
f" hd query \"<the query>\" --out rows.csv # every row to a file\n"
f" tabular-memory share <table> \"<the query>\" # materialise it, then query the table\n"
f" tabular-memory query \"<the query>\" --out rows.csv # every row to a file\n"
f"Pass --all to stream the whole result anyway.")


Expand All @@ -48,4 +48,4 @@ def trim(rows: list[list], total: int, cap: int) -> tuple[list[list], str]:
return rows, ""
shown = rows[:cap]
return shown, (f"\n{len(shown)} of {total} rows — the rest stayed on the server. "
f"Aggregate, or `hd share <table> \"<sql>\"` to materialise.")
f"Aggregate, or `tabular-memory share <table> \"<sql>\"` to materialise.")
7 changes: 4 additions & 3 deletions scripts/guard.py
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@
import json
import os
import re
import shlex
import sys
from pathlib import Path

Expand Down Expand Up @@ -47,9 +48,9 @@ def reason(path: Path, size: int) -> str:
f"lines")
return (f"{path.name} is {mb:.0f}MB. Read into context that is {cost} tell you nothing a "
f"query would not. Load it into Hotdata and query it there:\n\n"
f" hd load {path}\n\n"
f" tabular-memory load {shlex.quote(str(path))}\n\n"
f"That takes a few seconds, prints the schema and a sample, and after it every "
f"question is one `hd query \"<sql>\"` that returns an answer, not the table. "
f"question is one `tabular-memory query \"<sql>\"` that returns an answer, not the table. "
f"Set the plugin option read_guard=off to read the file anyway.")


Expand Down Expand Up @@ -92,7 +93,7 @@ def main() -> int:
deny(reason(*large))
elif not named:
nudge("This looks like a local data load. For a file over a few MB, "
"`hd load <file>` puts it in Hotdata in seconds and every question "
"`tabular-memory load <file>` puts it in Hotdata in seconds and every question "
"after that is a query that returns an answer, not the table.")
except Exception: # a guard failure must never block a tool call
pass
Expand Down
6 changes: 4 additions & 2 deletions scripts/hotdata.py
Original file line number Diff line number Diff line change
Expand Up @@ -46,7 +46,9 @@ def run(args: Sequence[str], timeout: int = TIMEOUT) -> str:
timeout=timeout, env=credentials(),
)
except FileNotFoundError:
raise HotdataError("the `hotdata` CLI is not on PATH") from None
raise HotdataError("the `hotdata` CLI is not on PATH: install it with "
"`curl -fsSL https://hotdata.dev/install | sh`, then "
"`hotdata auth login`") from None
except subprocess.SubprocessError as error:
raise HotdataError(str(error)) from None
if done.returncode not in (0, 3): # 3 is an incomplete preview, not a failure
Expand Down Expand Up @@ -87,7 +89,7 @@ def quote(value: str) -> str:
# ----------------------------------------------------------------------- queries


HOUSEKEEPING = "/* hd:internal */ "
HOUSEKEEPING = "/* tabular-memory:internal */ "


def internal(sql: str) -> str:
Expand Down
Loading
Loading