Skip to content
markhaydenPublic

About

No description, website, or topics provided.

Resources

Stars

2 stars

Watchers

0 watching

Forks

Repository files navigation

squint

Because "looks good to me" doesn't scale.

squint is an agentic harness for evaluating the integrity of images. You give it an image to check, whatever it should be checked against (a source photo, a reference subject, a brief), and the rules that matter. A vision model works through the question with a set of tools, then returns a verdict you can check: pass, fail or unsure, what it found, where it is in the image, and every step it took to get there.

Why

Image models are good enough to use in real pipelines and still wrong often enough that someone has to look at every output. That someone is usually a person squinting at two images, or a single similarity number that cannot tell a new background from a redrawn logo. Neither scales, and neither explains itself.

Asking a vision model "does this look right?" gets you part of the way. It is good at deciding whether a change matters and bad at noticing small ones: a logo nudged a few pixels, a spot that went missing, a sixth finger in a crowd. squint gives the model tools for exactly that, and makes it show its work.

What it checks

Each of these is the same shape: something to compare against, rules, tools, a verdict.

  • Source fidelity. An edited, restyled or re-generated image kept what mattered in the original. Swap the background of a portrait and the face, pose and wardrobe stay as shot; regrade a landscape for a campaign and the horizon and landmarks keep their shape. Only what was meant to change, changed.
  • Reference consistency. A subject from a reference image shows up faithfully in a new scene. Put your dog on the bridge of a starship and it should still be your dog: same breed, same build, same spots.
  • Reality checks. Mistakes no real photo would make: extra or missing limbs, fused fingers, impossible shadows and reflections, garbled text, objects melting into each other.
  • Brief compliance. The image contains what was asked for and nothing it should not: the right objects in the right numbers, the right colours, text spelled correctly, no stray logos, watermarks or people.
  • Product fidelity. A product shot keeps the exact product: label text, logo, shape, colour and proportions, whatever the scene around it.

These are where it starts. The harness is meant for any image evaluation you can write down as rules and back with tools.

How it works

You describe a job: the candidate image, the images it should be judged against (a source photo, one or more reference subjects), the use case, and any requirements of your own in plain language. squint turns the rules for that use case and your requirements into a numbered list of obligations, each resolved to the image it judges, and builds a checklist the model must answer item by item.

A vision model then looks at the images with a fixed set of tools: zoom into a region, align a candidate to its source, and, as the harness grows, find changed regions, read text, or match a subject. Every tool is deterministic, describes itself, and returns evidence, never a verdict. Tool calls, turns and spend are all bounded.

The model ends by submitting a structured answer: a disposition for every checklist item (violation, held, allowed_change, hidden, cant_tell) with a reason and a location. Code computes the verdict from those dispositions and the obligations' kinds. A confident violation of a mandatory obligation is fail; uncertainty on something that matters is unsure; otherwise pass. A provider error, a tool crash or a blown budget is an error, never a verdict. The model's own opinion is recorded and ignored.

Every run writes a journal and a report with every image the model saw, every tool result and every location, viewable offline. Details: SPEC.md.

Usage

squint init writes a personal default config (~/.config/squint/squint.toml, or under $XDG_CONFIG_HOME) so no check needs --config; it holds no secret and prints the environment variables to set (the API key, and the account id the config attests to). --provider and --model pick another provider or model from the catalog (squint catalog list).

squint check --use-case reference-consistency --candidate scene.png --ref dog=dog.jpg

prints the verdict (pass, fail or unsure) alone on stdout, so scripts can read it, and on stderr the decision directory and a summary: every checklist item with the rule it stands for, the model's answer and its reason, any findings with where they are, why the verdict is what it is, what the check spent against its cap, and the squint report view <dir> command that opens the full report. --quiet prints only the verdict and the decision directory.

Say what matters for this image in plain words, without writing a file:

squint check --use-case reference-consistency --candidate scene.png --ref dog=dog.jpg \
  --must-hold "The dog's coat pattern is unchanged." \
  --must-not "A white patch is added to the dog." \
  --allow "The dog's pose may change."

Each becomes a job requirement (RQ1, RQ2, …) checked beside the use case's built-in rules; a confident violation of any of them fails the check.

squint rules list names the built-in rule sets and squint rules list reference-consistency prints one set's rules. To change them for good, squint rules init reference-consistency writes a starter squint.toml and a rules.toml with every built-in rule commented out: uncomment one to replace it (same id), add disabled = true to remove it, or add your own. Writing rules: docs/rules.md.

Before the first request a check reserves its worst case against the spend cap: every tool turn, the final turn and the repair at full output, plus a reserve for retries. That bound can be far above what a check spends; when it is over the cap the refusal shows how it was built, which price tier applies and why, and the settings that bring it down.

How it is built

The method is the brief's: measure first, then earn each tool.

  1. Baseline. Each candidate model alone, no tools, on public labelled sets for all five use cases.
  2. Equal visual access. The same sets with zoom and alignment available but no measurements, so a later tool is compared against a model that could already look closely.
  3. One tool at a time. A tool is admitted for a use case only if it beats that control on a sealed set by predeclared gates, confirmed on a second set. inconclusive is an allowed result.

Rust workspace, one binary, an optional Python tool pack over MCP, provider adapters written against recorded fixtures. The harness core (milestones M0 and M1) runs a job end to end against a scripted provider: rules, checklist, tool turns, repairs, ledger, journal, report, cache, and the MCP transport with a provisioned pack; the three direct provider adapters (Anthropic, Google AI Studio, OpenAI) are written against recorded conversations and smoked live from the release archives (milestone M2). Install with cargo install squint-cli or from a release archive, copy a config from examples/ and set the one environment variable it names.

What has been measured so far, every number with its interval under docs/results/ and the story in docs/results/findings.md:

  • Baseline (M3). Six models, no tools, ten public development sets. Every model caught nearly every planted defect (52 to 60 of 60); the differences were in false rejects, hedging and cost, not detection. Opus cost about five times the cheapest model per item for no better detection.
  • Equal visual access (M4). Five models under arms A and C, paired item by item. Looking closer did not change verdicts: 39 of 500 paired items flipped, almost all between pass and unsure on clean images. The run also showed that two tool interfaces were hard for models to call, which were fixed before the next run.
  • First gate (M5). The regions change finder against the visual-access control on a sealed blind set of 20 source-fidelity items under Gemini 4 Argon: rejected, because the control already detected all ten planted failures. The tool took back the one false reject and lifted localisation to 10 of 10, neither of which the frozen gate scores. The sets need to be harder before a detection gate can separate tools.
  • Second gate. The same comparison on a harder sealed set of 25 items in five new scenes, counted as a second attempt: rejected again. Both arms caught all 15 planted failures (16 once the owner corrected one label), and the tool changed no verdict. Removals a generator can plant cleanly are at ceiling for this model. regions stays an optional tool; whether it helps a cheaper model is tracked in issue #3.

squint is one tool in a chain an agent drives (generate, check with squint, mask with rascal, paint, check again), so it presents the same external interface as its siblings, defined by the shared contract table-manners: squint serve (MCP over stdio), squint describe --json, squint estimate --json, squint call <action> --arguments <file> (any action from exactly its MCP arguments; --bundle <path> replays an invocation bundle), and a run.json in every decision directory (ADR 0018, milestones M7 to M10). It conforms to table-manners v0.5.0, core and images profile (pinned =0.5.0; by path until the release is published): it describes itself and serves with no configuration, each action a task saying what it still needs (readiness); an estimate is a control bound to the exact call, never a run; every result records the call it resolved to (invocation), so it can be run again elsewhere or replayed from a bundle with the original files gone; image paths are read only inside the directories --allow-dir names (with none, images come inline); and its report renders (report view, report export) are ones a host can embed. squint playground opens Wafflery, the shared local workspace, on squint. How an agent or a pipeline uses it: docs/agents.md.

Plan: PLAN.md. Decisions: docs/adr/. Reviews of the spec, the plan and each milestone: docs/reviews/. Developer setup: docs/bootstrap.md.

Origins

squint grew out of Pixel, the image-artist agent for Bakin, where generating images turned out to be the easy half and knowing whether they were right was the hard one.

The project was called dejaview until 2026-10-29. The five library crates published to crates.io under that name (dejaview-core and its siblings, 0.1.0) are superseded by the squint-* crates and yanked; the results documents and receipts written before the rename keep the old name, because their hashes cover it.

License

Apache-2.0.

About

No description, website, or topics provided.

Resources

Stars

2 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages