From 031189782e10247b105266348b252b5c8b35a41d Mon Sep 17 00:00:00 2001 From: Peter Corke Date: Tue, 13 Oct 2026 11:06:12 +1000 Subject: [PATCH] docs: llms.txt and llms-full.txt, and an updated AGENTS.md llms.txt (llmstxt.org) is a short introduction for AI assistants: what pgraph is, a quick start, key points and common mistakes, then links; llms-full.txt is the whole Sphinx site as plain text, API reference included. Both are built by docs/llms_txt.py from the hand-written docs/llms_intro.md and the nested toctrees (make html llms), and published with the site by the docs job. The introduction was checked claim by claim against the code. AGENTS.md follows ansitable and bdsim: it points assistants that are using pgraph to llms.txt, and records what an agent working on the source needs to know: the layout, the records that must agree, the two staleness flags, the hot Vertex.__setattr__, pickling, the tests and scanners, measuring performance against 0.6.5, and how a release is made. Co-Authored-By: Claude Sonnet 5.5 --- .github/workflows/ci.yml | 2 +- AGENTS.md | 19 ++++- README.md | 2 +- docs/Makefile | 6 +- docs/llms_intro.md | 43 ++++++++++ docs/llms_txt.py | 170 +++++++++++++++++++++++++++++++++++++++ 6 files changed, 235 insertions(+), 7 deletions(-) create mode 100644 docs/llms_intro.md create mode 100644 docs/llms_txt.py diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index e3b46ecf..4718bd4d 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -99,7 +99,7 @@ jobs: sudo apt-get install -y graphviz - name: Build docs run: | - make -C docs html + make -C docs html llms touch docs/build/html/.nojekyll - name: Deploy docs to gh-pages uses: peaceiris/actions-gh-pages@v4 diff --git a/AGENTS.md b/AGENTS.md index dd862621..0082366a 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -1,5 +1,7 @@ # pgraph-python — Agent Instructions +> **Audience:** agents working on pgraph's source code. If you're helping someone *use* pgraph, read https://petercorke.github.io/pgraph-python/llms.txt instead (or `llms-full.txt` for the complete documentation). + Part of the RVC ecosystem. **Read [rvc-ecosystem/AGENTS.md](https://github.com/petercorke/rvc-ecosystem/blob/main/AGENTS.md) first** — it defines shared conventions: repo ownership, math invariants, dependency boundaries, git/PR workflow, code standards, tech-debt tracking. This file only adds what's specific to this repo. | | | @@ -8,10 +10,19 @@ Part of the RVC ecosystem. **Read [rvc-ecosystem/AGENTS.md](https://github.com/p | Nickname | pgraph-python | | Owner | Peter Corke (`petercorke`) | | Default branch | `main` | -| Contribution model | Branch → PR; direct push to `main` at Peter's discretion | +| Contribution model | Branch → PR; `main` is protected: CI must pass and the branch must be up to date (`gh pr update-branch`) | ## Notes specific to this repo -- Minimal graph/path-planning utility package — no internal ecosystem dependencies. -- Used by `robotics-toolbox-python` (path/roadmap planning) and `machinevision-toolbox-python`. -- Pure Python — builds with Hatch (`hatchling`) directly. +- Minimal graph/path-planning package. Pure Python, builds with Hatch (`hatchling`); Python 3.9+. Depends on `numpy`, `matplotlib` and `spatialmath-python`; `ansitable` and `scipy` arrive through spatialmath. **Do not add scipy to `pyproject.toml`**: it is imported lazily, only inside `Graph.build_kdtree()`. +- Used, unpinned (`"pgraph-python"`), by `robotics-toolbox-python` (`PRMPlanner`, `RRTPlanner`, `LatticePlanner`, `PoseGraph`, which subclass `UVertex`/`DVertex`) and `machinevision-toolbox-python` (`BundleAdjust`). A breaking change needs a deprecation cycle with a working fallback (`warnings.warn(..., DeprecationWarning, stacklevel=2)`); run their pgraph-using tests before a release. +- **Layout.** `src/pgraph/PGraph.py` holds `Graph`, `UGraph`, `DGraph`, `Vertex`, `Edge` and `SE2Metric`; `exporters.py` the `export()` formats. There is one `Vertex` class: a vertex has no direction, the graph decides how an edge is recorded (`UGraph`/`DGraph` implement the hooks `_attach`, `_detach`, `_incoming` and `_endpoints_match`). `UVertex`/`DVertex` are deprecated aliases, by module `__getattr__`. +- **Records that must agree.** A vertex has `_edgelist` (leaving, or incident if undirected) and `_inlist` (entering, directed only); the graph has `_vertexlist`, `_vertexdict` (names) and `_edges` (an insertion-ordered dict). `g.validate()` checks them all; use it in tests of anything that edits the structure. +- **Two staleness flags, consumed independently.** `_connectivitychange` (components, reset by `_graphcolor`) and `_spatialchange` (the k-d tree). Set both wherever a vertex is added or removed; only connectivity cares about edges. Never share one flag. +- **`Vertex.__setattr__` is hot.** It exists to tell the graph that `coord` was assigned. Anything run per vertex (`Vertex.__init__`, `add_vertex`, components) writes `vertex.__dict__` directly to skip it; keep doing that, and declare new attributes at class level so mypy knows them. +- **Pickle and `copy` use a flat description** (`__getstate__`/`__setstate__`: vertices, edges by index, replayed through `_attach`); `subgraph` reuses it. Derived or transient state on a graph (the k-d tree) must be excluded there. A bare vertex or edge refuses to be pickled or copied. +- **Tests** are `unittest` classes in `tests/`, run with `pytest`; write expected values by hand or from an oracle in the test, never from the code under test. Import names explicitly (a star import is a Codacy `F405`); route `pickle` through one helper marked `# nosec B301 nosemgrep`. Codecov gates patch coverage, so every changed line needs a test. `mypy src` is a CI check. +- **Performance claims are measured.** `examples/benchmark.py` (`--md`, `--directed`) produces the README tables; regenerate them after a performance change and compare with `v0.6.5` (`git archive v0.6.5 src`), not with a development build. +- **Docs** are Sphinx (`make -C docs html`; `-n` for nitpicky) with `sphinx-pyrunblock` `.. runblock:: pycon` examples that execute at build. `docs/llms_intro.md` is the hand-written part of `llms.txt`: update it when the API changes; `docs/llms_txt.py` builds `llms.txt` and `llms-full.txt` from it and the toctrees (`make html llms`), and CI publishes them with the site. +- **Release.** Version is in `pyproject.toml`, history in `CHANGELOG.md`. Publishing is `.github/workflows/publish.yml` on a GitHub Release (trusted publishing, environment `pypi`): it refuses unless the tag equals the version and the commit is on `main`. Don't tag or upload by hand. +- `examples/london.json` is OpenStreetMap data (© OpenStreetMap contributors, ODbL); keep the attribution with it. diff --git a/README.md b/README.md index 436c9e07..d0ff4260 100644 --- a/README.md +++ b/README.md @@ -91,7 +91,7 @@ Map data © OpenStreetMap contributors, available under the [Open Database Licen ## Using pgraph -Full details are given in the [online documentation](https://petercorke.github.io/pgraph-python). +Full details are given in the [online documentation](https://petercorke.github.io/pgraph-python). For an AI assistant there is a short summary, [llms.txt](https://petercorke.github.io/pgraph-python/llms.txt), and the complete documentation as one file, [llms-full.txt](https://petercorke.github.io/pgraph-python/llms-full.txt). ### Properties and methods of the graph diff --git a/docs/Makefile b/docs/Makefile index d0c3cbf1..8abe31bd 100644 --- a/docs/Makefile +++ b/docs/Makefile @@ -12,7 +12,11 @@ BUILDDIR = build help: @$(SPHINXBUILD) -M help "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O) -.PHONY: help Makefile +# llms.txt and llms-full.txt for AI assistants, written into build/html; run after html +llms: + SPHINXBUILD="$(SPHINXBUILD)" python llms_txt.py + +.PHONY: help Makefile llms # Catch-all target: route all unknown targets to Sphinx using the new # "make mode" option. $(O) is meant as a shortcut for $(SPHINXOPTS). diff --git a/docs/llms_intro.md b/docs/llms_intro.md new file mode 100644 index 00000000..8969ac8b --- /dev/null +++ b/docs/llms_intro.md @@ -0,0 +1,43 @@ +pgraph is a pure-Python package of graph classes: directed and undirected graphs of vertices connected by edges, with edge costs and path planning (breadth-first, uniform-cost and A*). Vertices can be *embedded*, given a coordinate in a space of any dimension; edge costs then come from a distance metric (L2, L1, SE2 or your own), and the graph can be queried spatially, plotted and exported. Self-loops and parallel edges are allowed. It requires Python 3.9 or later and depends on NumPy, Matplotlib and spatialmath-python. Install with `pip install pgraph-python`; import it as `pgraph`. + +## Quick start + +```python +from pgraph import UGraph + +g = UGraph(dim=2) # DGraph for directed; Graph(directed=flag) chooses +a = g.add_vertex([0, 0], name="a") # returns the Vertex +b = g.add_vertex([3, 4], name="b") +c = g.add_vertex([6, 0], name="c") +g.add_edge(a, b) # cost defaults to the metric distance, 5.0 +g.add_edge("b", "c", cost=7) # vertices or their names; or give the cost + +path, length, _ = g.path_Astar("a", "c") # None if there is no path +print([v.name for v in path], length) # ['a', 'b', 'c'] 12.0 +g.plot() +g.highlight_path(path) +``` + +## Key points + +- **Classes.** `UGraph` (undirected) and `DGraph` (directed) share the base class `Graph`, which is concrete: `Graph(directed=True)` is a `DGraph`. There is one `Vertex` class for both and one `Edge` class. `UVertex` and `DVertex` are deprecated aliases of `Vertex`. Constructor arguments are keyword-only: `UGraph(dim=2, metric="L2", heuristic=None, vertex_class=None, edge_class=None, verbose=False)`; `UGraph(2)` is a `TypeError`. +- **Vertices.** `g.add_vertex(coord=None, name=None)` returns the vertex. Default names are `"#0"`, `"#1"`, ..., names are unique, and `g["a"]` or `g[0]` finds a vertex. `for v in g` iterates in the order added; `g.n`, `g.ne` and `g.nc` are the numbers of vertices, edges and components. `g.remove_vertex(v)` and `g.remove_edge(e)` remove them, with their edges. +- **Edges.** `g.add_edge(v1, v2, cost=None)`; with no `cost` it is the metric distance between the coordinates, and `None` if the vertices have none (planning then raises `ValueError`). `g.edges()` and `v.edges()` return new lists in the order added. `e.v1` and `e.v2` are read-only. `v.adjacent()` is the neighbours, `v.successors()` and `v.predecessors()` are the directed ones, `v.edgeto(w)` the cheapest edge to `w` and `v.edges_to(w)` all of them. A self-loop of an undirected vertex is in `v.edges()` twice and counts 2 in its degree. +- **Directed graphs.** `degree` is in plus out (`in_degree`, `out_degree`); `nc` and `component` are weakly connected, ignoring direction. +- **Planning.** `path_BFS(S, G)` returns `(path, length)` with the fewest edges; `path_UCS(S, G)` and `path_Astar(S, G)` return `(path, length, parents)` of least cost. `S` and `G` are vertices or names, and all three return `None` when there is no path, so check before unpacking. +- **Metrics.** `"L2"` (default), `"L1"`, `"SE2"` for `(x, y, theta)` poses, `SE2Metric(weight)` to weight the angle, or a function of one difference vector. A* uses `heuristic` (default: the metric), which must be in the same units as the edge costs and must not exceed the true cost to go, or the path is not the shortest. +- **Spatial queries** on an embedded graph: `closest(point)` returns `(vertex, distance)`, `near(point, radius)`, `within(box)` with `box` a 2 x n array of lower and upper corner, `coordinates()` the `(dim, n)` matrix of coordinates, and `subgraph(vertices)`, which takes vertices, names or a boolean mask and returns a new graph. `build_kdtree()` is an optional index (it needs scipy) that `closest` and `near` then use. +- **Components and matrices.** `g.component(c)` is the vertices in component number `c`, `v.label` is the component of a vertex, `g.samecomponent(v1, v2)`. `adjacency()`, `incidence()`, `laplacian()`, `degree()` and `connectivity()` return dense NumPy arrays. +- **Output.** `g.plot()` (arrowheads for a directed graph, arcs for parallel edges, circles for loops), `highlight_path(path)`, and `g.export(file)` for GraphViz dot, Mermaid, GraphML, ELK JSON or an image (images need the GraphViz executables). +- **Subclassing.** `class Foo(Vertex)` and `UGraph(vertex_class=Foo, edge_class=Bar)` make the graph build them; a subclass `__init__` must pass `coord` and `name` (vertices) or `v1, v2, cost, data` (edges) to its parent. +- **Copying.** `g.copy()` and `pickle` work for a whole graph; a vertex or edge cannot be copied or pickled on its own, and a graph whose metric is a `lambda` cannot be pickled. + +## Common mistakes + +- `UGraph(6)` raises `TypeError`: write `UGraph(dim=6)`. +- Unpacking the result of a planner without checking for `None`. +- Coordinates in one unit and costs in another (metres against kilometres): the A* heuristic is then too large and the path found is not the shortest. +- `g.component(v)` takes a component *number*; use `v.label` for the component of a vertex. +- Assigning `e.v1 = ...` fails, they are read-only; `add_vertex` raises `ValueError` for a name already used or a vertex that is already in a graph. +- After `build_kdtree()`, change a coordinate by assigning (`v.coord = x`, `v.coord += x`), not in place (`v.coord[0] = x`), which the index cannot see. +- `g.edges()` is a copy: changing the list does not change the graph. diff --git a/docs/llms_txt.py b/docs/llms_txt.py new file mode 100644 index 00000000..5a28a5d7 --- /dev/null +++ b/docs/llms_txt.py @@ -0,0 +1,170 @@ +"""Write llms.txt and llms-full.txt for the documentation site. + +``llms.txt`` (see https://llmstxt.org) is a short Markdown file that tells AI assistants +what the package is, how to use it, and where its documentation lives. ``llms-full.txt`` +is the whole documentation as plain text, so an assistant can read it in a single fetch. + +The package is small, so ``llms.txt`` is mostly prose: the hand-written ``llms_intro.md`` +(what it does, a quick start, key points, common mistakes) followed by a generated list +of links. ``llms-full.txt`` is this Sphinx site rendered by Sphinx's ``text`` builder, so +the autodoc API reference is included, in the order of the toctrees, which are followed +from ``index.rst`` through the nested pages. Run after the HTML build:: + + make html llms # from docs/ + +The files are written into ``build/html`` so they are published with the site. +""" + +from __future__ import annotations + +import os +import re +import shutil + +# Used only to re-invoke `sphinx-build`, resolved via shutil.which() below; this is a +# local/CI docs-build script, not code that runs against untrusted input. +import subprocess # nosec B404 +import sys +from importlib.metadata import metadata +from pathlib import Path + +DOCS = Path(__file__).resolve().parent +SOURCE = DOCS / "source" +BUILD = DOCS / "build" +TEXT = BUILD / "text" +HTML = BUILD / "html" +INTRO = DOCS / "llms_intro.md" + +# One-line descriptions of the pages listed in the toctrees +PAGE_NOTES = { + "index": "overview", + "intro": "tutorial: building a graph, vertices, edges, plotting, path planning, directed graphs", + "api": "the API, in three parts: the common methods, vertices and edges, and the policy page", + "common": "the methods that UGraph and DGraph share, by task", + "construction": "UGraph, DGraph, Graph, constructors from a dict or an adjacency matrix, distance metrics", + "modification": "adding and removing vertices and edges", + "analysis": "counts, adjacency, degree, components, coordinates, spatial queries, subgraphs", + "representations": "adjacency, incidence, degree and Laplacian matrices", + "visualization": "plot(), its options, highlighting vertices, edges and paths", + "export": "writing a graph as GraphViz dot, Mermaid, GraphML, ELK or an image", + "planning": "path_BFS, path_UCS and path_Astar", + "edges_vertices": "the data structures: Vertex and Edge, and the edge lists of a vertex", + "policy": "object lifecycle: what may be done with a vertex or edge outside a graph", +} + +# Links that belong in llms.txt but are not pages of this site +EXTRA_LINKS = [ + ("Source code", "https://github.com/petercorke/pgraph-python", "GitHub repository"), + ("PyPI", "https://pypi.org/project/pgraph-python/", "install with pip install pgraph-python"), + ("Changelog", "https://github.com/petercorke/pgraph-python/blob/main/CHANGELOG.md", "release history and how to migrate from 0.6"), + ("Examples", "https://github.com/petercorke/pgraph-python/tree/main/examples", "Queensland and central London route-planning notebooks, and a benchmark"), + ("RVC ecosystem", "https://github.com/petercorke/rvc-ecosystem", "how pgraph relates to the Robotics and Machine Vision Toolboxes"), + ("Contributing", "https://github.com/petercorke/pgraph-python/blob/main/AGENTS.md", "repo conventions and workflow, for agents working on pgraph's source code"), +] + + +def toctree_entries(document: Path) -> list[str]: + """Document names listed in the toctrees of one document, in order. + + :param document: an ``.rst`` file + :return: document names such as ``"common"`` + """ + entries, in_toctree = [], False + for line in document.read_text().splitlines(): + if line.startswith(".. toctree::"): + in_toctree = True + elif in_toctree and line.strip() and not line.startswith((" ", "\t")): + in_toctree = False # the directive's indented body has ended + elif in_toctree and line.strip() and not line.strip().startswith(":"): + entries.append(line.strip()) + return entries + + +def toctree_pages(index: Path) -> list[str]: + """All the documents below a master document, depth first, in toctree order. + + The pages of this site are nested: ``index`` lists ``api``, which lists + ``common``, which lists the pages of the common methods. + + :param index: the master document, ``index.rst`` + :return: document names such as ``"construction"``, each once + """ + pages: list[str] = [] + + def walk(document: Path) -> None: + for name in toctree_entries(document): + if name not in pages: + pages.append(name) + child = document.parent / f"{name}.rst" + if child.exists(): + walk(child) + + walk(index) + return pages + + +def title(text: str) -> str: + """The page title: the first non-blank line that isn't an overline of punctuation.""" + for line in text.splitlines(): + if line.strip() and not re.fullmatch(r"[*=\-~#^]+", line.strip()): + return line.strip() + return "" + + +def main() -> None: + # Read the name, summary and documentation URL from the installed package's metadata + # (built from pyproject.toml). Reading pyproject.toml directly would need tomllib, + # which is Python 3.11+, and the Pages workflow builds the docs on 3.9. + meta = metadata("pgraph-python") + urls = dict(u.split(", ", 1) for u in meta.get_all("Project-URL")) + base = urls["Documentation"].rstrip("/") + "/" + name, summary = meta["Name"], meta["Summary"] + + # SPHINXBUILD matches the override convention of docs/Makefile's + # `SPHINXBUILD ?= sphinx-build` -- resolved via shutil.which() to a full path. + sphinxbuild_name = os.environ.get("SPHINXBUILD", "sphinx-build") + sphinxbuild = shutil.which(sphinxbuild_name) + if sphinxbuild is None: + raise SystemExit(f"sphinx-build not found: {sphinxbuild_name!r}") + subprocess.run( # nosec B603 -- sphinxbuild resolved via shutil.which() above # nosemgrep: python.lang.security.audit.dangerous-subprocess-use-audit.dangerous-subprocess-use-audit + [sphinxbuild, "-q", "-b", "text", str(SOURCE), str(TEXT)], # nosemgrep: python.lang.security.audit.dangerous-subprocess-use-tainted-env-args.dangerous-subprocess-use-tainted-env-args + check=True, + ) + pages = ["index"] + toctree_pages(SOURCE / "index.rst") + texts = {p: (TEXT / f"{p}.txt").read_text() for p in pages} + + # llms.txt: prose introduction, then links + index = [ + f"# {name}", + "", + f"> {summary}.", + "", + INTRO.read_text().strip(), + "", + "## Documentation", + "", + f"The whole documentation, including the API reference, is available as a single text file: [llms-full.txt]({base}llms-full.txt).", + "", + *(f"- [{title(texts[p])}]({base}{p}.html): {PAGE_NOTES.get(p, '')}" for p in pages), + "", + "## Optional", + "", + *(f"- [{label}]({url}): {note}" for label, url, note in EXTRA_LINKS), + "", + ] + HTML.mkdir(parents=True, exist_ok=True) + (HTML / "llms.txt").write_text("\n".join(index)) + + # llms-full.txt: the prose introduction, then every page + def block(url: str, body: str) -> str: + return f"\n{'=' * 72}\nPage: {url}\n{'=' * 72}\n\n{body.strip()}\n" + + parts = [f"# {name}: {summary}\n\nDocumentation: {base}\n\n{INTRO.read_text().strip()}\n"] + parts += [block(f"{base}{p}.html", texts[p]) for p in pages] + full = re.sub(r"\n{3,}", "\n\n", "".join(parts)) # the text builder leaves runs of blank lines + (HTML / "llms-full.txt").write_text(full) + print(f"llms.txt: {len(pages)} pages; llms-full.txt: {len(full) // 1000} kB", file=sys.stderr) + + +if __name__ == "__main__": + main()