docs: llms.txt, llms-full.txt and AGENTS.md

.github/workflows/ci.yml

@@ -99,7 +99,7 @@ jobs:
           sudo apt-get install -y graphviz
       - name: Build docs
         run: |
-          make -C docs html
+          make -C docs html llms
           touch docs/build/html/.nojekyll
       - name: Deploy docs to gh-pages
         uses: peaceiris/actions-gh-pages@v4

AGENTS.md

@@ -1,5 +1,7 @@
 # pgraph-python — Agent Instructions
 
+> **Audience:** agents working on pgraph's source code. If you're helping someone *use* pgraph, read https://petercorke.github.io/pgraph-python/llms.txt instead (or `llms-full.txt` for the complete documentation).
+
 Part of the RVC ecosystem. **Read [rvc-ecosystem/AGENTS.md](https://github.com/petercorke/rvc-ecosystem/blob/main/AGENTS.md) first** — it defines shared conventions: repo ownership, math invariants, dependency boundaries, git/PR workflow, code standards, tech-debt tracking. This file only adds what's specific to this repo.
 
 | | |
@@ -8,10 +10,19 @@ Part of the RVC ecosystem. **Read [rvc-ecosystem/AGENTS.md](https://github.com/p
 | Nickname | pgraph-python |
 | Owner | Peter Corke (`petercorke`) |
 | Default branch | `main` |
-| Contribution model | Branch → PR; direct push to `main` at Peter's discretion |
+| Contribution model | Branch → PR; `main` is protected: CI must pass and the branch must be up to date (`gh pr update-branch`) |
 
 ## Notes specific to this repo
 
-- Minimal graph/path-planning utility package — no internal ecosystem dependencies.
-- Used by `robotics-toolbox-python` (path/roadmap planning) and `machinevision-toolbox-python`.
-- Pure Python — builds with Hatch (`hatchling`) directly.
+- Minimal graph/path-planning package. Pure Python, builds with Hatch (`hatchling`); Python 3.9+. Depends on `numpy`, `matplotlib` and `spatialmath-python`; `ansitable` and `scipy` arrive through spatialmath. **Do not add scipy to `pyproject.toml`**: it is imported lazily, only inside `Graph.build_kdtree()`.
+- Used, unpinned (`"pgraph-python"`), by `robotics-toolbox-python` (`PRMPlanner`, `RRTPlanner`, `LatticePlanner`, `PoseGraph`, which subclass `UVertex`/`DVertex`) and `machinevision-toolbox-python` (`BundleAdjust`). A breaking change needs a deprecation cycle with a working fallback (`warnings.warn(..., DeprecationWarning, stacklevel=2)`); run their pgraph-using tests before a release.
+- **Layout.** `src/pgraph/PGraph.py` holds `Graph`, `UGraph`, `DGraph`, `Vertex`, `Edge` and `SE2Metric`; `exporters.py` the `export()` formats. There is one `Vertex` class: a vertex has no direction, the graph decides how an edge is recorded (`UGraph`/`DGraph` implement the hooks `_attach`, `_detach`, `_incoming` and `_endpoints_match`). `UVertex`/`DVertex` are deprecated aliases, by module `__getattr__`.
+- **Records that must agree.** A vertex has `_edgelist` (leaving, or incident if undirected) and `_inlist` (entering, directed only); the graph has `_vertexlist`, `_vertexdict` (names) and `_edges` (an insertion-ordered dict). `g.validate()` checks them all; use it in tests of anything that edits the structure.
+- **Two staleness flags, consumed independently.** `_connectivitychange` (components, reset by `_graphcolor`) and `_spatialchange` (the k-d tree). Set both wherever a vertex is added or removed; only connectivity cares about edges. Never share one flag.
+- **`Vertex.__setattr__` is hot.** It exists to tell the graph that `coord` was assigned. Anything run per vertex (`Vertex.__init__`, `add_vertex`, components) writes `vertex.__dict__` directly to skip it; keep doing that, and declare new attributes at class level so mypy knows them.
+- **Pickle and `copy` use a flat description** (`__getstate__`/`__setstate__`: vertices, edges by index, replayed through `_attach`); `subgraph` reuses it. Derived or transient state on a graph (the k-d tree) must be excluded there. A bare vertex or edge refuses to be pickled or copied.
+- **Tests** are `unittest` classes in `tests/`, run with `pytest`; write expected values by hand or from an oracle in the test, never from the code under test. Import names explicitly (a star import is a Codacy `F405`); route `pickle` through one helper marked `# nosec B301 nosemgrep`. Codecov gates patch coverage, so every changed line needs a test. `mypy src` is a CI check.
+- **Performance claims are measured.** `examples/benchmark.py` (`--md`, `--directed`) produces the README tables; regenerate them after a performance change and compare with `v0.6.5` (`git archive v0.6.5 src`), not with a development build.
+- **Docs** are Sphinx (`make -C docs html`; `-n` for nitpicky) with `sphinx-pyrunblock` `.. runblock:: pycon` examples that execute at build. `docs/llms_intro.md` is the hand-written part of `llms.txt`: update it when the API changes; `docs/llms_txt.py` builds `llms.txt` and `llms-full.txt` from it and the toctrees (`make html llms`), and CI publishes them with the site.
+- **Release.** Version is in `pyproject.toml`, history in `CHANGELOG.md`. Publishing is `.github/workflows/publish.yml` on a GitHub Release (trusted publishing, environment `pypi`): it refuses unless the tag equals the version and the commit is on `main`. Don't tag or upload by hand.
+- `examples/london.json` is OpenStreetMap data (© OpenStreetMap contributors, ODbL); keep the attribution with it.

README.md

@@ -91,7 +91,7 @@ Map data © OpenStreetMap contributors, available under the [Open Database Licen
 
 ## Using pgraph
 
-Full details are given in the [online documentation](https://petercorke.github.io/pgraph-python).
+Full details are given in the [online documentation](https://petercorke.github.io/pgraph-python).  For an AI assistant there is a short summary, [llms.txt](https://petercorke.github.io/pgraph-python/llms.txt), and the complete documentation as one file, [llms-full.txt](https://petercorke.github.io/pgraph-python/llms-full.txt).
 
 ### Properties and methods of the graph
 

docs/Makefile

@@ -12,7 +12,11 @@ BUILDDIR      = build
 help:
 	@$(SPHINXBUILD) -M help "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)
 
-.PHONY: help Makefile
+# llms.txt and llms-full.txt for AI assistants, written into build/html; run after html
+llms:
+	SPHINXBUILD="$(SPHINXBUILD)" python llms_txt.py
+
+.PHONY: help Makefile llms
 
 # Catch-all target: route all unknown targets to Sphinx using the new
 # "make mode" option.  $(O) is meant as a shortcut for $(SPHINXOPTS).

docs/llms_intro.md

@@ -0,0 +1,43 @@
+pgraph is a pure-Python package of graph classes: directed and undirected graphs of vertices connected by edges, with edge costs and path planning (breadth-first, uniform-cost and A*). Vertices can be *embedded*, given a coordinate in a space of any dimension; edge costs then come from a distance metric (L2, L1, SE2 or your own), and the graph can be queried spatially, plotted and exported. Self-loops and parallel edges are allowed. It requires Python 3.9 or later and depends on NumPy, Matplotlib and spatialmath-python. Install with `pip install pgraph-python`; import it as `pgraph`.
+
+## Quick start
+
+```python
+from pgraph import UGraph
+
+g = UGraph(dim=2)                       # DGraph for directed; Graph(directed=flag) chooses
+a = g.add_vertex([0, 0], name="a")      # returns the Vertex
+b = g.add_vertex([3, 4], name="b")
+c = g.add_vertex([6, 0], name="c")
+g.add_edge(a, b)                        # cost defaults to the metric distance, 5.0
+g.add_edge("b", "c", cost=7)            # vertices or their names; or give the cost
+
+path, length, _ = g.path_Astar("a", "c")        # None if there is no path
+print([v.name for v in path], length)           # ['a', 'b', 'c'] 12.0
+g.plot()
+g.highlight_path(path)
+```
+
+## Key points
+
+- **Classes.** `UGraph` (undirected) and `DGraph` (directed) share the base class `Graph`, which is concrete: `Graph(directed=True)` is a `DGraph`. There is one `Vertex` class for both and one `Edge` class. `UVertex` and `DVertex` are deprecated aliases of `Vertex`. Constructor arguments are keyword-only: `UGraph(dim=2, metric="L2", heuristic=None, vertex_class=None, edge_class=None, verbose=False)`; `UGraph(2)` is a `TypeError`.
+- **Vertices.** `g.add_vertex(coord=None, name=None)` returns the vertex. Default names are `"#0"`, `"#1"`, ..., names are unique, and `g["a"]` or `g[0]` finds a vertex. `for v in g` iterates in the order added; `g.n`, `g.ne` and `g.nc` are the numbers of vertices, edges and components. `g.remove_vertex(v)` and `g.remove_edge(e)` remove them, with their edges.
+- **Edges.** `g.add_edge(v1, v2, cost=None)`; with no `cost` it is the metric distance between the coordinates, and `None` if the vertices have none (planning then raises `ValueError`). `g.edges()` and `v.edges()` return new lists in the order added. `e.v1` and `e.v2` are read-only. `v.adjacent()` is the neighbours, `v.successors()` and `v.predecessors()` are the directed ones, `v.edgeto(w)` the cheapest edge to `w` and `v.edges_to(w)` all of them. A self-loop of an undirected vertex is in `v.edges()` twice and counts 2 in its degree.
+- **Directed graphs.** `degree` is in plus out (`in_degree`, `out_degree`); `nc` and `component` are weakly connected, ignoring direction.
+- **Planning.** `path_BFS(S, G)` returns `(path, length)` with the fewest edges; `path_UCS(S, G)` and `path_Astar(S, G)` return `(path, length, parents)` of least cost. `S` and `G` are vertices or names, and all three return `None` when there is no path, so check before unpacking.
+- **Metrics.** `"L2"` (default), `"L1"`, `"SE2"` for `(x, y, theta)` poses, `SE2Metric(weight)` to weight the angle, or a function of one difference vector. A* uses `heuristic` (default: the metric), which must be in the same units as the edge costs and must not exceed the true cost to go, or the path is not the shortest.
+- **Spatial queries** on an embedded graph: `closest(point)` returns `(vertex, distance)`, `near(point, radius)`, `within(box)` with `box` a 2 x n array of lower and upper corner, `coordinates()` the `(dim, n)` matrix of coordinates, and `subgraph(vertices)`, which takes vertices, names or a boolean mask and returns a new graph. `build_kdtree()` is an optional index (it needs scipy) that `closest` and `near` then use.
+- **Components and matrices.** `g.component(c)` is the vertices in component number `c`, `v.label` is the component of a vertex, `g.samecomponent(v1, v2)`. `adjacency()`, `incidence()`, `laplacian()`, `degree()` and `connectivity()` return dense NumPy arrays.
+- **Output.** `g.plot()` (arrowheads for a directed graph, arcs for parallel edges, circles for loops), `highlight_path(path)`, and `g.export(file)` for GraphViz dot, Mermaid, GraphML, ELK JSON or an image (images need the GraphViz executables).
+- **Subclassing.** `class Foo(Vertex)` and `UGraph(vertex_class=Foo, edge_class=Bar)` make the graph build them; a subclass `__init__` must pass `coord` and `name` (vertices) or `v1, v2, cost, data` (edges) to its parent.
+- **Copying.** `g.copy()` and `pickle` work for a whole graph; a vertex or edge cannot be copied or pickled on its own, and a graph whose metric is a `lambda` cannot be pickled.
+
+## Common mistakes
+
+- `UGraph(6)` raises `TypeError`: write `UGraph(dim=6)`.
+- Unpacking the result of a planner without checking for `None`.
+- Coordinates in one unit and costs in another (metres against kilometres): the A* heuristic is then too large and the path found is not the shortest.
+- `g.component(v)` takes a component *number*; use `v.label` for the component of a vertex.
+- Assigning `e.v1 = ...` fails, they are read-only; `add_vertex` raises `ValueError` for a name already used or a vertex that is already in a graph.
+- After `build_kdtree()`, change a coordinate by assigning (`v.coord = x`, `v.coord += x`), not in place (`v.coord[0] = x`), which the index cannot see.
+- `g.edges()` is a copy: changing the list does not change the graph.

docs/llms_txt.py

@@ -0,0 +1,170 @@
+"""Write llms.txt and llms-full.txt for the documentation site.
+
+``llms.txt`` (see https://llmstxt.org) is a short Markdown file that tells AI assistants
+what the package is, how to use it, and where its documentation lives. ``llms-full.txt``
+is the whole documentation as plain text, so an assistant can read it in a single fetch.
+
+The package is small, so ``llms.txt`` is mostly prose: the hand-written ``llms_intro.md``
+(what it does, a quick start, key points, common mistakes) followed by a generated list
+of links. ``llms-full.txt`` is this Sphinx site rendered by Sphinx's ``text`` builder, so
+the autodoc API reference is included, in the order of the toctrees, which are followed
+from ``index.rst`` through the nested pages. Run after the HTML build::
+
+    make html llms          # from docs/
+
+The files are written into ``build/html`` so they are published with the site.
+"""
+
+from __future__ import annotations
+
+import os
+import re
+import shutil
+
+# Used only to re-invoke `sphinx-build`, resolved via shutil.which() below; this is a
+# local/CI docs-build script, not code that runs against untrusted input.
+import subprocess  # nosec B404
+import sys
+from importlib.metadata import metadata
+from pathlib import Path
+
+DOCS = Path(__file__).resolve().parent
+SOURCE = DOCS / "source"
+BUILD = DOCS / "build"
+TEXT = BUILD / "text"
+HTML = BUILD / "html"
+INTRO = DOCS / "llms_intro.md"
+
+# One-line descriptions of the pages listed in the toctrees
+PAGE_NOTES = {
+    "index": "overview",
+    "intro": "tutorial: building a graph, vertices, edges, plotting, path planning, directed graphs",
+    "api": "the API, in three parts: the common methods, vertices and edges, and the policy page",
+    "common": "the methods that UGraph and DGraph share, by task",
+    "construction": "UGraph, DGraph, Graph, constructors from a dict or an adjacency matrix, distance metrics",
+    "modification": "adding and removing vertices and edges",
+    "analysis": "counts, adjacency, degree, components, coordinates, spatial queries, subgraphs",
+    "representations": "adjacency, incidence, degree and Laplacian matrices",
+    "visualization": "plot(), its options, highlighting vertices, edges and paths",
+    "export": "writing a graph as GraphViz dot, Mermaid, GraphML, ELK or an image",
+    "planning": "path_BFS, path_UCS and path_Astar",
+    "edges_vertices": "the data structures: Vertex and Edge, and the edge lists of a vertex",
+    "policy": "object lifecycle: what may be done with a vertex or edge outside a graph",
+}
+
+# Links that belong in llms.txt but are not pages of this site
+EXTRA_LINKS = [
+    ("Source code", "https://github.com/petercorke/pgraph-python", "GitHub repository"),
+    ("PyPI", "https://pypi.org/project/pgraph-python/", "install with pip install pgraph-python"),
+    ("Changelog", "https://github.com/petercorke/pgraph-python/blob/main/CHANGELOG.md", "release history and how to migrate from 0.6"),
+    ("Examples", "https://github.com/petercorke/pgraph-python/tree/main/examples", "Queensland and central London route-planning notebooks, and a benchmark"),
+    ("RVC ecosystem", "https://github.com/petercorke/rvc-ecosystem", "how pgraph relates to the Robotics and Machine Vision Toolboxes"),
+    ("Contributing", "https://github.com/petercorke/pgraph-python/blob/main/AGENTS.md", "repo conventions and workflow, for agents working on pgraph's source code"),
+]
+
+
+def toctree_entries(document: Path) -> list[str]:
+    """Document names listed in the toctrees of one document, in order.
+
+    :param document: an ``.rst`` file
+    :return: document names such as ``"common"``
+    """
+    entries, in_toctree = [], False
+    for line in document.read_text().splitlines():
+        if line.startswith(".. toctree::"):
+            in_toctree = True
+        elif in_toctree and line.strip() and not line.startswith((" ", "\t")):
+            in_toctree = False  # the directive's indented body has ended
+        elif in_toctree and line.strip() and not line.strip().startswith(":"):
+            entries.append(line.strip())
+    return entries
+
+
+def toctree_pages(index: Path) -> list[str]:
+    """All the documents below a master document, depth first, in toctree order.
+
+    The pages of this site are nested: ``index`` lists ``api``, which lists
+    ``common``, which lists the pages of the common methods.
+
+    :param index: the master document, ``index.rst``
+    :return: document names such as ``"construction"``, each once
+    """
+    pages: list[str] = []
+
+    def walk(document: Path) -> None:
+        for name in toctree_entries(document):
+            if name not in pages:
+                pages.append(name)
+                child = document.parent / f"{name}.rst"
+                if child.exists():
+                    walk(child)
+
+    walk(index)
+    return pages
+
+
+def title(text: str) -> str:
+    """The page title: the first non-blank line that isn't an overline of punctuation."""
+    for line in text.splitlines():
+        if line.strip() and not re.fullmatch(r"[*=\-~#^]+", line.strip()):
+            return line.strip()
+    return ""
+
+
+def main() -> None:
+    # Read the name, summary and documentation URL from the installed package's metadata
+    # (built from pyproject.toml). Reading pyproject.toml directly would need tomllib,
+    # which is Python 3.11+, and the Pages workflow builds the docs on 3.9.
+    meta = metadata("pgraph-python")
+    urls = dict(u.split(", ", 1) for u in meta.get_all("Project-URL"))
+    base = urls["Documentation"].rstrip("/") + "/"
+    name, summary = meta["Name"], meta["Summary"]
+
+    # SPHINXBUILD matches the override convention of docs/Makefile's
+    # `SPHINXBUILD ?= sphinx-build` -- resolved via shutil.which() to a full path.
+    sphinxbuild_name = os.environ.get("SPHINXBUILD", "sphinx-build")
+    sphinxbuild = shutil.which(sphinxbuild_name)
+    if sphinxbuild is None:
+        raise SystemExit(f"sphinx-build not found: {sphinxbuild_name!r}")
+    subprocess.run(  # nosec B603 -- sphinxbuild resolved via shutil.which() above  # nosemgrep: python.lang.security.audit.dangerous-subprocess-use-audit.dangerous-subprocess-use-audit
+        [sphinxbuild, "-q", "-b", "text", str(SOURCE), str(TEXT)],  # nosemgrep: python.lang.security.audit.dangerous-subprocess-use-tainted-env-args.dangerous-subprocess-use-tainted-env-args
+        check=True,
+    )
+    pages = ["index"] + toctree_pages(SOURCE / "index.rst")
+    texts = {p: (TEXT / f"{p}.txt").read_text() for p in pages}
+
+    # llms.txt: prose introduction, then links
+    index = [
+        f"# {name}",
+        "",
+        f"> {summary}.",
+        "",
+        INTRO.read_text().strip(),
+        "",
+        "## Documentation",
+        "",
+        f"The whole documentation, including the API reference, is available as a single text file: [llms-full.txt]({base}llms-full.txt).",
+        "",
+        *(f"- [{title(texts[p])}]({base}{p}.html): {PAGE_NOTES.get(p, '')}" for p in pages),
+        "",
+        "## Optional",
+        "",
+        *(f"- [{label}]({url}): {note}" for label, url, note in EXTRA_LINKS),
+        "",
+    ]
+    HTML.mkdir(parents=True, exist_ok=True)
+    (HTML / "llms.txt").write_text("\n".join(index))
+
+    # llms-full.txt: the prose introduction, then every page
+    def block(url: str, body: str) -> str:
+        return f"\n{'=' * 72}\nPage: {url}\n{'=' * 72}\n\n{body.strip()}\n"
+
+    parts = [f"# {name}: {summary}\n\nDocumentation: {base}\n\n{INTRO.read_text().strip()}\n"]
+    parts += [block(f"{base}{p}.html", texts[p]) for p in pages]
+    full = re.sub(r"\n{3,}", "\n\n", "".join(parts))  # the text builder leaves runs of blank lines
+    (HTML / "llms-full.txt").write_text(full)
+    print(f"llms.txt: {len(pages)} pages; llms-full.txt: {len(full) // 1000} kB", file=sys.stderr)
+
+
+if __name__ == "__main__":
+    main()