[rajivpant]

Closes the last open items of the 4.0.0 upgrade.

Embedding model

RAG embeds with microsoft/harrier-oss-v1-0.6b (MIT licence, 1,024 dimensions, 32K-token context, 69.0 on Multilingual MTEB v2 per its model card) in place of all-MiniLM-L6-v2 (384 dimensions, 256 tokens).

• engines.yaml's embedding: block is the only place the model is set, with a pinned Hugging Face revision. get_embedding_config() validates it. The EMBEDDING_MODEL override is removed: model, dimension and query instruction must change together.
• Queries are encoded with the model's web_search_query instruction; documents carry none, as the model card specifies.
• Indexing embeds chunks in batches instead of one model call per chunk.
• Loading a model that contradicts engines.yaml (dimension, missing prompt) raises ConfigurationError.
• The pgvector backend reads the schema's dimension from the catalog. init_collection returns False on a mismatch; upsert_points and search raise. ragbot db init checks it, and ragbot db status reports it.
• Migration 0004_embedding_dimensions.sql moves chunks.embedding and entities.embedding to vector(1024). It clears chunks and documents (derived data; re-index rebuilds them) and entity embeddings (entities are kept). The runner executes every migration on every start, so each step is guarded by the column's current dimension.
• float32, not the shipped bfloat16. On CPU, bfloat16 measured 0.3 chunks/s against 4.4 for float32 on Apple Silicon. In the Linux API image float32 runs at 1.8 chunks/s, peaking at 4.6 GB during load, so Docker Desktop needs more than its 4 GB default.
• The API container keeps the model in a ragbot-hf-cache volume.

filelock

LiteLLM floor 1.109.0rc1, the first LiteLLM that accepts filelock 4; filelock floor 4.2.9. pip-audit: no known vulnerabilities (was 13 CVEs). 1.108.1, published a day later, is an LTS backport of fixes already on LiteLLM's main branch, none of them security fixes (BerriAI/litellm#50136). Details in docs/security/dependabot-triage-2026-11.md. transformers floor raised to 5.22.2, the latest.

Verification

• make test-fast: 1125 passed, 13 skipped (live-database tests).
• With a throwaway pgvector 18 database and the app/migration role split: 1134 passed, 4 skipped. The live pgvector, row-level security and memory tests ran.
• Upgrade path on a restored copy of a real 384-dimension index:
  • 315 chunks and 68 documents cleared, workspace rows kept;
  • both columns and HNSW indexes at 1,024;
  • re-running all migrations leaves new 1,024 rows untouched;
  • a 384-dimension init is refused.
• End to end with the real model: the demo workspace indexed at 1,024 dimensions, with model and revision recorded on each chunk, and searches return ranked hits.
• pip check clean; every requirements floor is at the latest PyPI release. Web packages are current except TypeScript 7, which typescript-eslint (8.72.1 and its canary) still caps below 6.1.

After merging: docker compose up -d --build, then ragbot index --workspace <name> --force for each workspace.

🤖 Generated with Claude Code