Skip to content

fix: upgrade dependencies and harden Autoflow security boundaries - #717

Draft
shizn wants to merge 22 commits into
mainfrom
chord/fix-dependabot-alerts
Draft

shizn wants to merge 22 commits into
mainfrom
chord/fix-dependabot-alerts

Conversation

@shizn

@shizn shizn commented Oct 23, 2026 •

Copy link
Copy Markdown
Collaborator

This update retains the dependency upgrades from #717 and hardens Autoflow's authorization, credential handling and model configuration. It is a draft while the remaining repository security findings and deployment validation are completed.

Implemented:

  • Central administrator authorization; signed browser identity; account/guest/API-key chat isolation; private-engine enforcement; server-side source redaction and authorized document downloads.
  • Atomic session revocation after credential or account changes, expiring resource-scoped API keys, and compatible chat evaluation grants. Legacy unrestricted keys are disabled by migration.
  • Independent AES-GCM credential encryption with an explicit dry-run/apply migration tool, retired example signing keys, secret-safe settings, and private administrator bootstrap input.
  • Operator-registered provider endpoints, explicit credentials without ambient-key fallback, credential replacement on destination changes, row-locked configuration updates, bounded provider transports, safe model-test errors, and a management form that updates configuration and encrypted credentials atomically.
  • Authenticated, private, non-root local model sidecars with fixed model revisions, offline runtime and workload bounds; unauthenticated Flower has been removed. Credential-bearing browser recordings and public E2E report uploads have been disabled.
  • Data-only prompt templates reject Python traversal, file/network filters and unbounded execution, while preserving every shipped template and LlamaIndex response synthesis.
  • Isolated credential-free web/sitemap crawling with DNS-pinned connections, TLS hostname verification, redirect/subrequest checks, browser fallback-network isolation, current Chromium and bounded execution.
  • Exact-origin CORS and browser write checks, verified same-origin frontend proxy forwarding, safe upstream error logging, read-only recommendation GETs, and bounded database-backed model history.
  • Atomic Redis admission across replicas with account/guest/IP/global budgets, per-chat concurrency locks, bounded request bodies, disconnect-safe streaming cleanup, and atomic recommendation cache replacement. Real Redis and HTTP-server regressions cover concurrent admission, outages, deadline checks and provider response bounds.
  • Attributed, value-free security events and committed mutation audits for graph/configuration/credential changes, including KB scope and recovery-token attribution. Audits and changes roll back together; legacy audit data remains distinguishable.
  • Frontend administrative access, safe document preview/login callback/analytics IDs, graph and retrieval bounds, referenced-model deletion protection, evaluation dataset isolation, and workflow input hardening.

Validation: 217 backend security tests, 51 frontend tests, frontend TypeScript check, changed Python Ruff checks, and diff checks passed locally. Database regression tests currently use real SQLite transactions; TiDB/container integration remains pending. The new Security Regression workflow runs the backend suite in CI. The sidecar container passed native-library, authentication and real inference checks for both pinned models. Real Chromium tests cover dynamic rendering, blocked internal HTTP/WebSocket/preconnect/WebRTC/service-worker access and timeout cleanup; HTTPS tests verify SNI/hostname checks on pinned connections. E2E now uses Playwright 1.64.1 (matching the validated crawler browser); all 24 tests collect. The previous E2E job stalled downloading Chromium 140 before executing tests. Browser installation has its own deadline, and integration/preview checks still require validation against the final branch.

Rollout requires coordinated API/worker downtime, private database/environment backups, schema migrations, provider-credential re-encryption, independent signing/encryption keys, new scoped API keys, and registration of custom provider origins. See backend/SECURITY_MIGRATION.md for the current procedure.

Still in progress: remaining provider-specific/DSPy deadline coverage and secret-safe diagnostic/Sentry logging, full CI/runtime deployment verification and Helix rescan. Do not deploy this draft independently of the documented migrations.

shizn and others added 7 commits October 22, 2026 11:31
Upgrade vulnerable packages in backend/uv.lock (fastapi/starlette,
fastapi-users, python-multipart, litellm, dspy 3, ragas 0.4, deepeval,
langchain 1.x, pypdf 6, pillow, nltk, aiohttp, cryptography, transformers
removed, etc.) and adapt the code to the new versions:

- fastapi-pagination >= 0.13 requires `apaginate` for async sessions.
- ragas 0.4 names mode-based metric columns like
  "factual_correctness(mode=f1)".
- cryptography moved the CFB mode to the `decrepit` module.
- ragas 0.4.3 still imports langchain_community.chat_models.vertexai,
  so constrain langchain-community < 0.4.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Resolves GHSA-jhr6-gm9c-rqjv and switches to the non-deprecated
CrossEncoder / encode() arguments.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Upgrade litellm, llama-index-core 0.13, dspy 3, banks, deepdiff, pypdf,
nltk, mistune, jupyter dev dependencies, etc. in core/uv.lock, and call
dspy modules directly instead of `.forward()` as recommended by dspy 3.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Pulls in a patched undici and drops @fastify/busboy.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…erts

- Upgrade next / @next/third-parties / eslint-config-next to 15.5.28,
  storybook to 8.6, @tanstack/react-form to 0.42, undici, postcss,
  js-cookie, vite, eslint.
- Add scoped pnpm overrides for vulnerable transitive dependencies.
- Replace the `ai` package (only `parseStreamPart` was used) with a small
  local parser for the data stream protocol, since newer `ai` majors
  dropped it.
- Adapt to @tanstack/react-form 0.42 (`batch` from @tanstack/store).
- Patch @storybook/nextjs's image loader for image-size 2 and the
  upgraded sharp.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@shizn shizn added the dependencies Pull requests that update a dependency file label Oct 23, 2026
@vercel

vercel Bot commented Oct 23, 2026 •

Copy link
Copy Markdown
Contributor

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
autoflow Ready Ready Preview Nov 3, 2026 12:35am UTC
tidb-ai-preview Error Error Nov 3, 2026 12:35am UTC
tidb-ai-storybook Error Error Nov 3, 2026 12:35am UTC

Request Review

ruff-action installs the latest ruff by default, and ruff >= 0.16 enables
a much larger default rule set (~1600 findings across the repo, also on
main). Use the same version as backend/uv.lock.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…advisories

- Require Python >= 3.11 (3.10 reached end of life in October 2026):
  the pillow fixes for the advisories published on 2026-10-20..22 are
  only available in pillow 13, which no longer supports Python 3.10.
  The backend Docker image already uses Python 3.11.
- backend: certifi 2026.10.19, virtualenv 21.14.13. langfuse 2.x caps
  packaging < 25 although it only uses packaging.version, so override it
  to allow the patched virtualenv (pre-commit dependency).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@shizn
shizn marked this pull request as draft November 2, 2026 21:58
@shizn shizn changed the title fix: upgrade dependencies to resolve Dependabot security alerts fix: upgrade dependencies and harden Autoflow security boundaries Nov 2, 2026

This branch had an error being deployed

2 failed and 1 active deployments
Preview – autoflow — 0fd90a52 Deployed Nov 3, 2026 by vercel[bot]
Preview – tidb-ai-preview — 0fd90a52 Deployed Nov 3, 2026 by vercel[bot]
Preview – tidb-ai-storybook — 0fd90a52 Deployed Nov 3, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

dependencies Pull requests that update a dependency file require-build

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant