Repository navigation
Run a local Langfuse for development - #232
Merged
Merged
Conversation
The Langfuse screens show something only with a connection that has data, and the local stack started no Langfuse. compose.langfuse.yaml runs 4.72.0, production's version, as one instance every worktree shares, and scripts/langfuse-seed.ts makes a project with keys and a judge through the routes Langfuse's own UI calls, since its public API makes no projects without a licence. Both run when read from origin/main, as walks do.
COMPOSE_PROJECT_NAME overrides the compose file's name, so npm run langfuse passes -p langfuse-genki. A deleted Langfuse project keeps its LLM connections, so the seed deletes them first. Renovate leaves the file alone, since it moves with production by hand, and 4.72.0 is the version production is moving to, not the one it runs.
Contributor
Author
|
Ready for review. Blind review, a claim-checking review and a check of the fixes are done and CI passes. Open: #226's live tests still default to the |
kentare
marked this pull request as ready for review
November 2, 2026 17:44
6 tasks done
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What and why
The Langfuse screens (judges, judge costs, labels, datasets) show something only with a Langfuse connection that has data in it, and the local stack starts no Langfuse, so a smoke walk cannot click through them before review. This adds one shared local Langfuse for development and walks:
compose.langfuse.yaml: Langfuse's own compose file at v4.72.0, the version production is moving to, with fixed local-only values. Compose projectlangfuse-genki, so every worktree shares one instance; only the web UI is published, on127.0.0.1:34620; the start project has the keyspk-lf-genki-test/sk-lf-genki-test; no relative paths, so it also runs when read fromorigin/maininto another folder.npm run langfusestarts it or reuses the running one (-p langfuse-genki, so aCOMPOSE_PROJECT_NAMEin the shell starts no second one), and returns once/api/public/healthanswers (a health check on the web container).npm run setupandnpm startleave it alone, and no workflow or deploy uses it. Renovate leaves the file alone too: it moves with production by hand.scripts/langfuse-seed.ts(npm run langfuse:seed -- <project>), with onlynode:imports: deletes a project of the same name with its LLM connections, makes it again with its keys and prints the host, keys and ids as JSON.--judge <name>adds an LLM-as-a-judge evaluator and an enabled rule withisExperimentItemRootSpan = true;--llm-from <env file>adds a Foundry Claude LLM connection from that file's lastFOUNDRY_CLAUDE_*values;--deletedeletes the project. Langfuse without a licence makes no projects through its public API (403), so the seed logs in as the start user and calls the routes Langfuse's own UI calls for projects and keys; everything else goes through the public API.docs/development.md: "A local Langfuse".Closes #230.
Not in this PR, cut to what a walk needs: the annotation queue (
--annotation-queue), the live Langfuse tests passing with onlyLANGFUSE_LOCAL_URLset (they live on the Langfuse PRs, and #226 defaults to other keys), and connecting the seeded project in the portal (no Langfuse connection on main yet; the first walk of a Langfuse PR does it).How it was tested
By hand on macOS with OrbStack, made-up data only:
npm run langfusefrom the worktree: healthy in 20 s;/api/public/healthanswers{"status":"OK","version":"4.72.0"}and the start keys answer 200. The same file copied to a folder outside the repository and started withdocker compose -f <copy> up -d --waitreused the running containers (1.2 s, none recreated). WithCOMPOSE_PROJECT_NAME=genai-secondset,npm run langfusereused them too.Seed run from that outside folder with
--judge relevans --llm-from <main checkout>/.env.local: the project, its keys, the LLM connection and the judge were made, and the evaluator came outactive. A second seed of the same name made a new project; the first project's keys then got 401 and its LLM connection was gone from Langfuse's database. A judge seeded without--llm-fromcomes outpaused("no default evaluation model is configured").--delete: the keys get 401, and Langfuse's own database lists the project only as deleted, with no API keys and no LLM connection. A second--deleteprints"deleted": null.A Langfuse that does not answer prints "No Langfuse answers at …: npm run langfuse starts it."; an env file without
FOUNDRY_CLAUDE_KEYstops the seed before it makes anything.docker compose stopby project name, thennpm run langfuseagain: healthy.npx tsc --stricton the script;npm run checkpasses.npm run checkpassesChecklist
packages/providers-*docs/architecture/modulesare updated if a contract, boundary or module changedNo tests: this is development tooling, checked by running it against a real Langfuse 4.72.0 as above.