diff --git a/README.md b/README.md index c9244e4..ec4fd41 100644 --- a/README.md +++ b/README.md @@ -1,6 +1,6 @@ -# labagent: Hands-on Labs CLI and AI Agent for RAG, Text-to-SQL, Databricks Lakebase and AWS Sandboxes +# labagent: hands-on labs CLI for RAG, text-to-SQL, Databricks and AWS sandboxes -> An open-source **hands-on labs CLI and AI agent**. It launches **RAG** and **text-to-SQL** labs on **Databricks Lakebase** (Postgres with **pgvector**) and Databricks Apps, sets up **AWS training sandboxes with Terraform**, and removes everything when you are done. One command to launch a lab. One command to destroy it. A small local **Ollama** model can turn plain English into the right command. +> Launch a hands-on lab with one command. Remove it with one more. Labs for **RAG**, **text-to-SQL** on **Databricks Lakebase** (pgvector), a **browser VS Code per learner**, and **AWS sandboxes** with Terraform. [![PyPI](https://img.shields.io/pypi/v/labagent)](https://pypi.org/project/labagent/) [![Python](https://img.shields.io/pypi/pyversions/labagent)](https://pypi.org/project/labagent/) @@ -8,181 +8,77 @@ [![License](https://img.shields.io/badge/license-Apache--2.0-blue)](LICENSE) [![Databricks Free Edition](https://img.shields.io/badge/Databricks-Free%20Edition-FF3621)](docs/packs.md) -**Related projects:** [DB Agent](https://github.com/db-agent/db-agent) (text-to-SQL agent) | [labagent-datasets](https://github.com/becloudready/labagent-datasets) (pinned lab datasets) | [BeCloudReady workshops](https://github.com/becloudready/workshops) (lab content) +**Related:** [DB Agent](https://github.com/db-agent/db-agent) | [labagent-datasets](https://github.com/becloudready/labagent-datasets) | [BeCloudReady workshops](https://github.com/becloudready/workshops) ---- +## What it does -## What is labagent? +A lab is a **pack**: ordered steps that are safe to re-run. `launch` builds it and resumes after a failure. `destroy` removes only what the pack created. -labagent is a command line tool and AI agent for running hands-on technical labs and workshops. A lab is a **pack**: an ordered list of steps that is safe to re-run. `launch` builds the lab and stops with a clear error if something fails. Run it again and it resumes. `destroy` removes only what the lab created, so you do not pay for leftovers. +- **Databricks labs.** RAG over real SEC 10-K filings, and a text-to-SQL agent, in your own Free Edition workspace. +- **Web IDE per learner.** Private browser VS Code with the RAG lab preinstalled, on Modal, EC2 or local Docker. +- **AWS sandboxes.** Cohort IAM with Terraform, plus plan-then-apply cleanup of shared accounts. +- **Plain English.** `labagent "deploy rag-sec"` shows the command it understood and asks first (local Ollama, any OpenAI-compatible endpoint). -It covers three jobs: - -- **RAG and text-to-SQL labs on Databricks.** Learners get a working retrieval-augmented generation app or a text-to-SQL agent in their own Databricks Free Edition workspace. -- **AWS training sandboxes.** Instructors create IAM users, a sandbox policy and a Lambda role per cohort with Terraform, then clean up afterwards. -- **A plain-English front end.** `labagent "deploy rag-sec on databricks"` shows the command it understood and asks before it runs anything. - ---- - -## Features - -- **Hands-on labs with no leftovers.** `labagent launch ` builds the lab. `labagent destroy ` removes it. -- **Safe to re-run.** Every step checks whether it is already done. Re-running resumes after a failure. -- **Real data, pinned.** The RAG lab loads SEC 10-K filings from a checksummed [dataset release](https://github.com/becloudready/labagent-datasets/releases/tag/sec-10k-2026.10). A download with a different SHA-256 is refused. -- **RAG with the parts that matter.** Section-aware chunking, hybrid search (pgvector plus Postgres full-text), metadata filters, cited answers and a retrieval quality check that must pass before the app deploys. -- **Text-to-SQL agent.** Deploys [db-agent](https://github.com/db-agent/db-agent) to Databricks Apps at a pinned commit, then asks a real test question through the live app. -- **Plain English with Ollama.** A small model (`qwen2.5-coder:0.5b` by default) picks the lab. Names and parameters are read from your sentence by exact rules, and you confirm before it runs. -- **AWS sandboxes with Terraform.** S3 state with KMS encryption, one state per cohort, learner passwords returned encrypted. -- **Safe cleanup of shared AWS accounts.** Plan first, then apply exactly that reviewed plan. Only an explicit `delete` rule can delete. -- **Your credentials stay yours.** Labs run on your machine with your Databricks CLI profile. The db-agent lab creates a 30-day token, stores it only as an app secret and revokes it on `destroy`. -- **Open source.** Apache-2.0, with a test suite of 170+ tests. - -### Architecture - -``` - labagent "deploy rag-sec" --> Ollama (qwen2.5-coder:0.5b) --> action + lab name - | you confirm the command - labagent launch --> pack steps: preflight, build, load, index, evaluate, deploy - | - Databricks CLI --> Lakebase (Postgres + pgvector), Databricks Apps, SQL warehouse - - labagent run (CI or local) --> AWS engine --> Terraform (cohort IAM) + boto3 (inventory, sweep, cleanup) -``` - ---- - -## Lab catalog - -| Lab | What you learn | Cost | Command | -|---|---|---|---| -| `rag-sec-filings` | RAG over real SEC 10-K filings: chunking, hybrid search, citations, a quality gate before deploy | Databricks Free Edition | `labagent launch rag-sec-filings --name demo` | -| `rag-lakebase` | The same RAG pipeline on a small demo corpus, to smoke-test your setup | Databricks Free Edition | `labagent launch rag-lakebase --name demo` | -| `rag-sec-filings-live` | Fetch fresh filings from SEC EDGAR at launch | Databricks Free Edition | `labagent launch rag-sec-filings-live --name demo --param "user_agent=Your Name you@example.com"` | -| `db-agent` | Deploy a text-to-SQL agent over a Databricks sample dataset | Databricks Free Edition | `labagent launch db-agent --name demo` | - -Free Edition limits apply: one Lakebase project, three apps, and apps stop after 24 hours. Details: [docs/packs.md](docs/packs.md). - -## Deployment modes - -| | Lab packs | AWS sandbox engine | -|---|---|---| -| **Best for** | Self-service learners, demos, workshop labs | Instructors running cohorts | -| **Runs** | On your machine | In a private CI repo with GitHub OIDC, or locally | -| **Targets** | Databricks: Lakebase, Apps, SQL warehouse | AWS: IAM, Lambda, and cleanup of untagged resources | -| **Auth** | Your Databricks CLI profile | OIDC, no static keys | -| **Cost** | Databricks Free Edition limits | Your AWS bill | - ---- - -## Quick start: run your first hands-on lab +## Quick start ```bash -pip install 'labagent[rag]' # Python 3.9+; labs use the Databricks CLI: databricks auth login -labagent packs # list the labs +pip install 'labagent[rag]' # Python 3.9+; sign in with: databricks auth login +labagent packs # list the labs labagent launch rag-sec-filings --name demo labagent destroy rag-sec-filings --name demo ``` -## Run a RAG lab on Databricks Lakebase (pgvector) +## Labs -`rag-sec-filings` builds a retrieval-augmented generation (RAG) app on Postgres with pgvector. It loads a pinned snapshot of 10-K filings, scores retrieval on a set of questions, and deploys the app only if the score passes. The app has company and year filters, suggested questions and cited answers. - -```bash -labagent launch rag-sec-filings --name demo -``` - -Data: 1,880 passages from 10 filings (AAPL, CVX, JPM, PFE, WMT). Known gaps, such as missing MD&A sections for JPM and CVX, are listed in the [dataset README](https://github.com/becloudready/labagent-datasets). - -## Text-to-SQL agent on Databricks - -`db-agent` clones [github.com/db-agent/db-agent](https://github.com/db-agent/db-agent) at a pinned commit and deploys it to Databricks Apps. It then asks a test question through the live app. The data is the `samples` catalog that every workspace has, for example `samples.bakehouse`. - -```bash -labagent launch db-agent --name demo --param dataset=bakehouse -``` +| Pack | What you get | Runs on | +|---|---|---| +| `rag-sec-filings` | RAG over a pinned SEC 10-K dataset: hybrid search, filters, citations, a quality gate before deploy | Databricks Free Edition | +| `rag-sec-filings-live` | Same, fetching fresh filings from EDGAR | Databricks Free Edition | +| `rag-lakebase` | Small demo corpus to smoke-test your setup | Databricks Free Edition | +| `db-agent` | [db-agent](https://github.com/db-agent/db-agent) text-to-SQL over a Databricks sample dataset, pinned commit | Databricks Free Edition | +| `rag-ide` | A private browser VS Code per learner with the RAG lab and notebooks | Modal, EC2 or local Docker | -## Use labagent from plain English with Ollama +Free Edition limits apply (one Lakebase project, three apps). Details: [docs/packs.md](docs/packs.md). -A small local model turns a sentence into a command. You see the command and confirm it before anything runs. +## Web IDE per learner (unreleased: install from GitHub) ```bash -ollama pull qwen2.5-coder:0.5b -labagent "deploy rag-sec on databricks" +pip install 'labagent[modal] @ git+https://github.com/becloudready/labagent' +labagent launch rag-ide --name cohort --param learners=2 --param platform=modal ``` -Any OpenAI-compatible endpoint works. See [docs/packs.md](docs/packs.md#plain-english). +| `platform` | How | Needs | +|---|---|---| +| `modal` (default) | One serverless sandbox per learner | `modal token new` | +| `ec2` | One shared instance per cohort, a container per learner, HTTPS | An AWS profile (`--param profile=...`) in `LABAGENT_ALLOWED_ACCOUNTS` | +| `local` | Containers on this machine, loopback only | Docker | -## AWS training sandboxes with Terraform +Each learner gets a URL and a password (credentials file, mode 0600). Sessions stop when idle and at a time cap (`hours`, default 4). Tested with small cohorts, not at 30 learners. -labagent creates IAM users, a sandbox policy and a Lambda role per cohort with Terraform. Passwords are returned encrypted to your public key, so Terraform state holds no secrets. It also tears down what learners created. +There is also a laptop-only RAG lab: `pip install 'labagent[rag-local]'`, then `labagent rag init` and `labagent rag ide`. -| Command | What it does | -|---|---| -| `provision` | Create a cohort's IAM layer with Terraform | -| `inventory`, `sweep` | List and delete what learners created. Plan first, then apply that reviewed plan | -| `reset`, `offboard` | Rotate a password. Destroy a cohort | -| `audit`, `cleanup` | Clean a long-lived account from a ruleset | +## AWS sandboxes ```bash labagent --config lab.json run --job job.json --output result.json ``` -Job format: [docs/job-contract.md](docs/job-contract.md). State and secrets: [docs/state-and-secrets.md](docs/state-and-secrets.md). - -## Clean up a shared AWS account safely - -`audit` lists everything in an account. `cleanup` builds a plan from an explicit ruleset. You review the plan, then apply exactly that plan. A `keep` rule always wins, and anything that matches no rule is reported, never removed. See [docs/account-cleanup.md](docs/account-cleanup.md). - -## Safety - -- Plan first, then apply. The plan is bound to the account, its items, an age limit and the hash a human approved. -- Only a `delete` rule can delete. Unmatched resources are reported, never removed. -- labagent refuses to run in an AWS account that is not on your allow list. -- No plaintext secrets or emails in output or logs. No personal data in this repo. Tests enforce both. - ---- - -## FAQ - -**How do I deploy a RAG app on Databricks Free Edition?** -Run `pip install 'labagent[rag]'`, sign in with `databricks auth login`, then `labagent launch rag-sec-filings --name demo`. - -**How do I remove everything a lab created?** -Run `labagent destroy --name demo`. It removes the app and its code. It also revokes the db-agent token and deletes the Lakebase project, but only when labagent created them. - -**Do I need Ollama?** -No. Ollama is only for the plain-English front end. The explicit `launch` and `destroy` commands work without it. - -**Does labagent store my Databricks credentials?** -No. Labs run with your Databricks CLI profile. The db-agent lab creates a 30-day token, stores it only as an app secret and revokes it on `destroy`. - -**What does a lab cost?** -The Databricks labs target Free Edition, within its limits. The AWS sandbox engine uses your AWS account and bill. - -**Can I add my own lab?** -Yes. Write a `Pack` class and register it through the `labagent.packs` entry point. See [docs/packs.md](docs/packs.md). - ---- - -## Learning the concepts - -The lab content that goes with these tools lives in the [BeCloudReady workshops](https://github.com/becloudready/workshops) repo. - -| Workshop | What you build | +| Command | What it does | |---|---| -| [Databricks Genie and AI agents](https://github.com/becloudready/workshops/tree/master/workshops/databricks/genie-ai-agents/fundamentals) | LLM basics, structured output, tool use, the agentic loop and an MCP server | -| [Text-to-SQL on Databricks Lakebase](https://github.com/becloudready/workshops/tree/master/workshops/databricks/db-agent-lakebase) | Lakebase OLTP data, Unity Catalog gold tables and a self-hosted LLM endpoint | -| [AWS data lake](https://github.com/becloudready/workshops/tree/master/workshops/aws-data-lake) | S3, Glue, Athena, Lake Formation, Redshift Serverless, DMS and OpenSearch labs | -| [LLMOps](https://github.com/becloudready/workshops/tree/master/workshops/llmops) | GPU setup, vLLM, observability and an LLM gateway | - ---- +| `provision`, `offboard` | Create or destroy a cohort's IAM layer with Terraform | +| `inventory`, `sweep` | List and delete learner resources: plan first, then apply that reviewed plan | +| `reset` | Rotate a password (returned encrypted to your public key) | +| `audit`, `cleanup` | Clean a long-lived account from a ruleset | -## Workshops and training +See [job-contract](docs/job-contract.md), [state and secrets](docs/state-and-secrets.md), [account cleanup](docs/account-cleanup.md). -The team behind labagent teaches hands-on bootcamps on AI agents, RAG, text-to-SQL, Databricks and AWS. Running labs for a class or team? Private batches and a dashboard are available from [BeCloudReady](https://becloudready.com/). Community: [TorontoAI](https://toronto-ai.org/). +## Safety -## Maintained by +- Plan first, then apply that exact plan. Only a `delete` rule can delete; unmatched resources are reported, never removed. +- AWS commands refuse to run in an account that is not on your allow list. +- Labs use your own credentials. The db-agent token is stored as an app secret and revoked on `destroy`. +- Tests keep secrets and personal data out of the repo and out of logs. -[BeCloudReady](https://becloudready.com/), a Databricks Registered Partner. [Book a discovery call](https://calendly.com/kchandank/30-mins-meeting) for a customized training or consulting engagement. +Early release: try labs in a throwaway workspace or account first. Apache-2.0. See [SECURITY.md](SECURITY.md) and [CONTRIBUTING.md](CONTRIBUTING.md). -Early release. Try labs in a throwaway workspace or AWS account first. Apache-2.0. See [SECURITY.md](SECURITY.md) and [CONTRIBUTING.md](CONTRIBUTING.md). +Maintained by [BeCloudReady](https://becloudready.com/). diff --git a/labagent/config.py b/labagent/config.py index 0dacebb..828fdd8 100644 --- a/labagent/config.py +++ b/labagent/config.py @@ -12,10 +12,8 @@ def getenv(name: str) -> Optional[str]: - """LABAGENT_, falling back to the legacy LABCTL_ used before the rename.""" - return os.environ.get(f"LABAGENT_{name}") or os.environ.get(f"LABCTL_{name}") or None -# Keys from the earlier boto3-only design: accepted and ignored so old config files keep loading. -LEGACY_KEYS = {"group_name", "policy_name", "lambda_role_name", "policy_file"} + """The value of LABAGENT_, or None when unset or empty.""" + return os.environ.get(f"LABAGENT_{name}") or None @dataclass @@ -67,8 +65,6 @@ def load_config(path: Optional[str] = None) -> Config: if path: raw = json.loads(Path(path).read_text()) known = {f.name for f in fields(Config)} - for k in LEGACY_KEYS & set(raw): - raw.pop(k) unknown = set(raw) - known if unknown: raise ValueError(f"unknown config keys: {sorted(unknown)}") diff --git a/labagent/deleters.py b/labagent/deleters.py index 34b662d..b98fa50 100644 --- a/labagent/deleters.py +++ b/labagent/deleters.py @@ -274,10 +274,6 @@ def cleanup_types(): return list(REGISTRY.keys()) -def order_of(r: Resource) -> int: - return REGISTRY.get(r.key, (None, 999))[1] - - # ---- additional types found in real accounts --------------------------------------------------- @guarded diff --git a/labagent/run.py b/labagent/run.py index 69773ce..c4e4189 100644 --- a/labagent/run.py +++ b/labagent/run.py @@ -6,7 +6,7 @@ from .context import Ctx from .discovery import Scope from .envelope import Envelope -from . import audit as audit_mod, cleanup as cleanup_mod, iam_ops, sweep +from . import cleanup as cleanup_mod, iam_ops, sweep COMMANDS = {"provision", "inventory", "sweep", "reset", "offboard", "audit", "cleanup"} diff --git a/pyproject.toml b/pyproject.toml index f642d36..12837b0 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -44,7 +44,6 @@ Documentation = "https://github.com/becloudready/labagent/tree/main/docs" [project.scripts] labagent = "labagent.cli:main" -labctl = "labagent.cli:main" # deprecated alias from before the rename [tool.setuptools] include-package-data = false diff --git a/tests/test_cloudfront_and_cli.py b/tests/test_cloudfront_and_cli.py index 1161cee..924ce0f 100644 --- a/tests/test_cloudfront_and_cli.py +++ b/tests/test_cloudfront_and_cli.py @@ -108,7 +108,7 @@ def update_distribution(self, Id, IfMatch, DistributionConfig): assert origins["c"]["DomainName"] == "api.example.org" # non-S3 untouched -def test_cli_bootstrap_path_and_legacy_alias(capsys): +def test_cli_bootstrap_path(capsys): from labagent import cli assert cli.main(["bootstrap-path"]) == 0 out = capsys.readouterr().out.strip() diff --git a/tests/test_packaging.py b/tests/test_packaging.py index ea7ea48..103fc4f 100644 --- a/tests/test_packaging.py +++ b/tests/test_packaging.py @@ -29,3 +29,10 @@ def test_every_runtime_data_file_is_covered_by_package_data(): def test_the_dataset_registry_pin_is_shipped(): assert any(fnmatch.fnmatch("packs/rag/sources/datasets.json", p) for p in patterns()) + + +def test_only_the_labagent_command_is_installed(): + """`labctl` is another project's name on PyPI: installing labagent must not put a command with that name on PATH.""" + text = (ROOT / "pyproject.toml").read_text() + scripts = re.search(r"\[project\.scripts\](.*?)(\n\[|\Z)", text, re.S).group(1) + assert re.findall(r"^(\S+)\s*=", scripts, re.M) == ["labagent"] diff --git a/tests/test_tf_wrapper.py b/tests/test_tf_wrapper.py index 04b92a8..66d3a76 100644 --- a/tests/test_tf_wrapper.py +++ b/tests/test_tf_wrapper.py @@ -116,11 +116,13 @@ def test_redact(): assert "@" not in redact("failed for a@example.com and c.d@example.org") -def test_legacy_config_keys_are_ignored(tmp_path): +def test_unknown_config_keys_are_rejected_not_ignored(tmp_path): + import pytest from labagent.config import load_config p = tmp_path / "c.json" - p.write_text(json.dumps({"group_name": "old", "policy_name": "old", "name_prefix": "x"})) - assert load_config(str(p)).name_prefix == "x" + p.write_text(json.dumps({"group_name": "old", "name_prefix": "x"})) + with pytest.raises(ValueError, match="group_name"): + load_config(str(p)) def test_module_never_manages_passwords(): @@ -130,13 +132,13 @@ def test_module_never_manages_passwords(): assert not re.search(r'resource\s+"aws_iam_user_login_profile"', text) -def test_legacy_labctl_env_vars_still_work(monkeypatch): +def test_only_labagent_env_vars_are_read(monkeypatch): from labagent.config import getenv monkeypatch.delenv("LABAGENT_TF_BIN", raising=False) - monkeypatch.setenv("LABCTL_TF_BIN", "/legacy/terraform") - assert getenv("TF_BIN") == "/legacy/terraform" + monkeypatch.setenv("LABCTL_TF_BIN", "/old/terraform") + assert getenv("TF_BIN") is None # the pre-rename prefix is no longer honoured monkeypatch.setenv("LABAGENT_TF_BIN", "/new/terraform") - assert getenv("TF_BIN") == "/new/terraform" # new name wins + assert getenv("TF_BIN") == "/new/terraform" def test_packaged_terraform_modules_are_present():