Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 11 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -35,9 +35,20 @@ labagent destroy rag-sec-tutorial --name demo # removes the uploaded folder
| `rag-sec-tutorial` | Nine notebooks that build RAG on a pinned SEC 10-K dataset one step at a time: embeddings, pgvector, hybrid search, filters, citations, an evaluation gate, and the deployed app. Each ends with a check cell | Databricks Free Edition |
| `db-agent` | [db-agent](https://github.com/db-agent/db-agent) text-to-SQL over a Databricks sample dataset, pinned commit | Databricks Free Edition |
| `rag-ide` | A private browser VS Code per learner with the RAG lab and notebooks | Modal, EC2 or local Docker |
| `aml` | A synthetic anti-money laundering lab with an answer key: a RAG lab for developers (`mode=rag`) and a Microsoft 365 Copilot lab for staff (`mode=copilot`), for Canada or the US. [Guide](docs/aml.md) | Your machine, no cloud account |

Free Edition limits apply (one Lakebase project, three apps). Details: [docs/packs.md](docs/packs.md).

## AML lab (synthetic data, RAG and Copilot)

```bash
pip install 'labagent[rag-local,aml]'
labagent launch aml --name demo --param mode=rag --param jurisdiction=ca # developers: indexed corpus, helper, quality checks
labagent launch aml --name spring --param mode=copilot --param jurisdiction=ca # staff: Excel and Word files, 13 exercises, instructor folder
```

It generates a made-up credit union with 8 laundering patterns, an answer key for every alert, policies, case files and regulation text. The labels come from how the data is built, not from an investigator. See [docs/aml.md](docs/aml.md).

## Web IDE per learner (unreleased: install from GitHub)

```bash
Expand Down
125 changes: 125 additions & 0 deletions docs/aml.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,125 @@
# AML lab: synthetic anti-money laundering data for RAG and Copilot

`labagent launch aml` builds a lab around a made-up credit union, **Maple Ridge Credit Union**. It generates customers, accounts, transfers and alerts, with an answer key for every alert. It also writes the policies, the case files and the regulation text. Every name and number is invented. The laws are real.

Two audiences use the same data:

| Mode | Who | What they get |
|---|---|---|
| `mode=rag` | Developers | An indexed corpus, a Python helper, three quality checks and a lab guide |
| `mode=copilot` | Staff at financial firms learning Microsoft 365 Copilot | A folder of Excel, Word and text files, a plain-English exercise guide, and a separate instructor folder with the answers |

```bash
pip install 'labagent[rag-local]' # rag mode: search and embeddings
pip install 'labagent[aml]' # copilot mode: writes Excel and Word files
labagent launch aml --name demo --param mode=rag --param jurisdiction=ca
labagent launch aml --name spring --param mode=copilot --param jurisdiction=ca --param size=small
labagent destroy aml --name demo
labagent "set up the AML copilot lab for Canada" # plain English; it shows the command and asks first
```

Options: `jurisdiction=ca|us|both`, `size=small|medium`, `seed=42`, `generator=builtin|snapshot`, `regulations=auto|fetch|skip`. Same seed, same data.

## Synthetic AML dataset for RAG

In `mode=rag` the pack writes a lab folder:

| Path | What it holds |
|---|---|
| `data/` | Policy manual, alert procedure, report guide, regulation text and one case file per alert, as Markdown |
| `tables/` | `customers`, `accounts`, `transactions`, `alerts`, `kyc_profiles`, `sanctions_hits` as CSV |
| `answer_key/` | `ground_truth.csv`: the decision and the reason for every alert, kept apart |
| `aml_tools.py` | `AmlLab`: `ask`, `show`, `check`, `triage`, `score` |

The corpus is indexed with the same client as the local RAG lab (`labagent.ai.rag`). Policies and regulations are chunked by section. Case files keep only the summary and the customer profile as text. Transactions stay in tables, because a table is the right shape for them.

Learners do three tasks: ask policy questions and read the citations, pull the **evidence bundle** for an alert, and draft a report narrative that cites real transaction IDs and sections. The bundle holds the customer profile, the alert trigger, the transactions around it, the matching policy and rule text, and similar earlier cases. The design follows the explainable triage paper (arXiv 2604.19755).

### Quality checks

Every launch runs three checks and reports them in the result:

1. **Retrieval hit rate** on 14 policy and regulation questions per country, each with a known answer. Gate: 0.6. With a real embedding model (all-MiniLM-L6-v2, top 5, seed 42) the hit rate was 0.79 to 0.86 for Canada, the United States and both. With the keyword-only fallback embedder it is about 0.43, so the gate is reported but not applied there.
2. **Citation validity.** Every transaction ID, regulation section and policy section a reference narrative cites must exist. Gate: 1.0.
3. **Triage agreement** with the answer key on 12 alerts (4 per label). Reported only. A fixed reference rule agrees on 99.9 percent of 1,152 alerts across 16 generated datasets. That shows the evidence is enough to decide. It does not measure a model. Set `llm=env` to score your chat model the same way.

## AML transaction monitoring data with labels

The built-in generator is pure Python, needs no Java and no network, and is deterministic for a seed.

| Size | Customers | Accounts | Transfers | Alerts |
|---|---|---|---|---|
| `small` | 300 | 429 | about 42,000 | 72 |
| `medium` | 2,000 | 2,889 | about 281,000 | 552 |

Normal activity comes first: payroll, rent, card spending, business sales and supplier payments. Labelled cases are laid on top:

- **The 8 laundering patterns** from Altman et al. (2023): fan-out, fan-in, gather-scatter, scatter-gather, cycle, random, bipartite and stack.
- **Two disguised-payment cases**: illegal money paid out as payroll, or as supplier invoices just under the 10,000 limit.
- **Name matches** to a fixed set of 12 entity names from the OFAC SDN list. Fuzzy variants are included. Every row is tagged TEST DATA.
- **Monitor look-alikes**: many small gifts, cash at a cash business, a loan and its repayment, a payment passed to a supplier.
- **Dismiss cases**: a house sale, a bonus, a repeat family payment, a tuition wire.

Each alert has a label (`escalate`, `monitor` or `dismiss`), a pattern, the accounts and transactions involved, and a plain-English reason. In the small set: 22 escalate, 20 monitor and 30 dismiss.

**The labels come from how the data was built, not from an investigator.** They show what a good answer looks like. They do not show that a model which scores well here will do well on a real bank's data.

Tables are written as CSV, and as Parquet when `pyarrow` is installed. The schema is the same for every generator.

## Copilot training data for banking

In `mode=copilot` the pack writes two folders and two zips:

```
spring/ learner folder (spring.zip)
Policies/ policy manual, alert procedure, report guide (.docx)
Regulations/ excerpts of the rules, and FINTRAC guidance if fetched (.docx)
Cases/ one Excel workbook and one Word file per case, plus a case list
Data/ all transfers, customers, accounts, alerts; one file per month
Exercise Guide.docx 13 exercises, no answers
spring-instructor/ instructor folder (spring-instructor.zip)
Exercise Guide with Answers.docx
Instructor Answer Key.docx
```

Every workbook holds real Excel tables, because Copilot in Excel works on tables. Learners upload the folder to OneDrive or SharePoint and turn on AutoSave. Microsoft documents these limits: Copilot in Excel needs `.xlsx` files in OneDrive or SharePoint with AutoSave on, files under 150 MB, and at most five files in one multi-file action. The guide tells learners about each one.

The 13 exercises cover four parts: policies, regulations, cases and data. Examples: "Summarize this case and list the red flags", "Which section covers cash deposits just under 10,000?", "Find all transfers over 5,000 to new payees in March". Each part ends with a **catch the AI's mistake** exercise: a short AI-style draft with planted errors that learners find by checking the files. The expected answers are computed from the data.

The learner files use no technical terms (no RAG, vectors, Databricks or code). The launch fails if a learner file contains one, if an instructor file is in the learner folder, or if a workbook has no table.

I have not run these files against a live Microsoft 365 Copilot tenant. Check the large Transactions file in your tenant before a class, and use the monthly files if Copilot struggles.

## FINTRAC STR narrative practice

For Canada, the guide to writing the narrative asks six questions: who, what, when, where, why it is suspicious, and how the money moved. The rules the pack cites were read from the real text:

- PCMLTFA s. 7 requires a report when there are reasonable grounds to suspect.
- PCMLTF STR Regulations s. 9: send the report as soon as practicable after the measures that let you establish those grounds.
- PCMLTF Regulations s. 10 and s. 132: a financial entity keeps a record of $10,000 or more in cash, and reports it within 15 days.
- PCMLTF Regulations s. 148: keep records at least five years.

The US version cites 31 CFR 1020.320 (a SAR within 30 calendar days of detecting the facts), 31 CFR 1010.311 (currency transactions of **more than** $10,000) and 31 CFR 1010.306 (15 days). Note the difference: Canada says "$10,000 or more", the United States says "more than $10,000". The data contains cases where that difference matters.

In `mode=rag`, `lab.check(text)` tests a narrative: every transaction ID and section it cites must exist. Section numbers are checked against `labagent/packs/aml/data/reg_index.json`.

## Sources and licences

| Source | How it is used | Licence |
|---|---|---|
| Built-in generator, policies, case files | Written by labagent | CC BY 4.0 for published data; Apache-2.0 for code |
| OFAC SDN list | 12 entity names, kept in the repository with their list uids and the list date | US government work (public domain) |
| eCFR, 31 CFR Chapter X parts 1010 and 1020 | Downloaded at launch from govinfo. Redistributable, so it may go in a snapshot | US government work. The eCFR is an unofficial compilation |
| PCMLTFA and two regulations (Justice Laws XML) | Downloaded at launch. Redistributable with attribution | Open Government Licence - Canada. Not the official version |
| FINTRAC indicators and STR guidance | Downloaded on the instructor's machine at launch, never committed | FINTRAC's terms (not reviewed) |
| IBM AMLSim and Altman et al. (arXiv 2306.16424) | Source of the 8 patterns and the idea of disguised payments. No IBM code or data is included | AMLSim is Apache-2.0 |
| Triage paper (arXiv 2604.19755) | Design of the evidence bundle and the three labels. No code or data used | CC BY 4.0 (paper) |

A snapshot can only contain sources marked "bundled" in `labagent/packs/aml/licences.py`. Building a snapshot with a fetch-only source fails. OpenSanctions (CC BY-NC 4.0) is not used. FFIEC manual sections, the IBM Kaggle dataset, SAML-D and Elliptic2 are not used either: their redistribution terms were not verified.

## Not built yet

- Databricks Lakebase platform and the `rag-ide` option.
- `generator=amlsim` (run IBM AMLSim in Docker) and the optional importers.
- A published `aml-synthetic` snapshot. Until it exists, `generator=snapshot` stops with a clear message.
- Upload to SharePoint through Microsoft Graph.
76 changes: 76 additions & 0 deletions docs/aml/dataset-card-huggingface.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,76 @@
---
license: cc-by-4.0
language:
- en
pretty_name: Synthetic AML Transactions with Ground Truth (Canada and US)
size_categories:
- 10K<n<100K
task_categories:
- tabular-classification
- question-answering
tags:
- finance
- anti-money-laundering
- aml
- synthetic
- transaction-monitoring
- fintrac
- bank-secrecy-act
- rag
---

# Synthetic AML Transactions with Ground Truth (Canada and US)

Draft dataset card. The release is not published yet. Replace the placeholders marked `TODO` after you publish it.

A synthetic anti-money laundering dataset for a made-up credit union, Maple Ridge Credit Union. It has customers, accounts, transfers, alerts, a short profile for each customer, name-screening results, and an answer key for every alert. Policies, case files and exercises are built from the same data by the [labagent](https://github.com/becloudready/labagent) `aml` pack.

## What is in it

| File | Rows (small set) | Holds |
|---|---|---|
| `customers` | 300 | Synthetic name, person or business, occupation or industry, country, risk rating, expected monthly activity, onboarding date |
| `accounts` | 429 | Account, owner, type, open date, branch |
| `transactions` | about 42,000 | Time, from, to, amount, currency, channel (wire, cash, e-transfer, card, crypto off-ramp), description |
| `alerts` | 72 | Rule triggered, account, date, amount, risk score |
| `ground_truth` | 72 | Label (escalate, monitor, dismiss), pattern, accounts and transfers involved, reason |
| `kyc_profiles` | 300 | One short text profile per customer |
| `sanctions_hits` | 6 | Customer names seeded to match 12 OFAC SDN entity names, tagged TEST DATA |

A `medium` set has 2,000 customers, about 281,000 transfers and 552 alerts. Files are CSV, and Parquet when available. Generated with seed 42 unless the release says otherwise.

## How it was made

A seeded Python generator creates normal activity first (payroll, rent, card spending, business sales, supplier payments). It then adds labelled cases:

- the 8 laundering patterns from Altman et al. (2023), arXiv 2306.16424: fan-out, fan-in, gather-scatter, scatter-gather, cycle, random, bipartite and stack;
- laundering disguised as payroll or supplier invoices, as described in the same paper;
- name matches to OFAC SDN entity names, exact and slightly misspelled;
- look-alikes that deserve a closer look (monitor), such as many small gifts or a loan and its repayment;
- ordinary events that trigger a rule for a harmless reason (dismiss), such as a house sale.

The patterns follow IBM's [AMLSim](https://github.com/IBM/AMLSim) (Apache-2.0) and the paper. No IBM code or data is included. The evidence-bundle design follows arXiv 2604.19755.

## Intended uses

- Teaching and testing retrieval-augmented generation over policies, regulations and case files.
- Practising alert triage, narrative writing and citation checking.
- Training non-technical staff to use Microsoft 365 Copilot on ordinary Excel and Word files.

## Limits

- **Labels come from how the data was built.** No investigator reviewed them. A model that scores well here may not do well on real data.
- Amounts, timing and customer behaviour are simplified. Real laundering is rarer and messier.
- Names are invented. A name may match a real person by chance.
- The sanctions names are real entity names from a US government list, used only to practise screening. The list changes. This is not a current list.
- It covers Canada and the United States. Rules cited in the documents were read from the real text on the date in each file. They are not legal advice.

## Licence

Data: CC BY 4.0. Code: Apache-2.0. Regulation text, if you download it with the pack, keeps its own terms: US government work for 31 CFR Chapter X, and the Open Government Licence - Canada for the PCMLTFA and its regulations. FINTRAC guidance is downloaded on your machine and is not part of this dataset.

## Cite

Altman, E., Blanu拧a, J., von Niederh盲usern, L., Egressy, B., Anghel, A., Atasu, K. (2023). Realistic Synthetic Financial Transactions for Anti-Money Laundering Models. arXiv:2306.16424. Please cite it if you use the patterns.

Dataset: BeCloudReady, labagent AML pack. TODO: release URL and version.
28 changes: 28 additions & 0 deletions docs/aml/kaggle-description.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
# Kaggle dataset description (draft, not published)

**Title:** Synthetic AML Transactions with Ground Truth (Canada and US)

**Subtitle:** Customers, accounts, transfers, alerts and an answer key for a made-up credit union. For RAG, triage and Copilot practice.

**Description**

This dataset is synthetic. Maple Ridge Credit Union does not exist, and every name and number is invented.

It has seven tables: customers, accounts, transactions, alerts, ground truth, customer profiles and name-screening results. The small set has 300 customers, about 42,000 transfers and 72 alerts. Each alert has a decision (escalate, monitor or dismiss), a pattern and a plain-English reason.

The laundering cases use the 8 patterns from the IBM paper "Realistic Synthetic Financial Transactions for Anti-Money Laundering Models" (fan-out, fan-in, gather-scatter, scatter-gather, cycle, random, bipartite, stack), plus laundering disguised as payroll or supplier invoices. There are also look-alikes that deserve a closer look and ordinary events that trigger a rule for a harmless reason.

**Labels:** they come from how the data was built, not from an investigator.

**Use it to:**
- test a retrieval-augmented generation app over policies and regulations,
- practise alert triage and report narrative writing,
- teach Microsoft 365 Copilot on Excel and Word files.

**Generate more:** `pip install labagent`, then `labagent launch aml --name demo`. Same seed, same data. See the lab guide in the labagent repository, docs/aml.md.

**Licence:** CC BY 4.0. Credit: BeCloudReady labagent. Patterns after Altman et al. (2023), arXiv 2306.16424, and IBM AMLSim (Apache-2.0). No IBM data is included.

**Tags:** finance, banking, anti-money laundering, synthetic data, tabular data, classification

**Link back:** https://github.com/becloudready/labagent
Loading
Loading