Philter Diffuse

Philter Diffuse releases PII count aggregations with differential privacy, using OpenDP, which bounds how much a release reveals about any one document.

It uses the Discrete Laplace mechanism to privatize PII counts and prints an audit report with the epsilon (privacy loss) spent and a plain-language reading of it. The tool also features persistent privacy budget tracking to prevent privacy loss exhaustion over time.

The protected unit is one document: one redaction in Philter, or one record counted by the source. A person who appears in many documents can move a count by as many documents as they appear in, and their protection weakens in proportion: epsilon times that number.

Designed to be used with Phileas and Philter.

View the documentation at https://philterd.github.io/philter-diffuse. New users should start with the Quickstart.

Project Status

Philter Diffuse is early-access software at version 0.1.0. The differential-privacy mechanism, budget accounting, and CLI are usable and tested, but as a 0.x release the interface may change before a stable 1.0. See Project Status for the maturity checklist and roadmap. Check your checkout with python main.py --version.

Installation

pip install -r requirements.txt

Usage

Every run names the PII types to release with --pii-types. Choose the list before looking at the counts, from your redaction policy or from what your report needs. The output has exactly those rows whether or not the data holds each type, so it does not reveal which types are present. Each type costs 1 / scale of epsilon.

1. Option A: Philter's API (recommended) With Philter 4.0 or later, recording PII counts:

   export PHILTER_API_KEY=<api key>
   python main.py --philter-url https://localhost:8080 --pii-types ssn,email-address,phone-number --output privatized_counts.csv

   See Reading Philter's PII counts.

2. Option B: JSON file Provide a JSON document with PII entity counts:

   python main.py --input pii_counts.json --source claims-2026 --pii-types ssn,age,zipcode,creditcard --output privatized_counts.csv

   --source names the dataset, and the privacy budget is tracked under that name. Sample pii_counts.json, a flat object of whole counts of 0 or more:

   {
     "ssn": 10,
     "age": 45,
     "zipcode": 12,
     "creditcard": 5
   }

3. Option C: MongoDB Count the documents in the pii_counts collection that have each listed field:

   python main.py --mongo-uri "mongodb://localhost:27017/analytics_db" --pii-types ssn,age,zipcode --output privatized_counts.csv

4. Using Docker Pull the image from Docker Hub:

   docker pull philterd/philter-diffuse:0.1.0

   Or, from a checkout, build the image and run a sample:

   ./run-docker.sh

   This script builds the image and runs a sample privatization task using volume mounts. The image's entrypoint is main.py and it runs in /data, so mount a directory there: it holds the input, the output, and privacy_budget.json, which is otherwise lost when the container exits. See Installation for reading counts from a Philter container.

Privacy Budget Management

Philter Diffuse tracks the total epsilon spent per source in a persistent store. When using MongoDB, it is stored in the privacy_metadata collection. Otherwise (JSON input, or reading counts through Philter's API), it is stored in ~/.philter-diffuse/privacy_budget.json, or the file named by --budget-file or the PHILTER_DIFFUSE_BUDGET_FILE environment variable. JSON input is tracked under its --source name, so a renamed or copied file keeps its budget.

If a release would pass the --budget-ceiling, the tool exits with an error (exit code 1), writes no output, leaves any existing --output file unchanged, and spends no budget.

CLI Options

Option	Description	Required?	Default
--input	Path to a JSON document containing PII entity counts. One of --input, --philter-url, or --mongo-uri is required.	Optional	None
--source	With --input, a name for the dataset the counts come from. The privacy budget is tracked under this name, so keep it the same for every release from that dataset.	With --input	None
--mongo-uri	MongoDB connection URI (format: mongodb://host:port/database).	Optional	None
--pii-types	The PII types to release, comma-separated, such as ssn,email-address. Output has one row per type whether or not the source holds it, and types not listed are left out. Each type costs 1 / scale of epsilon. See Choosing the PII types.	Required	None
--scale	Scale for Laplace noise (higher = more privacy).	Optional	2.0
--output	Path to write the privatized PII counts to a CSV file.	Required	None
--threshold	Counts below this threshold will be output as None.	Optional	0
--budget-ceiling	The maximum total epsilon allowed per source.	Optional	10.0
--budget-file	Where to keep the privacy budget, for sources other than MongoDB.	Optional	PHILTER_DIFFUSE_BUDGET_FILE, then ~/.philter-diffuse/privacy_budget.json
--aggregates	Read Philter's aggregated, document-presence PII counts (summing the count buckets that Philter records per user's (context, day)) instead of counting documents per field. With --philter-url they are read through Philter's API; otherwise from the pii_count_aggregates collection in MongoDB (deprecated).	Optional	off
--context	When using --aggregates, restrict to Philter contexts of this name. Through Philter's API, contexts of that name belong to the API key's user unless Philter's ADMIN_CROSS_USER_ACCESS_ENABLED is on, when they include every user's.	Optional	all contexts
--philter-url	Read the aggregates through Philter's API (Philter 4.0 or later) at this URL, such as https://localhost:8080. Implies --aggregates. The API key is read from the PHILTER_API_KEY environment variable.	Optional	None
--philter-ca-cert	With --philter-url, a PEM file holding a certificate to trust in addition to the system's certificate authorities, such as Philter's self-signed certificate.	Optional	None
--from	With --philter-url, read only counts for this UTC day (YYYY-MM-DD) or later.	Optional	None
--to	With --philter-url, read only counts for this UTC day (YYYY-MM-DD) or earlier.	Optional	None

Reading Philter's PII counts

When Philter is configured to record PII count statistics (its diffuseCountsEnabled admin setting, off by default), it records document-presence counts, one entry per user's (context, day). These are document-presence counts: each redaction contributes at most one per type, so each count has a sensitivity of 1. A redaction can contribute to several types, so a query is charged one 1 / scale per type in --pii-types.

Through Philter's API (recommended, Philter 4.0 or later)

Philter 4.0 serves the counts at GET /api/pii-counts, so Philter Diffuse needs only an API key, not access to Philter's database. Put the key in the PHILTER_API_KEY environment variable (it is never taken as a command-line argument, so it stays out of shell history and process listings, and it is never printed) and pass Philter's URL:

export PHILTER_API_KEY=<api key>
python main.py --philter-url https://localhost:8080 --pii-types ssn,email-address,phone-number --output privatized_counts.csv

• The key must hold the pii-counts:read scope, and its user must be a Philter administrator, because the counts cover the whole deployment. Philter records each read in its audit log.
• Narrow the counts with --context <name> and with --from and --to (UTC days, YYYY-MM-DD, both inclusive). Philter applies these filters, and Philter Diffuse reads every page of the result. Philter keeps each user's counts apart; without --context, Philter Diffuse sums every user's. Through Philter's API, contexts of that name belong to the API key's user unless Philter's ADMIN_CROSS_USER_ACCESS_ENABLED is on, when they include every user's.
• If Philter reports that count recording is turned off, Philter Diffuse says so and exits without writing privatized output or spending privacy budget, rather than privatizing an empty or partial result as if no PII had been found.
• If Philter refuses the request (missing scope, not an administrator, invalid key) or cannot be reached, Philter Diffuse prints the reason and Philter's request ID when one is returned, so the request can be found in Philter's audit log.
• Certificates are verified. Philter's Docker image generates a self-signed certificate; to trust it, export it and pass it with --philter-ca-cert philter.pem rather than turning verification off. Redirects are not followed, so the API key is only ever sent to the URL you give.
• The privacy budget for API reads is tracked in the budget file, not in Philter's database, keyed by the Philter URL (pii_count_aggregates@https://localhost:8080). No MongoDB connection is made. Moving from the MongoDB path starts a new budget entry: copy the epsilon already spent (in MongoDB's privacy_metadata collection) into the budget file so earlier releases still count against the ceiling.

From MongoDB (deprecated)

Philter Diffuse can still read the pii_count_aggregates collection directly from Philter's MongoDB:

python main.py --mongo-uri "mongodb://localhost:27017/philter" --aggregates --pii-types ssn,email-address,phone-number --output privatized_counts.csv

This path is deprecated and prints a warning naming --philter-url. It depends on Philter's internal storage layout and on MongoDB storage, and it will be removed in a future release. Move to --philter-url once you run Philter 4.0 or later.

Community

Users and developers are invited to join the Philterd Slack workspace to ask questions, share feedback, and discuss Philter Diffuse and the rest of the Philterd toolkit.

License

Copyright 2026 Philterd, LLC. "Philterd", "Phileas", and "Philter" are registered trademarks of Philterd, LLC.

This project is licensed under the Apache License, Version 2.0. See the LICENSE file for details.