Robustness Tool — User Guide

Evaluate machine learning models against adversarial attacks. Test your models for robustness and get detailed metrics on what perturbations break them.

---

What This Tool Does

Input: A trained ML model + dataset
Process: Run adversarial attacks (FGSM, PGD, Bayesian optimization, etc.)
Output: JSON metrics + interactive HTML report showing robustness under attack

Supports classification, regression, and multi-output models across sklearn, XGBoost, LightGBM, PyTorch, and MLflow.

---

🚀 Quick Start (5 minutes)

1. Install

pip install -e .

2. Prepare a config file

Copy and edit examples/configs/mlflow/blackbox_pytorch_classification.yaml:

name: "my_robustness_eval"
description: "Testing model robustness"

dataset:
  path: "data/my_data.csv"           # CSV with features + target column
  format: "csv"
  target_column: "target"

model:
  backend: "sklearn"                 # or pytorch, xgboost, lightgbm, mlflow
  path: "models/my_model.joblib"    # Saved model path
  task_type: "classification"        # or regression

robustness:
  attacks:
    - type: "fgsm"
      epsilon: 0.1
    - type: "pgd"
      epsilon: 0.1
      max_iter: 20
  sample_size: 50                    # How many samples to attack

3. Run

python -m runner run --config my_config.yaml

4. View results

# Interactive HTML report
open outputs/report.html

# Raw JSON metrics
cat outputs/metrics.json | python -m json.tool

---

📋 Supported Models

Framework	Format	Status	Notes
scikit-learn	.joblib, .pkl	✅ Full	RandomForest, SVM, LogisticRegression, etc.
XGBoost	.joblib, .pkl	✅ Full	Classification & regression
LightGBM	.joblib, .pkl	✅ Full	Classification & regression
PyTorch	.pt, .pth	✅ Full	Gradient-based white-box attacks
PyTorch Lightning	Checkpoint	✅ Full	Trained model state
MLflow	Model Registry	✅ Full	Any MLflow-registered model

Task types: Classification, Regression, Multi-output regression

---

🎯 Supported Attacks

White-box (requires gradients)

Attack	Best For	Example Params
FGSM	Fast gradient	epsilon: 0.1
PGD	Stronger gradient-based	epsilon: 0.1, max_iter: 20, eps_step: 0.01
BIM	Iterative version of FGSM	epsilon: 0.1, max_iter: 5

When to use: PyTorch models, classification tasks, you need strong attacks.

Black-box (query-based, no gradients)

Attack	Query Efficiency	Use Case
HopSkipJump	Medium	Decision boundaries, sklearn models
Bayesian	High	Regression, small query budgets
Random Search	Low	Baseline, sanity check

When to use: sklearn/tree models, regression, production APIs where gradients unavailable.

---

⚙️ Configuration Reference

Top-level sections

name: "string"              # Unique run name (appears in reports)
description: "string"       # What you're testing

dataset section

dataset:
  path: "string"           # CSV file path
  format: "csv"            # Only CSV supported
  target_column: "string"  # Column name for labels/targets
  # Optional:
  target_columns:          # For multi-output regression
    - "output_1"
    - "output_2"

model section

For sklearn/XGBoost/LightGBM:

model:
  backend: "sklearn"                    # Type
  path: "path/to/model.joblib"         # File
  task_type: "classification"           # or regression
  clip_values: [0.0, 1.0]              # Input bounds

For PyTorch:

model:
  backend: "pytorch"
  path: "path/to/model.pt"
  task_type: "classification"
  input_shape: [n_features]            # e.g., [28, 28] for images
  clip_values: [0.0, 1.0]
  # If using custom scaling:
  scaler_path: "path/to/scaler.pkl"

For MLflow:

model:
  backend: "mlflow"
  mlflow_tracking_uri: "http://localhost:5000"  # or local path
  mlflow_run_id: "abc123def456"                 # From MLflow UI
  mlflow_model_uri: "models:/my-model/production"
  task_type: "classification"
  mlflow_validate_signature: true

robustness section

robustness:
  attacks:
    - type: "fgsm"
      epsilon: 0.1
      norm: "inf"              # or "2" for L2
      targeted: false
    
    - type: "pgd"
      epsilon: 0.1
      eps_step: 0.01           # Step size per iteration
      max_iter: 20             # PGD iterations
      norm: "inf"
      targeted: false
  
  # How many samples to evaluate
  sample_size: 50              # For white-box attacks
  eval_subset_size: 100        # For pure black-box attacks
  
  # Query budget for Bayesian optimization
  query_budget_auto: true      # Auto-calibrate from data
  query_budget_pilot_samples: 5
  query_budget_pilot_percentile: 0.90

Output Files

1. metrics.json — Full technical metrics

Structure:

{
  "report_meta": {
    "family": "white_box",
    "task_type": "classification",
    "attack_name": "Pgd",
    "model_backend": "pytorch",
    "dataset": "path/to/data.csv"
  },
  "attack_setup": {
    "epsilon": 0.1,
    "norm": "inf",
    "steps": 20,
    "samples_evaluated": 50
  },
  "performance_summary": {
    "primary_metrics": [
      {
        "key": "clean_accuracy",
        "label": "Clean Accuracy",
        "value": 0.95,
        "format": "percent"
      },
      {
        "key": "adversarial_accuracy",
        "label": "Adversarial Accuracy",
        "value": 0.78,
        "format": "percent"
      },
      {
        "key": "attack_success_rate",
        "label": "Attack Success Rate",
        "value": 0.22,
        "format": "percent"
      }
    ]
  },
  "charts": {
    "robustness_curve": {
      "type": "line",
      "x_label": "epsilon",
      "y_label": "robust_accuracy",
      "points": [
        [0.05, 0.92],
        [0.1, 0.78],
        [0.2, 0.55]
      ]
    },
    "attack_efficiency": {
      "type": "line",
      "x_label": "queries",
      "y_label": "success_rate",
      "points": [[50, 0.22]]
    },
    "attack_profile": {
      "type": "bar",
      "x_label": "feature",
      "y_label": "mean_perturbation",
      "points": [
        ["feature_5", 0.095],
        ["feature_12", 0.089]
      ]
    }
  },
  "warnings": [
    "Small sample size may not represent full dataset"
  ]
}

2. report.html — Interactive Dashboard

Shows at a glance:

• Clean accuracy vs. adversarial accuracy
• Attack success rate
• Per-feature vulnerability (which features attacked?)
• Robustness curve (how does performance degrade with epsilon?)
• Attack efficiency (queries vs. success for black-box)
• Prediction shifts (how much did outputs change?)

Metrics displayed:

• Clean metrics: Baseline performance on unperturbed data
• Adversarial metrics: Performance under attack
• Robustness metrics: Robust accuracy, attack success rate, accuracy drop
• Perturbation analysis: Feature-level vulnerability, mean perturbation size

3. config_resolved.yaml — Reproducibility

Exact configuration used (with defaults filled in). Use this to reproduce runs exactly.

---

🎛️ Frontend Dashboard

Access via browser: http://localhost:9005

Three main views:

1. Config Upload

• Upload a YAML config file
• View preprocessing steps
• See model loading status

2. Processing (Live)

• Shows evaluation progress
• Real-time status updates
• Estimated time remaining

3. Results Report

Shows interactive visualizations:

• Performance Summary: Cards showing clean accuracy, adversarial accuracy, drop
• Robustness Curve: How accuracy falls as attack strength increases
• Attack Efficiency: For black-box attacks, how many queries needed for success
• Feature Vulnerability: Which features are most attacked
• Per-Sample Perturbations: Magnitude of change per sample
• Warnings: Any issues or limitations

Download button: Export full metrics.json for analysis

---

🔍 Interpreting Results

High robustness (✅ Good)

• Adversarial accuracy ≥ 80% of clean accuracy
• Attack requires large epsilon values
• Few features are vulnerable

Medium robustness (⚠️ Investigate)

• Adversarial accuracy 50-80% of clean accuracy
• Attacks succeed with moderate epsilon
• Specific features are highly vulnerable

Low robustness (❌ Concerning)

• Adversarial accuracy < 50% of clean accuracy
• Small perturbations (epsilon < 0.05) break the model
• Many features are vulnerable

Next steps:

1. Try different attack types — black-box vs white-box may show different weaknesses
2. Check attack_profile — retrain focusing on vulnerable features
3. Increase sample_size — small samples may not represent full dataset
4. Try ensemble methods or adversarial training

---

Common Questions

Q: Which attack should I use?
A: Start with random_search (fast baseline), then try pgd (stronger) if you have PyTorch. For sklearn, use hopskipjump.

Q: My config failed with "timeout"?
A: Reduce sample_size, max_iter, or max_evaluations. Evaluations taking >5 min need optimization.

Q: What does epsilon mean?
A: Maximum allowed perturbation size (in input units). Lower = harder for attacker. For normalized [0,1] inputs, try 0.05-0.2.

Q: Can I test on my production model?
A: Yes! Use MLflow backend and point to your model registry. For API models, treat as black-box attacks.

Q: How many samples do I need?
A: At least 10 for quick tests, 50-100 for reliable results. Larger datasets need more samples for confidence.

Q: Does this work on images?
A: Not directly (this tool is for tabular data). For images, use ART or Foolbox directly.

---

📚 Learn More

• Full architecture details: See docs/ARCHITECTURE.md
• Advanced configurations: See docs/USER_INSTRUCTIONS.md
• Metrics explained: See docs/ADVERSARIAL_ROBUSTNESS_METRICS.md
• MLflow integration: See docs/MLFLOW_GUIDE.md
• Example configs: Browse examples/configs/

---

🐛 Troubleshooting

"Model loading failed"

• Check file path is correct and readable
• Verify model format matches backend (e.g., .joblib for sklearn)
• For MLflow: verify run_id and model_uri in your tracking server

"Attack not supported for this backend"

• White-box attacks (FGSM, PGD) require PyTorch models
• sklearn/tree models automatically use black-box fallbacks
• Check supported attacks table above

"Out of memory"

• Reduce sample_size
• Reduce max_iter for PGD/BIM
• Use fewer attack types in one config

"Results look wrong"

• Check that clip_values match your input bounds
• Verify task_type is correct (classification vs regression)
• Look at config_resolved.yaml to see what actually ran

---

🤝 Support

• Browse example configs: examples/configs/
• Run smoke test: pytest tests/test_smoke.py
• Check logs in outputs/ directory
• File an issue on GitHub with config + error message

Enjoy testing! 🛡️