Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
26 changes: 21 additions & 5 deletions .env.example
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,10 @@ SPLUNK_TOKEN=
# $XDG_STATE_HOME/vct-splunk/audit.log, else ~/.local/state/vct-splunk/audit.log).
# VCT_SPLUNK_AUDIT=/path/to/audit.log

# Required for any real write on every backend. --dry-run sends no request and
# does not need this opt-in. This is never a CLI flag.
# SPLUNK_ENABLE_WRITES=true

# Initial password for `splunk user create` — a secret, so it is read from the
# environment (or an interactive prompt), never accepted as a CLI flag.
# SPLUNK_USER_PASSWORD=
Expand All @@ -57,10 +61,10 @@ SPLUNK_TOKEN=
# VCT_SPLUNK_CONFIG=

# --- Splunk Cloud (ACS) ------------------------------------------------------
# Read-only this release. The backend is deduced from SPLUNK_URL: on a
# *.splunkcloud.com host, supported reads route via the ACS API automatically
# (there is no flag or variable to pick a backend). `splunk inspect` reports
# what the deduced backend supports.
# The backend is deduced from SPLUNK_URL: on a *.splunkcloud.com host,
# supported reads route via the ACS API automatically (there is no flag or
# variable to pick a backend). `splunk inspect` reports what the deduced
# backend supports.

# ACS authentication token (Bearer). The stack name is derived from SPLUNK_URL;
# set SPLUNK_ACS_STACK only to override it.
Expand All @@ -71,6 +75,17 @@ SPLUNK_TOKEN=
# FedRAMP stacks use https://admin.splunkcloudgc.com.
# SPLUNK_ACS_BASE_URL=

# Cloud writes need SPLUNK_ENABLE_WRITES=true (above) AND this, and only unlock
# create/update/delete for index, role, and hec-token -- enable/disable and
# every other resource stay refused regardless. Never a CLI flag. Checked even
# for a --dry-run preview, since it also proves the object is one of the three
# ACS-writable resources. Previews also need an ACS token to build the request.
# SPLUNK_CLOUD_WRITE=true

# Optional: a separate ACS token scoped to writes only. Falls back to
# SPLUNK_ACS_TOKEN when unset; reads always use SPLUNK_ACS_TOKEN, never this one.
# SPLUNK_ACS_WRITE_TOKEN=

# Hide the Cloud stack name at untrusted output boundaries (e.g. CI logs) in
# prompts, rendered JSON/table output, and error messages and details. The audit
# log is unaffected -- it always records the real host.
Expand All @@ -82,7 +97,8 @@ SPLUNK_TOKEN=
# SPLUNK_INTEGRATION_TEST=true
# SPLUNK_WRITE_TEST=true

# Enables the read-only Cloud ACS canary.
# Enables the Cloud ACS integration suites (read and write). Cloud write tests
# also require SPLUNK_CLOUD_WRITE=true, same as the CLI itself.
# SPLUNK_ACS_LIVE_TEST=true

# Server-side staging directory used only by disposable Enterprise write tests.
Expand Down
43 changes: 0 additions & 43 deletions .github/scripts/detect-cloud-stack.sh

This file was deleted.

46 changes: 33 additions & 13 deletions .github/workflows/cloud-read.yml
Original file line number Diff line number Diff line change
@@ -1,16 +1,29 @@
# Certify the Cloud read path against a real Splunk Cloud stack.
#
# The Cloud contract itself is proved on every pull request, credential-free, by
# tests/unit/test_acs_loopback.py and tests/unit/test_cloud_write_refusal.py.
# tests/unit/test_acs_loopback.py and tests/unit/test_write_gating.py.
# What only a live stack can show is that a real one answers the way the ACS
# specification says it does. Until a stack is configured, this workflow ends as
# an explicit no-op rather than a green run that checked nothing.
# an explicit no-op rather than a green run that checked nothing. The write
# canary calls it with `require_credentials: true`, where a no-op fails instead.
name: Splunk Cloud Read Canary

on:
schedule:
- cron: "41 6 * * 3"
workflow_dispatch:
workflow_call:
inputs:
require_credentials:
description: Fail instead of no-op when no Cloud read credentials are configured
type: boolean
default: false
secrets:
SPLUNK_URL:
SPLUNK_ACS_TOKEN:
SPLUNK_ACS_STACK:
SPLUNK_ACS_BASE_URL:
SPLUNK_TOKEN:

permissions:
contents: read
Expand All @@ -33,52 +46,59 @@ jobs:
# Untrusted output boundary: hide the Cloud stack name in prompts, JSON
# and table output, and error messages and details.
VCT_SPLUNK_REDACT_TARGET: "1"
CLOUD_CONFIGURED: ${{ secrets.SPLUNK_URL != '' || secrets.SPLUNK_ACS_TOKEN != '' || secrets.SPLUNK_TOKEN != '' }}
CLOUD_READY: ${{ secrets.SPLUNK_URL != '' && secrets.SPLUNK_ACS_TOKEN != '' && secrets.SPLUNK_TOKEN != '' }}
PYTEST_ADDOPTS: >-
-q --tb=line -r N -o 'addopts=--strict-markers --import-mode=importlib'
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7
with:
persist-credentials: false

# The secrets context is not available to a job-level `if`, so this is a
# step that reports what can be certified rather than a skipped job.
- name: Check for a configured Cloud stack
id: stack
run: bash .github/scripts/detect-cloud-stack.sh
- name: Require complete Cloud read credentials when configured
if: env.CLOUD_READY != 'true'
env:
CLOUD_REQUIRED: ${{ inputs.require_credentials == true }}
run: |
if [ "$CLOUD_CONFIGURED" = "true" ] || [ "$CLOUD_REQUIRED" = "true" ]; then
echo "A partial Cloud read credential set is configured; SPLUNK_URL, SPLUNK_ACS_TOKEN, and SPLUNK_TOKEN are all required."
exit 1
fi
echo "::notice::No Cloud read credentials are configured; no stack was certified."

- if: steps.stack.outputs.ready == 'true'
- if: env.CLOUD_READY == 'true'
uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7
with:
python-version: "3.14"
cache: pip

- name: Install project
if: steps.stack.outputs.ready == 'true'
if: env.CLOUD_READY == 'true'
run: |
python -m venv .venv
.venv/bin/python -m pip install --require-hashes -r requirements-ci.txt
.venv/bin/python -m pip install -e . --no-deps

- name: Cloud reads (every catalogued read command)
if: steps.stack.outputs.ready == 'true'
if: env.CLOUD_READY == 'true'
run: >-
.venv/bin/pytest tests/integration/cloud/read/test_catalog.py
-m "integration and cloud and read" --junitxml=cloud-read.xml

- name: Cloud ACS operations (below the CLI)
if: steps.stack.outputs.ready == 'true'
if: env.CLOUD_READY == 'true'
run: >-
.venv/bin/pytest tests/integration/cloud/read/test_acs_operations.py
-m "integration and cloud and read" --junitxml=cloud-acs.xml

- name: Publish test summary
if: always() && steps.stack.outputs.ready == 'true'
if: always() && env.CLOUD_READY == 'true'
uses: test-summary/action@37b508cfee6d4d080eedd00b5bb240a6a784a6a5 # v2
with:
paths: cloud-*.xml

- name: Upload test report
if: always() && steps.stack.outputs.ready == 'true'
if: always() && env.CLOUD_READY == 'true'
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7
with:
name: cloud-read-${{ github.run_id }}
Expand Down
123 changes: 123 additions & 0 deletions .github/workflows/cloud-write.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,123 @@
# Certify the Cloud write path end-to-end against a real, non-production
# Splunk Cloud stack: a `read` job first (the Splunk Cloud Read Canary, called
# as a reusable workflow), then a destructive `write` job gated
# behind a protected `splunk-cloud-write` GitHub Environment with required
# reviewers. The approval pause sits in front of the destructive half only --
# a credential or connectivity problem surfaces in `read` before anyone is
# asked to approve anything. One dispatch is the full read-then-write
# certification.
#
# Never runs on pull_request or a schedule -- workflow_dispatch only, and even
# then the confirm job fails closed unless `confirm` is typed exactly `WRITE`
# and HEAD's commit subject starts with `tests: splunk cloud write`. The Cloud
# write contract itself (opt-in gating, the ACS-writable allowlist) is proved
# credential-free on every pull request in tests/unit/test_write_gating.py;
# what only a live stack can show is that a real one answers the way that
# contract expects.
name: Splunk Cloud Write Canary

on:
workflow_dispatch:
inputs:
confirm:
description: Type WRITE to run destructive ACS tests on the configured stack
required: true
type: string

permissions:
contents: read

concurrency:
group: splunk-cloud-write
cancel-in-progress: false

jobs:
confirm:
name: Confirm WRITE
runs-on: ubuntu-latest
timeout-minutes: 2
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7
with:
persist-credentials: false

- name: Fail closed unless confirm is WRITE
env:
CONFIRM: ${{ github.event.inputs.confirm }}
run: |
if [ "$CONFIRM" != "WRITE" ]; then
echo "confirm must be exactly WRITE"
exit 1
fi

- name: Require the write-canary commit prefix
run: |
subject=$(git log -1 --format=%s)
case "$subject" in
"tests: splunk cloud write"*) ;;
*)
echo "HEAD subject must start with: tests: splunk cloud write"
exit 1
;;
esac

read:
name: Cloud / ACS reads (pre-write readiness)
needs: confirm
uses: ./.github/workflows/cloud-read.yml
with:
require_credentials: true
secrets:
SPLUNK_URL: ${{ secrets.SPLUNK_URL }}
SPLUNK_ACS_TOKEN: ${{ secrets.SPLUNK_ACS_TOKEN }}
SPLUNK_ACS_STACK: ${{ secrets.SPLUNK_ACS_STACK }}
SPLUNK_ACS_BASE_URL: ${{ secrets.SPLUNK_ACS_BASE_URL }}
SPLUNK_TOKEN: ${{ secrets.SPLUNK_TOKEN }}

write:
name: Cloud / ACS writes
needs: read
environment: splunk-cloud-write
runs-on: ubuntu-latest
timeout-minutes: 15
env:
CLOUD_READY: ${{ secrets.SPLUNK_URL != '' && secrets.SPLUNK_ACS_WRITE_TOKEN != '' }}
PYTEST_ADDOPTS: >-
-q --tb=line -r N -o 'addopts=--strict-markers --import-mode=importlib'
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7
with:
persist-credentials: false

- name: Require write secrets when dispatched
if: env.CLOUD_READY != 'true'
run: |
echo "SPLUNK_URL and SPLUNK_ACS_WRITE_TOKEN are required for a WRITE run"
exit 1

- uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7
with:
python-version: "3.14"
cache: pip

- name: Install project
run: |
python -m venv .venv
.venv/bin/python -m pip install --require-hashes -r requirements-ci.txt
.venv/bin/python -m pip install -e . --no-deps

- name: Cloud writes (index, role, hec-token) with undo
env:
SPLUNK_ACS_LIVE_TEST: "true"
SPLUNK_ENABLE_WRITES: "true"
SPLUNK_CLOUD_WRITE: "true"
SPLUNK_URL: ${{ secrets.SPLUNK_URL }}
SPLUNK_ACS_WRITE_TOKEN: ${{ secrets.SPLUNK_ACS_WRITE_TOKEN }}
SPLUNK_ACS_TOKEN: ${{ secrets.SPLUNK_ACS_WRITE_TOKEN }}
SPLUNK_ACS_STACK: ${{ secrets.SPLUNK_ACS_STACK }}
SPLUNK_ACS_BASE_URL: ${{ secrets.SPLUNK_ACS_BASE_URL }}
VCT_SPLUNK_REDACT_TARGET: "1"
VCT_SPLUNK_AUDIT: ${{ runner.temp }}/vct-splunk-audit.log
run: >-
.venv/bin/pytest tests/integration/cloud/write
-m "integration and cloud and write" --junitxml=cloud-write.xml
30 changes: 19 additions & 11 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ shell in `commands/` (the "functional core, imperative shell" pattern). See
the Authorization header per request, `RetryTransport` retries 429/503) and
the envelope-aware `SplunkClient`; `endpoint_factory.py` is the generic CRUD
engine (`EndpointConfig`/`Field`/`Endpoints`); `endpoints/` holds one module
per hand-written operation (`server`, `search`, `jobs`, `saved_searches`,
per hand-written operation (`server`, `search`, `jobs`, `saved_searches`, `config`,
`kvstore`, `hec`, `apps`, `cluster`, `license`, `deploy`, `lookups`,
`datamodel`, `health`, `raw`); `acs/` is the Splunk Cloud ACS
management-plane client.
Expand All @@ -46,8 +46,8 @@ shell in `commands/` (the "functional core, imperative shell" pattern). See
- `src/vct_splunk/output/` — `formatter.py`: JSON/table rendering and the
error envelope.
- `src/vct_splunk/commands/` — Click adapters, one module per hand-written
command group (`server`, `api`, `auth`, `search`, `health`, `inspect`, plus
`saved_search`'s `run`), plus shared plumbing: `context` (the `command`
command group (`server`, `api`, `auth`, `search`, `health`, `inspect`, `config`,
plus `saved_search`'s `run`), plus shared plumbing: `context` (the `command`
decorator and `Ctx`), `write` (the single gated write path), `dispatch`
(routes a few reads to Cloud ACS), and `registry` + `command_factory`
(resource configs declared as data, turned into generated CRUD groups —
Expand All @@ -69,9 +69,13 @@ Two cross-cutting ideas to know about:
require an explicit app and never silently default to `search`.
- **Transparent backend.** When `SPLUNK_URL` points at `*.splunkcloud.com`, a
few reads (`index list`, `role list`, `hec-token list`) route through the
Cloud ACS API and writes are refused; everything else talks to splunkd REST.
The backend is deduced from the URL — there is no flag to pick it. `splunk
inspect` reports what the deduced backend supports, offline.
Cloud ACS API; everything else talks to splunkd REST. Cloud writes need both
`SPLUNK_ENABLE_WRITES=true` (every write, every backend) and
`SPLUNK_CLOUD_WRITE=true`, and only unlock `create`/`update`/`delete` for
`index`, `role`, and `hec-token` (never a CLI flag; enable/disable and every
other resource stay refused regardless). The backend is deduced from the
URL — there is no flag to pick it. `splunk inspect` reports what the deduced
backend supports, offline.

## Conventions

Expand All @@ -85,14 +89,18 @@ Two cross-cutting ideas to know about:

## Safety

- Writes are gated. Every mutation — index lifecycle, saved-search CRUD and
`search cancel`, and all factory-generated create/update/delete/enable/
disable — funnels through one shared path (`commands/write.py`): `--dry-run`
previews the exact request and sends nothing; otherwise it confirms on a TTY
or requires `--yes` when non-interactive (it never hangs on a hidden prompt).
- Writes are gated and disabled by default. Every mutation — index lifecycle,
saved-search CRUD and `search cancel`, and all factory-generated
create/update/delete/enable/disable — funnels through one shared path
(`commands/write.py`): `--dry-run` previews the exact request and sends
nothing, needing no opt-in; a real write needs `SPLUNK_ENABLE_WRITES=true`
first (there is no CLI flag), then confirms on a TTY or requires `--yes`
when non-interactive (it never hangs on a hidden prompt).
- Each applied write is appended to a local audit log: `$VCT_SPLUNK_AUDIT` if
set, else `$XDG_STATE_HOME/vct-splunk/audit.log`, else
`~/.local/state/vct-splunk/audit.log`.
- `--dry-run` needs no `SPLUNK_ENABLE_WRITES` opt-in. A Cloud preview still
needs `SPLUNK_CLOUD_WRITE=true` and an ACS token to build its request.
- `search run` is bounded by default (time window, row cap, timeout) so an agent
cannot trigger an unbounded export by accident.

Expand Down
Loading
Loading