opnsense: add new integration for OPNsense firewall logs

[TocharianOU]

Proposed commit message

OPNsense is an open-source firewall and routing platform based on FreeBSD. This repository ships a pfsense integration but has nothing for OPNsense.

One log data stream collecting filterlog events — the packet filter decisions made by pf — over syslog, with udp, tcp and logfile inputs.

The filterlog body is CSV whose column layout is not fixed:

• It changes with the IP version. IPv4 carries tos, ecn, ttl, id, offset, flags and reports the protocol as number, name; IPv6 carries class, flow, hoplimit and reverses those two columns to name, number.
• It then changes again with the protocol. TCP appends flags, sequence and acknowledgment numbers, window, urgent pointer and options; UDP stops after the payload length; ICMP switches to key=value pairs.

The pipeline parses this positionally in a single script rather than as a stack of grok alternatives, which keeps the column order documented in one place. opnsense.log.tcp.flags is worth calling out: it distinguishes an ordinary SYN connection attempt from an ACK scan.

Checklist

• I have reviewed tips for building integrations and this pull request is aligned with them.
• I have verified that all data streams collect metrics or logs.
• I have added an entry to my package's changelog.yml file.
• I have verified that Kibana version constraints are current according to guidelines.
• I have verified that any added dashboard complies with Kibana's Dashboard good practices

Author's Checklist

• Owning team: @elastic/integration-experience, as recommended in review, in line with the other network and syslog firewall integrations. owner.type is community.
• Please confirm source.license: Apache-2.0 is acceptable — the same question as on crowdsec: add integration for CrowdSec intrusion prevention alerts #21270.
• The changelog.yml link points at this pull request.
• /review-integration from elastic/integration-skills has been run and its findings addressed.

How to test this PR locally

cd packages/opnsense
elastic-package format --fail-fast
elastic-package check
elastic-package test pipeline
elastic-package test script

The pipeline test fixtures are captured syslog from a live OPNsense 26.7, covering both IP versions, the match and ip-option reasons, and tcp, udp, icmp and ipv6-icmp. A second fixture covers the branches the capture did not reach: IPv4 and IPv6 TCP with a FIN/PSH/URG flag combination, non-filterlog messages from the same syslog stream, RFC 3164 framing, and a line with no syslog header. A script test covers corrupt source and destination address columns.

To exercise it end to end, in the OPNsense UI go to System → Settings → Logging / targets, add a target pointing at the agent's listener with Applications including filter, and make sure the rules you want to observe have logging enabled.

Related issues

• Relates crowdsec: add integration for CrowdSec intrusion prevention alerts #21270

Screenshots

The package ships the [Logs OPNsense] Firewall activity dashboard: traffic volume and block rate, activity over time by action and by interface, a port-over-time heatmap where a port sweep reads as a vertical stripe, TCP flag distribution, interface-by-action and protocol-by-direction nested donuts, rule hit ranking, top talkers, and a source map. Screenshots are included in the package under img/.

[elastic-vault-github-plugin-prod]

Reviewers

Buildkite won't run for external contributors automatically; you need to add a comment:

• /test : will kick off a build in Buildkite.

NOTE: https://github.com/elastic/integrations/blob/main/.buildkite/pull-requests.json contains all those details.

[TocharianOU]

Applied the same /review-integration pass that @qcorporation suggested on #21270, since this package was built from the same template and had the same issues.

• format_version → 3.4.2, conditions.kibana.version → ^8.19.0 || ^9.1.0, ECS pinned to git@v9.3.0
• base-fields.yml uses external: ecs; added beats.yml
• Pipeline follows JSE00001 — the syslog grok now reads from event.original. Messages that are not filterlog events still get message repopulated from the parsed body so the line stays readable.
• on_failure reordered to error.message → event.kind → tags
• Added test-common-config.yml, renamed fixtures to convention, and added a variants fixture covering branches the filterlog fixture never reached: IPv4 and IPv6 TCP with a FIN/PSH/URG flag combination, two non-filterlog syslog messages from the same stream, a message in RFC 3164 rather than RFC 5424 format, and a line with no syslog header at all
• Dashboard saved objects stored unstringified so kbdash can read them; UUID identifiers, sentence-case title
• README rewritten against docs/extend/documentation-guidelines.md

elastic-package format, lint, check and test pipeline all pass.

The two open questions from #21270 — source.license: Apache-2.0 and the CODEOWNERS team — apply to this package identically.

[qcorporation]

/test

[qcorporation]

@vera-review-bot review

[github-actions]

Elastic Docs Style Checker (Vale)

Summary: 7 warnings, 6 suggestions found

⚠️ Warnings (7): Fix when the suggestion improves clarity or correctness.

File	Line	Rule	Message
packages/opnsense/_dev/build/docs/README.md	86	Elastic.MenuArrowsBold	Use '→' to separate menu items, not '' or '='. Example: Select Manage index → Add lifecycle policy.
packages/opnsense/_dev/build/docs/README.md	86	Elastic.MenuArrows	Use '→' to separate menu items, not '' or '='. Example: Select Manage index → Add lifecycle policy.
packages/opnsense/_dev/build/docs/README.md	118	Elastic.MenuArrowsBold	Use '→' to separate menu items, not '' or '='. Example: Select Manage index → Add lifecycle policy.
packages/opnsense/_dev/build/docs/README.md	118	Elastic.MenuArrows	Use '→' to separate menu items, not '' or '='. Example: Select Manage index → Add lifecycle policy.
packages/opnsense/_dev/build/docs/README.md	118	Elastic.MenuArrows	Use '→' to separate menu items, not '' or '='. Example: Select Manage index → Add lifecycle policy.
packages/opnsense/_dev/build/docs/README.md	191	Elastic.BritishSpellings	Use American English spelling 'acknowledgment' instead of British English 'acknowledgement'.
packages/opnsense/data_stream/log/fields/fields.yml	99	Elastic.BritishSpellings	Use American English spelling 'acknowledgment' instead of British English 'acknowledgement'.

💡 Suggestions (6): Optional style improvements. Apply when helpful.

File	Line	Rule	Message
packages/opnsense/_dev/build/docs/README.md	25	Elastic.Semicolons	Use semicolons sparingly. Consider splitting the sentence or using a comma or conjunction.
packages/opnsense/_dev/build/docs/README.md	137	Elastic.Semicolons	Use semicolons sparingly. Consider splitting the sentence or using a comma or conjunction.
packages/opnsense/_dev/build/docs/README.md	159	Elastic.Semicolons	Use semicolons sparingly. Consider splitting the sentence or using a comma or conjunction.
packages/opnsense/_dev/build/docs/README.md	166	Elastic.Semicolons	Use semicolons sparingly. Consider splitting the sentence or using a comma or conjunction.
packages/opnsense/_dev/build/docs/README.md	189	Elastic.Semicolons	Use semicolons sparingly. Consider splitting the sentence or using a comma or conjunction.
packages/opnsense/changelog.yml	1	Elastic.Versions	Use 'later versions' instead of 'newer versions' when referring to versions.

---

The Vale linter checks documentation changes against the Elastic Docs style guide. To use Vale locally or report issues, refer to Elastic style guide for Vale.

[infra-vault-gh-plugin-prod]

💔 Build Failed

• Buildkite Build
• Commit: 6a891fe

Failed CI Steps

• ✅ Check go sources

History

[elastic-vault-github-plugin-prod]

✅ All changelog entries have the correct PR link.

[vera-review-bot]

Review summary

Issues found across the latest commits 6a891fe — 1 medium, 2 low

• 🟡 owner.github names an individual account while CODEOWNERS assigns the package to an Elastic team (link) (Unresolved)
• 🔵 The new README uses the retired term "Agentless" (link) (Unresolved)

Package-level:

• 🔵 Proposed commit message

  opnsense: add new integration for OPNsense firewall logs
  
  Add a new community package that collects OPNsense filterlog events, the
  accept and block decisions made by the pf packet filter, over syslog. The
  package has one `log` data stream with `udp`, `tcp` and `logfile` inputs,
  an ingest pipeline that maps events to ECS with vendor detail under
  `opnsense.log.*`, field definitions, pipeline test fixtures captured from
  OPNsense 26.7, and the "[Logs OPNsense] Firewall activity" dashboard.
  
  The filterlog body is CSV whose column layout depends on the IP version
  and then on the transport protocol, so the pipeline parses it positionally
  in a single script rather than a stack of grok alternatives.
  `opnsense.log.tcp.flags` is kept so that a SYN connection attempt can be
  distinguished from an ACK scan.
  
  Relates #​21270
  

Since this is a community PR, a new commit triggers another review — at most once every 30 minutes. I skip the PR while it's approved or has merge conflicts.

🤖 AI-Generated Review | Vera Review Bot - v0.4.1 | 📚 Knowledge base: integration-skills

⚠️ Automated review — verify suggestions before applying.

[qcorporation]

@jamiehynds @narph we need to recommend an owner for this new contribution: can you please help

[TocharianOU]

For context on ownership: every comparable firewall package in the repository is owned by @elastic/integration-experience — including pfsense, which shares OPNsense's filterlog format, as well as fortinet_fortigate, panw, checkpoint, sonicwall_firewall and iptables. I used @elastic/security-service-integrations in CODEOWNERS and the manifest; I will switch both to whichever team you settle on.

[jamiehynds]

@jamiehynds @narph we need to recommend an owner for this new contribution: can you please help

Thanks for the contribution @TocharianOU! @qcorporation I'd say set to integration experience for now inline with the other network/syslog focused integration.

[TocharianOU]

Thanks @jamiehynds — done: the CODEOWNERS entry and owner.github now both name @elastic/integration-experience, with owner.type still community.

[TocharianOU]

@jamiehynds @qcorporation A quick follow-up on this one. Its sibling package from the same template, #21270, has been merged, and the review feedback from there that also applies here is already on this branch: the error message form, the panel and saved search titles, the declared agent fields. The owner is set to @elastic/integration-experience as suggested, and the branch has no conflicts with main.

CI has not run since the owner change. Could someone from the team trigger /test and take a look when there is time? If a system test is wanted here as it was on #21270, I am happy to add one for the udp input.