Skip to content
PetitPrinc3Public

About

A light, high-performance and secure data pipeline written in Rust, designed to ingest real-time UDP multicast traffic and efficiently relay it to an Apache Kafka broker.

Resources

Contributing

Security policy

Stars

0 stars

Watchers

0 watching

Forks

Repository files navigation

Nothing But Data

Nothing But Data

The No-Bullshit Daemon.



Nothing But Data (NBD) is a daemon that relays UDP multicast traffic to an Apache Kafka broker. It is written in Rust with unsafe code forbidden, and it can run without root privileges or Linux capabilities.

NBD doesn't authenticate the multicast feed: it relays what it receives and assumes a trusted network segment. See SECURITY.md for the threat model and the self-assessed ANSSI Secure Rust Guide compliance matrix.

Important

While this project was written by a human and not vibe-coded, its documentation was generated by Claude Opus 4.6 (Thinking), Sonnet 5, Sonnet 5.5 and Opus 5.5.

Warning

0.2.0-beta refuses configuration files written for 0.1.x. See the migration notes.

How it works

Each [[provider]] of the configuration maps one multicast group and port to one Kafka topic.

Each multicast group is received by a provider, which submits its datagrams to a sink. In production, the sink delivers them to the Kafka broker through librdkafka. An optional Prometheus exporter runs alongside.

A provider is a Tokio task that owns one UDP socket. It reads datagrams into a preallocated buffer and submits them to Kafka without blocking. When the producer queue is full, the datagram is dropped and counted.

All providers share one Kafka producer. A supervisor replaces it when librdkafka reports a fatal error. If it can't be rebuilt within recovery_timeout, NBD exits with an error. See Producer recovery.

At startup, NBD validates the configuration, creates the sockets, checks that the broker answers, and only then joins the multicast groups.

Getting started

Requirements

  • Rust, pinned by rust-toolchain.toml
  • A C/C++ toolchain and CMake, to build librdkafka
  • A Kafka broker

The optional features need system libraries, see CONTRIBUTING.md.

Build

cargo build --locked --release --features kafka-tls,metrics-exporter
Feature Adds
kafka-tls TLS to the broker, and the SASL SCRAM mechanisms
kafka-auth-gssapi SASL Kerberos. Doesn't enable TLS: combine it with kafka-tls
kafka-auth-oauth SASL OAuth 2.0/OIDC. Enables kafka-tls
metrics-exporter Prometheus endpoint

Without kafka-tls, NBD can only talk to Kafka in plaintext.

Released binaries

GitHub Releases provides Linux and Windows binaries, with and without the metrics exporter. They have no TLS support and only SASL PLAIN: build from source for anything that crosses an untrusted link.

sha256sum --check --ignore-missing SHA256SUMS
gh attestation verify nbd-<tag>-linux-x86_64 --repo PetitPrinc3/nbd

Run

A minimal config.toml, which talks to Kafka in plaintext:

[kafka]
broker = "192.168.1.100:9092"

[[provider]]
topic = "market-data-feed"
group = "239.255.0.1"
port = 20001
interface = "eth0"

The file and its directory must not be writable by group or others.

nbd --check-config-file config.toml   # validate without starting
nbd --config-file config.toml
Option Description
--config-file <PATH> Configuration file. Without it, NBD logs an error and uses ./config.toml
--check-config-file <PATH> Validate a configuration file and exit
--about Show version, features, build date and architecture
-V, --version Show version and features
-h, --help Show usage

--config-file, --check-config-file and --about are mutually exclusive. The exit status is 0 on success, 1 on error, and 2 on invalid arguments.

Documentation

Document Content
CONFIGURATION.md Parameters, validation rules, secret sources, file permissions
PRODUCTION.md Host setup, network, tuning trade-offs, secrets, recovery, monitoring
SECURITY.md Vulnerability reporting, threat model, limitations, ANSSI compliance matrix
PERFORMANCE.md Hot path, benchmarks and results
CONTRIBUTING.md Development setup, tests, CI, pull requests
CHANGELOG.md Release history

Platforms

Linux x86_64 is the reference platform. Windows x86_64 is supported, with these differences:

  • interface is an interface index, not a name
  • sockets are bound to the wildcard address, without per-group or per-interface isolation
  • file permission checks only log a warning
  • a graceful shutdown is triggered by Ctrl+C only

License

GPL-3.0-or-later.


Built with 🦀 by PetitPrinc3

About

A light, high-performance and secure data pipeline written in Rust, designed to ingest real-time UDP multicast traffic and efficiently relay it to an Apache Kafka broker.

Resources

Contributing

Security policy

Stars

0 stars

Watchers

0 watching

Forks

Releases

Contributors

Languages