Skip to content

Connect Linux Endpoints

Once Logster is running — either on OpenShift, or as the App Node + GPU Node appliance — connect your Linux endpoints so they start reporting in.

The endpoint side is identical for both deployments; only the Kafka address and transport you pass in Step 2 differ.

Three components run on a Linux endpoint, all delivered by a single package:

  1. The Logster eBPF sensor — a collector. It observes process, file, and network activity directly in the kernel. Requires a modern kernel (see Before you start).
  2. auditd — a collector. The Linux Audit daemon, driven by the ruleset Logster ships. Runs on every endpoint. Installed as a package dependency; the Logster ruleset is activated automatically, merged alongside any audit rules you already run.
  3. rsyslog — the shipper. It reads both collectors' log files and forwards the events to Logster's Kafka listener — the appliance's :29092 port, or the OpenShift bootstrap route on :443 over TLS. Installed as a package dependency and configured for you.
%%{init: {"theme":"base","themeVariables":{"fontFamily":"Roboto, Helvetica, Arial, sans-serif","fontSize":"15px","lineColor":"#8B6DB8","edgeLabelBackground":"#FFFFFF"},"flowchart":{"curve":"basis","nodeSpacing":45,"rankSpacing":75,"padding":16}}}%%
flowchart LR
    subgraph EP[" Linux endpoint "]
        direction LR
        SN["<b>logster-sensor</b><br><small>eBPF collector</small>"]
        AU["<b>auditd</b><br><small>audit collector</small>"]
        LOGS[("/var/log/ebpf_security/<br>*.log")]
        ALOG[("/var/log/audit/<br>audit.log")]
        RS["<b>rsyslog</b><br><small>shipper</small>"]
        SN -- "writes" --> LOGS
        AU -- "writes" --> ALOG
        LOGS -- "read by" --> RS
        ALOG -- "read by" --> RS
    end
    K["<b>Logster</b><br><small>Kafka listener</small>"]
    RS -- "linux-ebpf-* / linux-auditd-logs<br>TCP 29092 · TCP 443 TLS" --> K

    classDef collector fill:#E8F1FC,stroke:#3B82C4,stroke-width:2px,color:#12324F
    classDef store fill:#FFF6E5,stroke:#D9922B,stroke-width:2px,color:#4A3208
    classDef shipper fill:#EFE6FB,stroke:#7A3FBF,stroke-width:2px,color:#2D0A6B
    classDef sink fill:#E4DAF6,stroke:#5B1FA8,stroke-width:3px,color:#2D0A6B
    class SN,AU collector
    class LOGS,ALOG store
    class RS shipper
    class K sink

Run all of the commands below as root, or with sudo, on the endpoint.


Before you start

The sensor ships as a fully static binary, so it has no library or glibc version requirements. What it does need is a recent kernel with BTF:

Requirement Details
Architecture x86_64
Kernel 5.17 or newer — or RHEL / Rocky / AlmaLinux 9.2+ (kernel 5.14.0-284 or later, which backports the required BPF features)
Kernel BTF /sys/kernel/btf/vmlinux present — standard on every distribution that meets the kernel requirement
Kernel lockdown Not in confidentiality mode (this mode blocks loading BPF programs entirely)

Typical endpoints that qualify: Ubuntu 22.04 with the HWE kernel, Ubuntu 24.04+, Debian 12+, RHEL / Rocky / AlmaLinux 9.2+, current Fedora, Amazon Linux 2023.

You do not need to check any of this by hand — the package verifies every requirement at install time and prints a PASS/FAIL report.

Ubuntu 22.04 needs the HWE kernel

Ubuntu 22.04's GA kernel is 5.15, which is too old. Install the hardware enablement kernel and reboot before continuing:

sudo apt install linux-generic-hwe-22.04

Older kernels

Endpoints below this requirement — RHEL / CentOS 7 and 8, Ubuntu 20.04, Debian 11 — cannot run the eBPF sensor. They are still monitored, with auditd alone — the same install degrades automatically. See Older kernels — auditd-only collection.


Step 1 — Install the sensor package

Copy the package for your distribution onto the endpoint — .deb for Debian and Ubuntu, .rpm for RHEL-family and SUSE — then install it.

Debian / Ubuntu:

sudo apt update
sudo apt install ./logster-sensor_*_amd64.deb

RHEL / Rocky / AlmaLinux:

sudo dnf install ./logster-sensor-*.x86_64.rpm

The leading ./ matters — without it the package manager looks for the name in your configured repositories instead of the local file.

Installing pulls in rsyslog, its Kafka output module, and auditd (RHEL-family and SUSE: audit) automatically, places both collector configurations on the endpoint, activates the auditd ruleset, and finishes by printing the preflight report. Every line should read PASS.

Both configurations come from the public logster-endpoint-configs repository, so you can review either baseline before or after deploying:

Configuration Installed to Status after install
eBPF sensor filter — filter.json /etc/logster/ebpf-sensor/filter.json Active as soon as the sensor starts
auditd ruleset — audit.rules /etc/audit/rules.d/40-logster.rules (source copy at /usr/share/logster-sensor/audit.rules) Active — merged alongside any audit rules you already run; see Step 3

Both are safe to tune for your fleet — see Tuning. For what each one covers, see What auditd monitors and What the eBPF sensor monitors.

Ubuntu: enable the universe repository

The rsyslog-kafka dependency lives in Ubuntu's universe repository. If apt reports it cannot be found:

sudo add-apt-repository universe && sudo apt update

SUSE: install the Kafka module separately

SUSE names the module differently, so install it alongside the package:

sudo zypper install ./logster-sensor-*.x86_64.rpm rsyslog-module-omkafka

Step 2 — Point the endpoint at Logster

One command configures the shipper and starts everything. What you pass it depends on where Logster runs — the broker address and the transport differ:

Deployment Broker address Transport
OpenShift (installed with the Helm guide) Kafka bootstrap route, port 443 TLS — needs the cluster CA (--ca-file)
Appliance (App Node OVA) EXTERNAL_KAFKA_LAN_HOST, port 29092 Plaintext

Follow the one that matches your install.

Step 2a — OpenShift deployments

Kafka is exposed through TLS-passthrough OpenShift routes, so the endpoint must connect over TLS and needs the CA that signed the broker certificate. Strimzi / AMQ Streams signs those certificates with its own cluster CA, published in the <kafka-cluster-name>-cluster-ca-cert secret in the Logster namespace — with the chart's default values, logster-cluster-ca-cert in namespace logster.

1. On a machine with oc access, extract the CA certificate and read the bootstrap address:

# CA bundle -> ./ca.crt
oc extract secret/logster-cluster-ca-cert -n logster --keys=ca.crt --to=. --confirm

# bootstrap address (host:443) for --brokers
oc get kafka logster -n logster \
  -o jsonpath='{.status.listeners[?(@.name=="external")].bootstrapServers}{"\n"}'

If you changed kafka.clusterName or the namespace at install time, substitute yours in both commands — the secret is always <clusterName>-cluster-ca-cert.

2. Copy ca.crt to the endpoint (scp, configuration management, or whatever you already use), then run setup there:

sudo logster-sensor-setup \
  --brokers logster-kafka-bootstrap-logster.apps.<cluster-domain>:443 \
  --ca-file ./ca.crt \
  --enable

Setup copies the bundle to /etc/logster/ebpf-sensor/kafka-ca.crt, so it survives the temp location you copied it to and gets an SELinux label that confined rsyslogd can read. Re-runs should pass --ca-file /etc/logster/ebpf-sensor/kafka-ca.crt rather than the original path.

Both Kafka routes must resolve from the endpoint

After the bootstrap handshake the Kafka client is redirected to the advertised per-broker route (logster-kafka-0-…). Every endpoint must resolve both hostnames in DNS and reach them on port 443, or the connection is established and no data flows. List the exact hostnames with oc get routes -n logster and add them to endpoint DNS (or hosts files) as needed. See B6 — Ship endpoint logs to Logster.

The cluster CA rotates

Strimzi renews the cluster CA periodically. When it does, re-extract ca.crt and re-run logster-sensor-setup --brokers … --ca-file ./ca.crt --enable on your endpoints, or they stop shipping.

If distributing the CA to every endpoint is not practical, --tls-insecure encrypts the channel without verifying the broker — see Shipping over TLS for the trade-off and its limitations.

Step 2b — Appliance (App Node) deployments

Use your appliance's EXTERNAL_KAFKA_LAN_HOST address; this listener is plaintext, so no transport flag is needed:

sudo logster-sensor-setup --brokers <APP_NODE_IP>:29092 --enable

What the command does

This writes the rsyslog shipping configuration, validates it with rsyslogd -N1 before applying it, makes sure auditd is running with the Logster ruleset loaded, starts the sensor, and restarts rsyslog. On SELinux-enforcing hosts it also loads the policy module the shipper needs.

The command is safe to re-run at any time — to change the broker address, for example, or after moving Logster. It is non-interactive and idempotent, so it is safe to drive from Ansible or cloud-init.

Full option list:

Option Meaning
--brokers <host:port>[,<host:port>…] Required. Kafka bootstrap address(es).
--ca-file <path> Ship over TLS and verify the broker chain against this PEM CA bundle. See Shipping over TLS.
--tls-insecure Ship over TLS without verifying the broker certificate.
--enable Start and enable everything now, rather than only writing config.
--no-selinux Skip loading the SELinux policy module.

Before rendering the config, setup probes each broker address on TCP. An unreachable broker is a warning, not an error — configuring an endpoint before Logster is up is legitimate — but it tells you immediately that nothing will ship until the address or firewall is fixed.

On a kernel too old for the eBPF sensor, --enable does not fail: it prints a DEGRADED mode warning, skips the sensor, and still configures auditd shipping — see Older kernels — auditd-only collection.

Verify the sensor is running:

systemctl status logster-sensor

Shipping over TLS

Two transport modes are available for brokers that expose an SSL listener — the OpenShift routes, or any Kafka you have put behind TLS. The appliance's :29092 listener is plaintext and needs neither.

Verified (recommended). --ca-file <path> points at the PEM bundle for the CA that signed the broker certificate — on OpenShift, the Strimzi cluster CA extracted in Step 2a. Setup installs it at /etc/logster/ebpf-sensor/kafka-ca.crt and configures omkafka with security.protocol=ssl + ssl.ca.location, so the broker chain is verified. The file must be PEM — a DER bundle is rejected up front; convert it with openssl x509 -inform der -in <file> -out ca.crt.

Encrypted but unverified. --tls-insecure encrypts the channel and skips certificate and hostname checking:

sudo logster-sensor-setup --brokers <BOOTSTRAP_HOST>:443 --tls-insecure --enable

The two flags are mutually exclusive.

--tls-insecure does not authenticate the broker

The wire is encrypted, but nothing proves you are talking to your own Kafka — there is no protection against an active man-in-the-middle. Prefer --ca-file in production.

RHEL 8 cannot use --tls-insecure

Turning verification off needs librdkafka ≥ 1.2; RHEL 8 ships 0.11.4, which has no such switch. Setup probes the host's librdkafka and refuses rather than rendering a config omkafka would silently disable at startup — an endpoint that looks healthy and ships nothing. Use --ca-file on those hosts. On that same older librdkafka, --ca-file validates the certificate chain but not the hostname; setup prints a note when it detects this.


Step 3 — Review the auditd ruleset

Nothing to run here — the ruleset was applied automatically in Step 1. This section explains what was done, so you can review it.

The ruleset is Logster's published auditd baseline — audit.rules in the logster-endpoint-configs repository. See What auditd monitors for a breakdown of its coverage.

At install time the package:

  • Staged the unmodified baseline at /usr/share/logster-sensor/audit.rules.
  • Generated /etc/audit/rules.d/40-logster.rules from it, with the -D (delete all rules) and -f (failure mode) directives stripped, so your existing rules and site policy survive. augenrules compiles everything under /etc/audit/rules.d/ into the active ruleset — the Logster rules coexist with any baseline you already run; files you maintain there are never modified.
  • On Debian and Ubuntu, set log_group = syslog in /etc/audit/auditd.conf and restarted auditd — rsyslog runs as user syslog there and otherwise cannot read /var/log/audit/audit.log, so no auditd events would reach Logster. (On RHEL-family hosts the SELinux policy module covers this instead.)
  • Enabled auditd and loaded the merged ruleset with augenrules --load.

Verify the rules loaded — the count should be in the hundreds, and the status line reports the backlog and any lost events:

sudo auditctl -l | wc -l
sudo auditctl -s

Edge cases

  • Immutable audit config (-e 2 set by your site policy): rules cannot be changed at runtime, so the merged ruleset loads at the next reboot. The installer prints a warning when it detects this.
  • Rule file ordering: augenrules concatenates /etc/audit/rules.d/*.rules in filename sort order. A site file sorting after 40-logster.rules that itself contains -D will wipe the Logster rules — remove the -D from drop-in files, or rename them to sort first.

Removing the package deletes /etc/audit/rules.d/40-logster.rules and re-runs augenrules --load, restoring your site rules exactly as they were. auditd itself and log_group are left in place.


Verify the endpoint is reporting

Confirm events are being written locally:

sudo tail -f /var/log/ebpf_security/process_monitoring.log

Running any command in a second shell produces an [EXEC] line. To confirm security tagging works end to end, trigger a flagged event — this writes an [EXEC!] line, because /tmp is a staging directory:

cp /bin/true /tmp/logster-test && /tmp/logster-test && rm /tmp/logster-test

Generate a known-good auditd event and confirm it was recorded:

whoami                          # matches the 'recon' rule
sudo ausearch -k recon -ts recent

Within a minute or two the endpoint appears on the Logster Console — the App Node's http://<app-node>:5001, or the dashboard route on OpenShift (oc get route dashboard -n logster).


Older kernels — auditd-only collection

Endpoints running a kernel older than 5.17 that are not RHEL 9.2+ — RHEL / CentOS 7 and 8, Ubuntu 20.04, Debian 11 — cannot run the eBPF sensor. eBPF as the sensor uses it did not exist on those kernels, and no backport makes it available.

These endpoints are still monitored, using auditd alone — and the install is the same one, no separate procedure:

  • Install the package per Step 1. The preflight report will show FAIL lines for the kernel requirements — that is expected; auditd and the merged ruleset are still set up.
  • Run logster-sensor-setup per Step 2. It detects that the eBPF sensor cannot run, prints a DEGRADED mode warning, and continues: auditd and rsyslog are configured and started, only the sensor itself is skipped. The command still exits 0, so configuration management treats the host as converged.
  • After a kernel upgrade, re-run the same logster-sensor-setup --brokers … --enable command to bring the eBPF sensor online. Include the transport flag (--ca-file … / --tls-insecure) if you used one — re-running without it re-renders the shipper as plaintext and silently stops the events this host was shipping. Setup prints the exact command to re-run in its DEGRADED warning.

On these hosts the auditd ruleset is regenerated in a DEGRADED form: general execve auditing, scoped connect/bind, and PATH/CWD records enabled, so auditd substitutes for the missing eBPF coverage. Package upgrades preserve that form; re-running setup once preflight passes converges the host back to the base ruleset.

One caveat: the rsyslog Kafka output module must exist in the distribution's repositories. Where it does not (some very old releases), preflight reports the shipping path as broken and setup refuses to enable anything.

Coverage on these hosts is reduced. auditd still reports process execution, file access, privileged operations, and network syscalls through the shipped ruleset, but you lose the sensor's in-kernel filtering — so event volume is higher — along with the process, file, and network detail the sensor contributes. Contact Logster Support when planning a rollout that includes a significant number of older endpoints.


Endpoint not appearing on the console?

  • Run sudo logster-sensor-preflight. Every line should read PASS; any FAIL explains itself and tells you how to fix it.
  • Check the service: systemctl status logster-sensor. If it refused to start, the reason is in journalctl -u logster-sensor -n 50.
  • Confirm events are reaching disk: sudo sh -c 'wc -l /var/log/ebpf_security/*.log'. If these files are growing but nothing reaches the console, the problem is the shipper, not the collector — check systemctl status rsyslog and journalctl -u rsyslog for Kafka output errors.
  • Confirm the endpoint can reach the broker — <EXTERNAL_KAFKA_LAN_HOST>:29092 on the appliance, or both the bootstrap and per-broker route hostnames on :443 for OpenShift (check DNS and any firewalls in between). Re-running logster-sensor-setup reports unreachable brokers explicitly.
  • On a TLS deployment, a broker that rejects the handshake shows up in journalctl -u rsyslog as an SSL error. Check that the port really is an SSL listener, and — with --ca-file — that /etc/logster/ebpf-sensor/kafka-ca.crt is the CA that signed the broker certificate (on OpenShift, re-extract it; the cluster CA rotates).
  • If /var/spool/logster is growing, the collectors are fine and the shipper is spooling to disk because Kafka is unreachable — see Buffering during an outage.
  • On SELinux-enforcing hosts, confirm the policy module loaded: semodule -l | grep logster-sensor, and check for denials with sudo ausearch -m avc -ts recent.

Reference

The configuration files both collectors are driven by, how to tune them, how the shipper buffers events during an outage, and exactly what each collector monitors are documented in Linux Log Collection Reference.