Skip to content

Connect Linux Endpoints

Once the App Node and GPU Node are running, connect your Linux endpoints so they start reporting in.

Three components run on a Linux endpoint, all delivered by a single package:

  1. The Logster eBPF sensor — a collector. It observes process, file, and network activity directly in the kernel. Requires a modern kernel (see Before you start).
  2. auditd — a collector. The Linux Audit daemon, driven by the ruleset Logster ships. Runs on every endpoint. Installed as a package dependency; the Logster ruleset is activated automatically, merged alongside any audit rules you already run.
  3. rsyslog — the shipper. It reads both collectors' log files and forwards the events to the appliance's Kafka listener. Installed as a package dependency and configured for you.
flowchart LR
    subgraph EP["Linux endpoint"]
        SN["logster-ebpf-sensor<br>(collector)"] -- "writes" --> LOGS["/var/log/ebpf_security/*.log"]
        AU["auditd<br>(collector)"] -- "writes" --> ALOG["/var/log/audit/audit.log"]
        LOGS -- "read by" --> RS["rsyslog<br>(shipper)"]
        ALOG -- "read by" --> RS
    end
    RS -- "linux-ebpf-* / linux-auditd-logs topics · TCP 29092" --> K["App Node<br>Kafka listener"]

Run all of the commands below as root, or with sudo, on the endpoint.


Before you start

The sensor ships as a fully static binary, so it has no library or glibc version requirements. What it does need is a recent kernel with BTF:

Requirement Details
Architecture x86_64
Kernel 5.17 or newer — or RHEL / Rocky / AlmaLinux 9.2+ (kernel 5.14.0-284 or later, which backports the required BPF features)
Kernel BTF /sys/kernel/btf/vmlinux present — standard on every distribution that meets the kernel requirement
Kernel lockdown Not in confidentiality mode (this mode blocks loading BPF programs entirely)

Typical endpoints that qualify: Ubuntu 22.04 with the HWE kernel, Ubuntu 24.04+, Debian 12+, RHEL / Rocky / AlmaLinux 9.2+, current Fedora, Amazon Linux 2023.

You do not need to check any of this by hand — the package verifies every requirement at install time and prints a PASS/FAIL report.

Ubuntu 22.04 needs the HWE kernel

Ubuntu 22.04's GA kernel is 5.15, which is too old. Install the hardware enablement kernel and reboot before continuing:

sudo apt install linux-generic-hwe-22.04

Older kernels

Endpoints below this requirement — RHEL / CentOS 7 and 8, Ubuntu 20.04, Debian 11 — cannot run the eBPF sensor. They are still monitored, with auditd alone — the same install degrades automatically. See Older kernels — auditd-only collection.


Step 1 — Install the sensor package

Copy the package for your distribution onto the endpoint — .deb for Debian and Ubuntu, .rpm for RHEL-family and SUSE — then install it.

Debian / Ubuntu:

sudo apt update
sudo apt install ./logster-ebpf-sensor_*_amd64.deb

RHEL / Rocky / AlmaLinux:

sudo dnf install ./logster-ebpf-sensor-*.x86_64.rpm

The leading ./ matters — without it the package manager looks for the name in your configured repositories instead of the local file.

Installing pulls in rsyslog, its Kafka output module, and auditd (RHEL-family and SUSE: audit) automatically, places both collector configurations on the endpoint, activates the auditd ruleset, and finishes by printing the preflight report. Every line should read PASS.

Both configurations come from the public logster-endpoint-configs repository, so you can review either baseline before or after deploying:

Configuration Installed to Status after install
eBPF sensor filter — filter.json /etc/logster/ebpf-sensor/filter.json Active as soon as the sensor starts
auditd ruleset — audit.rules /etc/audit/rules.d/40-logster.rules (source copy at /usr/share/logster-ebpf-sensor/audit.rules) Active — merged alongside any audit rules you already run; see Step 3

Both are safe to tune for your fleet — see Tuning. For what each one covers, see What auditd monitors and What the eBPF sensor monitors.

Ubuntu: enable the universe repository

The rsyslog-kafka dependency lives in Ubuntu's universe repository. If apt reports it cannot be found:

sudo add-apt-repository universe && sudo apt update

SUSE: install the Kafka module separately

SUSE names the module differently, so install it alongside the package:

sudo zypper install ./logster-ebpf-sensor-*.x86_64.rpm rsyslog-module-omkafka

Step 2 — Point the endpoint at your App Node

One command configures the shipper and starts everything. Use your appliance's EXTERNAL_KAFKA_LAN_HOST address:

sudo logster-sensor-setup --brokers <APP_NODE_IP>:29092 --enable

This writes the rsyslog shipping configuration, validates it before applying it, makes sure auditd is running with the Logster ruleset loaded, starts the sensor, and restarts rsyslog. On SELinux-enforcing hosts it also loads the policy module the shipper needs.

The command is safe to re-run at any time — to change the broker address, for example, or after moving the App Node.

On a kernel too old for the eBPF sensor, --enable does not fail: it prints a DEGRADED mode warning, skips the sensor, and still configures auditd shipping — see Older kernels — auditd-only collection.

Verify the sensor is running:

systemctl status logster-ebpf-sensor

Step 3 — Review the auditd ruleset

Nothing to run here — the ruleset was applied automatically in Step 1. This section explains what was done, so you can review it.

The ruleset is Logster's published auditd baseline — audit.rules in the logster-endpoint-configs repository. See What auditd monitors for a breakdown of its coverage.

At install time the package:

  • Staged the unmodified baseline at /usr/share/logster-ebpf-sensor/audit.rules.
  • Generated /etc/audit/rules.d/40-logster.rules from it, with the -D (delete all rules) and -f (failure mode) directives stripped, so your existing rules and site policy survive. augenrules compiles everything under /etc/audit/rules.d/ into the active ruleset — the Logster rules coexist with any baseline you already run; files you maintain there are never modified.
  • On Debian and Ubuntu, set log_group = syslog in /etc/audit/auditd.conf and restarted auditd — rsyslog runs as user syslog there and otherwise cannot read /var/log/audit/audit.log, so no auditd events would reach Logster. (On RHEL-family hosts the SELinux policy module covers this instead.)
  • Enabled auditd and loaded the merged ruleset with augenrules --load.

Verify the rules loaded — the count should be in the hundreds, and the status line reports the backlog and any lost events:

sudo auditctl -l | wc -l
sudo auditctl -s

Edge cases

  • Immutable audit config (-e 2 set by your site policy): rules cannot be changed at runtime, so the merged ruleset loads at the next reboot. The installer prints a warning when it detects this.
  • Rule file ordering: augenrules concatenates /etc/audit/rules.d/*.rules in filename sort order. A site file sorting after 40-logster.rules that itself contains -D will wipe the Logster rules — remove the -D from drop-in files, or rename them to sort first.

Removing the package deletes /etc/audit/rules.d/40-logster.rules and re-runs augenrules --load, restoring your site rules exactly as they were. auditd itself and log_group are left in place.


Verify the endpoint is reporting

Confirm events are being written locally:

sudo tail -f /var/log/ebpf_security/process_monitoring.log

Running any command in a second shell produces an [EXEC] line. To confirm security tagging works end to end, trigger a flagged event — this writes an [EXEC!] line, because /tmp is a staging directory:

cp /bin/true /tmp/logster-test && /tmp/logster-test && rm /tmp/logster-test

Generate a known-good auditd event and confirm it was recorded:

whoami                          # matches the 'recon' rule
sudo ausearch -k recon -ts recent

Within a minute or two the endpoint appears on the Logster Console (http://<app-node>:5001).


Older kernels — auditd-only collection

Endpoints running a kernel older than 5.17 that are not RHEL 9.2+ — RHEL / CentOS 7 and 8, Ubuntu 20.04, Debian 11 — cannot run the eBPF sensor. eBPF as the sensor uses it did not exist on those kernels, and no backport makes it available.

These endpoints are still monitored, using auditd alone — and the install is the same one, no separate procedure:

  • Install the package per Step 1. The preflight report will show FAIL lines for the kernel requirements — that is expected; auditd and the merged ruleset are still set up.
  • Run logster-sensor-setup per Step 2. It detects that the eBPF sensor cannot run, prints a DEGRADED mode warning, and continues: auditd and rsyslog are configured and started, only the sensor itself is skipped. The command still exits 0, so configuration management treats the host as converged.
  • After a kernel upgrade, re-run the same logster-sensor-setup --brokers … --enable command to bring the eBPF sensor online.

One caveat: the rsyslog Kafka output module must exist in the distribution's repositories. Where it does not (some very old releases), preflight reports the shipping path as broken and setup refuses to enable anything.

Coverage on these hosts is reduced. auditd still reports process execution, file access, privileged operations, and network syscalls through the shipped ruleset, but you lose the sensor's in-kernel filtering — so event volume is higher — along with the process, file, and network detail the sensor contributes. Contact Logster Support when planning a rollout that includes a significant number of older endpoints.


Endpoint not appearing on the console?

  • Run sudo logster-sensor-preflight. Every line should read PASS; any FAIL explains itself and tells you how to fix it.
  • Check the service: systemctl status logster-ebpf-sensor. If it refused to start, the reason is in journalctl -u logster-ebpf-sensor -n 50.
  • Confirm events are reaching disk: sudo sh -c 'wc -l /var/log/ebpf_security/*.log'. If these files are growing but nothing reaches the console, the problem is the shipper, not the collector — check systemctl status rsyslog and journalctl -u rsyslog for Kafka output errors.
  • Confirm the endpoint can reach <EXTERNAL_KAFKA_LAN_HOST>:29092 (check firewalls between the endpoint and the appliance).
  • On SELinux-enforcing hosts, confirm the policy module loaded: semodule -l | grep logster-sensor, and check for denials with sudo ausearch -m avc -ts recent.

Reference

The rest of this page documents the configuration files, how to tune them, and exactly what each collector monitors.

Configuration files

Both collectors are driven by a configuration file, and both baselines are published so you can review coverage before deploying:

Collector Configuration On the endpoint
auditd audit.rules /etc/audit/rules.d/40-logster.rules
eBPF sensor filter.json /etc/logster/ebpf-sensor/filter.json

Both files are available in the logster-endpoint-configs repository, alongside the shipper configuration and the Windows baseline. The sensor release package ships both as well, so a standard deployment does not need to download anything.

Both are active as soon as the package is installed. filter.json is laid down as-is; the auditd ruleset is installed as a merge-safe drop-in — the -D/-f directives are stripped so rules you already run survive (see Step 3). Audit rules can overlap with a baseline you already maintain, which is why the applied file and its unmodified source are both kept on the endpoint for review.

Tuning

Both files are safe to tune for your fleet: add a watched path, exclude a noisy in-house process, widen or narrow coverage. Keep local changes in your own configuration management so they survive package upgrades. filter.json is marked as an operator-editable config file and is never overwritten on upgrade.

To apply an updated filter.json — the sensor reads it only at startup, so every change needs a restart:

sudo curl -fsSL -o /etc/logster/ebpf-sensor/filter.json \
  https://raw.githubusercontent.com/eunomatix/logster-endpoint-configs/main/linux/ebpf-sensor/filter.json
sudo systemctl restart logster-ebpf-sensor

For auditd, re-run sudo augenrules --load.

What auditd monitors

Logster ships an auditd ruleset based on Florian Roth's widely-used Linux Audit Daemon — Best Practice Configuration, with Logster-specific tuning applied on top. It contains 328 active rules:

  • 219 path watches — in two shapes: write/attribute change on configuration files, and execution of a binary.
  • 84 syscall rules — matching on syscall, architecture, and outcome.
  • 18 suppression rules and 3 exclusions — noise control.

By intent, the biggest clusters (shown with the rule keys they are tagged with, which appear in the audit records themselves):

Area Keys / coverage
Recon & attacker tooling exec susp_activity (36: wget, curl, nc, ncat, socat, nmap, ss, netstat, ssh/scp/sftp, tcpdump, tshark, rdesktop/xfreerdp, tftp), recon (whoami, id, hostname, uname), sbin_susp (iptables/nft/ifconfig/tcpdump/traceroute/ufw)
Shells / interpreters susp_shell (21: ash, csh, fish, tcsh, tclsh, xonsh, rbash, wish, yash, tmux, clush), shell_profiles (/etc/profile, bashrc, csh., zsh/)
Credentials & accounts etcpasswd, etcgroup, opasswd, passwd_modification, user_modification, group_modification, login, session (wtmp/btmp/lastlog), rootkey (/root/.ssh), sshd
Privilege escalation priv_esc (su, sudo), pkexec, actions (sudoers), rootcmd (execve with euid=0 & auid>=1000)
Persistence cron (9), systemd + systemd_generator (9), init, libpath, systemwide_preloads (ld.so.preload), pam
Kernel / evasion modules (init_module/finit_module/delete_module + insmod/modprobe/rmmod), KEXEC, mac_policy (selinux), auditlog/auditconfig/audittools
Injection code_injection/data_injection/register_injection/tracing (ptrace with a0=0x4/0x5/0x6/0x10)
Fileless anon_file_create (memfd_create)
Reverse shell remote_shell (successful connect from /bin/bash or /usr/bin/bash)
Failed access file_access, file_creation, file_modification (EACCES/EPERM on open/creat/mkdir/link/rename/chmod/setxattr…)
System state mount, swap, time, network_modifications, sysctl, specialfiles (mknod), perm_mod (xattrs)
Software/infra inventory software_mgmt, third_party_software_mgmt (pip/npm/gem/cpan), docker, kubelet, qemu/virtualbox, falcon_*, filebeat, chef/salt/puppet
Misc 32bit_abi (execve/socket/connect on b32), detect_execve_www (execve as euid 33)

The ruleset also carries deliberate suppressions to keep known false-positive sources out of the pipeline. Worth knowing, because they explain events you may expect to see but won't:

  • openat returning EACCES for find, getcap, and ps — a permission-denied flood from routine filesystem scans.
  • execve as root by a logged-in user for date, awk, mawk, gawk, and nawk.
  • The crond_t subject, vmtoolsd, /dev/shm, and /var/lock/lvm.
  • The CWD, PATH, and CRYPTO_KEY_USER record types.

What the eBPF sensor monitors

The sensor attaches to 14 kernel tracepoints covering process execution, file access, file destruction, and network activity. That fixed set defines everything the sensor is able to see. Surviving events are written as text lines to three log files under /var/log/ebpf_security/.

Tracepoint(s) Emits
sys_enter_execve, sys_enter_execveat [EXEC] / [EXEC!] → process_monitoring.log
sys_enter_openat, sys_enter_openat2 [FILE] / [FILE!] → file_monitoring.log
sys_enter_unlink/unlinkat/rename/renameat/renameat2/truncate [FILE!] with flags=UNLINK, RENAME, or TRUNCATE → file_monitoring.log
sys_enter_connect [CONNECT] / [CONNECT!] → network_monitoring.log
sys_enter_bind [BIND] / [BIND!] → network_monitoring.log
sys_enter_accept4 + sys_exit_accept4 [ACCEPT] → network_monitoring.log
sys_enter_socket [SOCKET] → network_monitoring.log

A trailing ! on the tag — [EXEC!], [FILE!], [CONNECT!], [BIND!] — marks an event the sensor flagged as security-relevant: an execution from a staging directory, access to a credential or persistence path, a listener or connection on a known attacker port. Flagged events are prioritized downstream.

Note

The sensor filters events on the endpoint, in the kernel, before anything is written to disk. This keeps volume low on busy hosts: routine library loads, loopback traffic, and repeated identical operations never leave the kernel. Which events survive is governed by filter.json — contact Logster Support if your environment needs coverage beyond the shipped baseline.