Connect Windows Endpoints
Once the App Node and GPU Node are running, connect your Windows endpoints so they start reporting in.
To monitor a Windows machine, install two components on it:
- Sysmon — the collector. It records detailed process, network, and file activity to the Windows event log. Always required.
- A shipper — it reads Sysmon's events and forwards them to the appliance. This guide covers the Splunk Universal Forwarder, which ships raw XML to the appliance's Splunk receiver on TCP 9997. On a fleet that does not already run the UF, use Winlogbeat instead — it ships to the appliance's Kafka listener on TCP 29092.
%%{init: {"theme":"base","themeVariables":{"fontFamily":"Roboto, Helvetica, Arial, sans-serif","fontSize":"15px","lineColor":"#8B6DB8","clusterBkg":"#FAF7FE","clusterBorder":"#D9C9F0","edgeLabelBackground":"#FFFFFF"},"flowchart":{"curve":"basis","nodeSpacing":50,"rankSpacing":80,"padding":16}}}%%
flowchart LR
subgraph EP[" Windows endpoint "]
direction LR
SM["<b>Sysmon</b><br><small>collector</small>"]
EVT[("Windows event log<br><small>Sysmon/Operational</small>")]
UF["<b>Splunk UF</b><br><small>shipper (raw mode)</small>"]
SM -- "writes events" --> EVT
EVT -- "read by" --> UF
end
R[["<b>App Node</b><br><small>Splunk receiver</small>"]]
IDX[["<b>Your indexers</b><br><small>cooked S2S</small>"]]
UF -- "raw XML · TCP 9997" --> R
UF -- "everything else" --> IDX
classDef collector fill:#E8F1FC,stroke:#3B82C4,stroke-width:2px,color:#12324F
classDef store fill:#FFF6E5,stroke:#D9922B,stroke-width:2px,color:#4A3208
classDef shipper fill:#EFE6FB,stroke:#7A3FBF,stroke-width:2px,color:#2D0A6B
classDef sink fill:#E4DAF6,stroke:#5B1FA8,stroke-width:3px,color:#2D0A6B
class SM collector
class EVT store
class UF shipper
class R sink
class IDX sink
Install Sysmon first: the shipper reads the
Microsoft-Windows-Sysmon/Operational channel, which does not exist until
Sysmon is installed.
One shipper per host
Run either the Splunk UF or Winlogbeat on a given endpoint, never both — each ships the same Sysmon channel, and running both duplicates every event in the detection pipeline.
Run all of the commands below in an Administrator PowerShell session on the endpoint.
Step 1 — Install Sysmon (the collector)
Download Sysmon from Microsoft Sysinternals and extract it:
cd C:\
Invoke-WebRequest -Uri "https://download.sysinternals.com/files/Sysmon.zip" -OutFile Sysmon.zip
Expand-Archive -Path Sysmon.zip -DestinationPath "C:\Program Files\Sysmon"
Add the configuration. Sysmon does almost nothing useful without one — the configuration decides which events are recorded and which noise is filtered out. Logster ships a ready-to-use baseline: download sysmon-config.xml and save it on the endpoint as:
It is a high-signal baseline that records the process, network, and file
activity Logster analyzes while filtering out routine Windows noise. It is
based on the widely-used community
SwiftOnSecurity sysmon-config
(licensed CC BY 4.0; attribution is retained in the file header).
Note
The configuration requires Sysmon 13 or newer (the Sysmon.zip download
above is always the latest version, so this is satisfied automatically).
Install Sysmon with the configuration:
Verify the service is running:
It should report Running. Sysmon now starts automatically on every boot.
Tip
To apply an updated configuration later — without reinstalling — use:
Step 2 — Ship events with the Splunk Universal Forwarder
Sysmon remains the collector; the Splunk Universal Forwarder (UF) already running on your fleet forwards its events to the appliance's Splunk receiver on TCP port 9997.
This section assumes the UF is already deployed and forwarding to your own
indexers. Logster is added as an additional destination, so your indexers
keep receiving exactly what they receive today. Nothing in
etc\system\local\ is touched — the configuration ships as a Splunk app and
deploys through a Deployment Server like any other.
The UF normally speaks Splunk's proprietary S2S protocol, which Logster does
not implement. Instead, the Logster output group is configured with
sendCookedData = false, which makes the UF stream raw, newline-delimited
events over plain TCP — with Sysmon collected as rendered XML
(renderXml = 1), each event arrives as a single-line Windows Event XML
document that the receiver parses and feeds into the detection pipeline.
The mechanism is _TCP_ROUTING in inputs.conf, which overrides
defaultGroup for a single input. Two facts make this work:
sendCookedDatais per output group. Thelogstergroup can be raw while your existing group stays cooked S2S, on the same UF._TCP_ROUTINGis per input. Only the Sysmon channel is redirected; every other input keeps followingdefaultGroup.
Never point defaultGroup at Logster
Setting defaultGroup = logster under [tcpout], or disabling the UF's
internal log monitors with disabled = 1, re-points every input at
Logster and stops collecting splunkd.log, metrics.log and license
usage — a monitoring outage in your existing Splunk. Route per input, as
below.
The UF must run as Local System
Reading the Microsoft-Windows-Sysmon/Operational channel requires it.
A UF running under a restricted service account collects nothing from
that channel. This is the default for an MSI install
(SplunkForwarder service).
1. Find your existing output group
Read it from the effective config rather than guessing:
Look for defaultGroup under [tcpout]. The examples below call it
primary_indexers — substitute your own name everywhere it appears.
2. Install the logster_forwarder app
Logster ships as a Splunk app so nothing in etc\system\local\ is touched and
the whole thing deploys through a Deployment Server. A ready-made copy of the
files below lives alongside this guide in
logster_forwarder/ — substitute CHANGEME and
EXISTING_GROUP and drop it straight into etc\apps\. Create:
C:\Program Files\SplunkUniversalForwarder\etc\apps\logster_forwarder\
default\app.conf
local\inputs.conf
local\outputs.conf
default\app.conf:
[install]
state = enabled
is_configured = 0
[package]
id = logster_forwarder
check_for_updates = 0
[ui]
is_visible = 0
label = Logster Forwarder
local\outputs.conf — note what is absent: there is no [tcpout] stanza
and no defaultGroup. Adding either would re-point every input at Logster.
[tcpout:logster]
server = <appliance-host>:9997
sendCookedData = false
useACK = false
autoLBFrequency = 86400
maxQueueSize = 64MB
dropEventsOnQueueFull = 5
Set server to your appliance's address.
Note
sendCookedData = false is required — it switches this group from the
proprietary S2S protocol to raw TCP, which is what the Logster receiver
accepts. Both settings are per group, so your existing group stays cooked
S2S. useACK must stay false here: indexer acknowledgement is an S2S
feature and is not supported in raw mode.
Keep dropEventsOnQueueFull set
Without it the group's queue blocks indefinitely when the Logster receiver is unreachable, which can apply back-pressure to the shared input pipeline and stall delivery to your existing indexers. Five seconds bounds the blast radius of a Logster outage to Logster.
OpenShift deployments
On the OpenShift deployment the receiver always runs in-cluster but is not
exposed through a Route (Routes cannot carry raw TCP). Expose it to
out-of-cluster endpoints and get the external server address — a
LoadBalancer IP or NodePort — from
Install Logster on OpenShift → B7,
then use that in place of <appliance-host>:9997 above.
local\inputs.conf:
[WinEventLog://Microsoft-Windows-Sysmon/Operational]
disabled = 0
renderXml = 1
_TCP_ROUTING = logster
# PowerShell Script Block Logging (Event ID 4104) — optional but
# recommended. Requires the policy enabled on the endpoint (step 2b).
[WinEventLog://Microsoft-Windows-PowerShell/Operational]
disabled = 0
renderXml = 1
whitelist = 4104
_TCP_ROUTING = logster
# PowerShell 7 (pwsh) logs to its own channel. On an endpoint without
# PowerShell 7 the channel does not exist: the UF logs one error at
# startup and skips this stanza; every other input keeps working.
[WinEventLog://PowerShellCore/Operational]
disabled = 0
renderXml = 1
whitelist = 4104
_TCP_ROUTING = logster
# Keep the UF's internal logs off the Logster socket — see below.
[monitor://$SPLUNK_HOME\var\log\splunk\splunkd.log]
_TCP_ROUTING = primary_indexers
[monitor://$SPLUNK_HOME\var\log\splunk\metrics.log]
_TCP_ROUTING = primary_indexers
[monitor://$SPLUNK_HOME\etc\splunk.version]
_TCP_ROUTING = primary_indexers
The three internal monitors are not optional
Splunk ships those three stanzas with _TCP_ROUTING = * in
etc\system\default\inputs.conf. * means every tcpout group, so
they reach the logster group even though only the Sysmon input names
it. Measured on a live UF, omitting these overrides sends 35,327 noise
lines for every 85 real events — 99.8% waste.
Re-pointing them at your existing group suppresses them for Logster while
leaving collection fully intact. Do not use disabled = 1 here: that
stops collecting them for your own Splunk too.
To keep shipping Sysmon to your existing indexers as well, list both groups:
2b. (Recommended) Enable PowerShell Script Block Logging
The logster_forwarder app also forwards PowerShell Script Block Logging
(Event ID 4104) so the console can reason over what scripts actually did.
Turn the policy on once per endpoint (an elevated PowerShell):
$k = 'HKLM:\SOFTWARE\Policies\Microsoft\Windows\PowerShell\ScriptBlockLogging'
New-Item -Path $k -Force | Out-Null
Set-ItemProperty -Path $k -Name EnableScriptBlockLogging -Value 1 -Type DWord
If PowerShell 7 (pwsh) is installed, enable its policy too. PowerShell 7
does not read the Windows PowerShell key above. It logs to its own
PowerShellCore/Operational channel, which the app also forwards:
$k = 'HKLM:\SOFTWARE\Policies\Microsoft\PowerShellCore\ScriptBlockLogging'
New-Item -Path $k -Force | Out-Null
Set-ItemProperty -Path $k -Name EnableScriptBlockLogging -Value 1 -Type DWord
Leave EnableScriptBlockInvocationLogging unset — it multiplies event volume
by two orders of magnitude and adds nothing Logster uses.
Restart the forwarder after any inputs.conf change
The UF does not pick up a newly added event-log input until
SplunkForwarder restarts. An added stanza with no restart looks exactly
like a working one — the channel is simply never collected.
3. Verify the merged config, then restart
btool shows the effective config and which file each setting came from.
Check this before restarting:
cd "C:\Program Files\SplunkUniversalForwarder\bin"
.\splunk btool inputs list --debug | findstr /I "Sysmon PowerShell ROUTING renderXml"
.\splunk btool outputs list --debug | findstr /I "tcpout server sendCooked"
You should see both tcpout groups with independent sendCookedData, the
Sysmon stanza routed to logster, and the three internal monitors routed to
your existing group — no remaining _TCP_ROUTING = *.
Config precedence
etc\system\local\ beats any app. If the Sysmon channel or those
monitors are already defined in your system\local\inputs.conf, this
app's stanzas are ignored — add _TCP_ROUTING to your existing stanzas
instead. btool is what tells you which file won.
Generate some activity on the endpoint (start a program, open a website) and
confirm events for the endpoint's hostname appear on the Logster Console
(http://<app-node>:5001) within a few minutes.
To confirm the PowerShell channel specifically, run any short script on the
endpoint, e.g. powershell -NoProfile -Command "Get-Date; Get-ChildItem $env:TEMP | Measure-Object",
then check System Health → Ingest → Receiver events by channel on the
console: a Microsoft-Windows-PowerShell/Operational band should appear within
a minute. (Without the dashboard: curl -s <appliance>:8005/metrics | grep channel_events.)
For PowerShell 7, run the same script with pwsh instead of powershell; the
receiver metrics then show a channel="PowerShellCore/Operational" line.
Already collecting Sysmon into your own Splunk?
This is the one case that does not resolve cleanly. renderXml is a
per-stanza setting and there is one stanza per channel, so a single UF
cannot render the same channel as XML for Logster and non-XML for your
indexers. The Logster receiver requires XML — non-XML input is discarded as
non_xml_noise.
Three options:
- Switch the channel to
renderXml = 1and re-test your extractions. Your indexers keep receiving Sysmon (via_TCP_ROUTING = primary_indexers, logster), but_rawbecomes XML, which can affect existing props, transforms, CIM mappings and dashboards built on the Sysmon TA. - Run a second UF instance under a separate
SPLUNK_HOMEand management port, dedicated to Logster. Clean isolation at the cost of a second agent. - Leave your existing collection alone and feed Logster by a different mechanism — for example Winlogbeat on the same host. Note the one-shipper-per-channel caution above.
UF notes and limitations
- Hostname is taken from the
Computerelement inside each event's XML (raw mode carries no Splunk metadata such as host/source/sourcetype). - Tenant assignment is by hostname via the appliance's
splunk_receiver.tenant_mapconfiguration; unmatched hosts land indefault_tenant_id. The mapping is unauthenticated — appropriate for a trusted LAN, not for untrusted networks. - TLS between the UF and the receiver is off by default — see TLS to the receiver below.
- Per-event filtering is not possible on a UF. A UF has no index-time
props/transformspipeline, so you cannot drop individual Sysmon event codes on the way out. Filter at the collector instead, in the Sysmon config XML — the better place for it regardless. - Only the Sysmon channel above is supported. The receiver enforces a channel
allowlist (
splunk_receiver.allowed_channels, defaultMicrosoft-Windows-Sysmon/Operational): any other Windows event channel shipped through the same UF is dropped before Kafka and counted under thewrong_channeloutcome. Re-pointing the_internalmonitors at your existing group ininputs.confabove keeps the UF's own logs off the Logster socket in the first place.
TLS to the receiver
By default the UF ships cleartext to port 9997. Encrypt it whenever the endpoints reach the receiver over anything but a trusted LAN.
On the receiver side first: appliance — set splunk_receiver.tls in
service-config.yaml; OpenShift — use splunkReceiver.tls (see
Install Logster on OpenShift → B7, step 4).
The receiver does not ask for or validate client certificates.
Then, on each endpoint, add TLS settings to [tcpout:logster] in
outputs.conf. Pick one of the two configurations below and restart the UF.
Copy the CA that signed the receiver's certificate, plus a client
certificate for the UF, into %SPLUNK_HOME%\etc\auth\:
No certificate files needed on the endpoint — the transport is encrypted but the server's identity is not checked:
[tcpout:logster]
server = <receiver-host>:9997
sendCookedData = false
useACK = false
useSSL = true
sslVerifyServerCert = false
The UF logs ERROR SSLCommon - No valid cert found in sslContext once at
startup. That is expected in this mode — forwarding still works.
useSSL = true is what actually switches TLS on
sslRootCAPath / sslVerifyServerCert on their own do not enable TLS.
Without useSSL, the UF uses its legacy rule — SSL only if clientCert is
set — and otherwise keeps sending cleartext into a TLS listener. Nothing on
the tcpout path reports an error; the events are simply lost. Confirm the
mode in splunkd.log:
clientCert is required to verify the server
Counter-intuitively, the UF cannot build a verifying client SSL context
without a clientCert of its own — with sslVerifyServerCert = true and no
clientCert, it fails with Can't read ca certificate file /
Failed to create SSL context, then falls back to cleartext. Any
self-signed client certificate works; the receiver ignores it.
Path gotchas for clientCert / sslRootCAPath
Use $SPLUNK_HOME/... with forward slashes, as above. Absolute Windows
paths with backslashes (C:\Program Files\...) are not read reliably from
.conf files and produce the same Can't read ca certificate file error —
and the same silent cleartext fallback.
Also note the certificate must not have the same Common Name as its CA,
and sslCommonNameToCheck must match the receiver certificate's CN/SAN, or
the UF rejects the connection with
X509 certificate ... did not match any alternate names.
Verify. On the receiver, the connection appears as usual (UF connected)
and logster_splunk_receiver_events_total{outcome="produced"} keeps climbing.
Repeated TLS handshake failed ... WRONG_VERSION_NUMBER warnings mean an
endpoint is still sending cleartext.
Endpoint not appearing on the console?
- Confirm Sysmon is running on the endpoint and writing to its
Microsoft-Windows-Sysmon/Operationallog. - Confirm exactly one shipper is installed and running (
SplunkForwarderorwinlogbeat, not both). - Confirm the endpoint can reach
<appliance-host>:9997, and checkdocker compose logs -f splunk-receiveron the appliance for a connection and fornon_xml_noise/wrong_channeloutcomes. - Check firewalls between the endpoint and the appliance.
- PowerShell events missing but Sysmon fine? Confirm Script Block Logging is
enabled (registry key above) and that
SplunkForwarderwas restarted after the PowerShell stanza was added; the Ingest panel's per-channel chart shows whether the channel is arriving at all. For PowerShell 7 scripts, check the separatePowerShellCorepolicy key. The forwarder only looks for thePowerShellCore/Operationalchannel when it starts, so restartSplunkForwarderif PowerShell 7 was installed after it started.