Skip to content

Connect Windows Endpoints

Once the App Node and GPU Node are running, connect your Windows endpoints so they start reporting in.

To monitor a Windows machine, install two components on it:

  1. Sysmon — the collector. It records detailed process, network, and file activity to the Windows event log. Always required.
  2. A shipper — it reads Sysmon's events and forwards them to the appliance. This guide covers the Splunk Universal Forwarder, which ships raw XML to the appliance's Splunk receiver on TCP 9997. On a fleet that does not already run the UF, use Winlogbeat instead — it ships to the appliance's Kafka listener on TCP 29092.
%%{init: {"theme":"base","themeVariables":{"fontFamily":"Roboto, Helvetica, Arial, sans-serif","fontSize":"15px","lineColor":"#8B6DB8","clusterBkg":"#FAF7FE","clusterBorder":"#D9C9F0","edgeLabelBackground":"#FFFFFF"},"flowchart":{"curve":"basis","nodeSpacing":50,"rankSpacing":80,"padding":16}}}%%
flowchart LR
    subgraph EP[" Windows endpoint "]
        direction LR
        SM["<b>Sysmon</b><br><small>collector</small>"]
        EVT[("Windows event log<br><small>Sysmon/Operational</small>")]
        UF["<b>Splunk UF</b><br><small>shipper (raw mode)</small>"]
        SM -- "writes events" --> EVT
        EVT -- "read by" --> UF
    end
    R[["<b>App Node</b><br><small>Splunk receiver</small>"]]
    IDX[["<b>Your indexers</b><br><small>cooked S2S</small>"]]
    UF -- "raw XML · TCP 9997" --> R
    UF -- "everything else" --> IDX

    classDef collector fill:#E8F1FC,stroke:#3B82C4,stroke-width:2px,color:#12324F
    classDef store fill:#FFF6E5,stroke:#D9922B,stroke-width:2px,color:#4A3208
    classDef shipper fill:#EFE6FB,stroke:#7A3FBF,stroke-width:2px,color:#2D0A6B
    classDef sink fill:#E4DAF6,stroke:#5B1FA8,stroke-width:3px,color:#2D0A6B
    class SM collector
    class EVT store
    class UF shipper
    class R sink
    class IDX sink

Install Sysmon first: the shipper reads the Microsoft-Windows-Sysmon/Operational channel, which does not exist until Sysmon is installed.

One shipper per host

Run either the Splunk UF or Winlogbeat on a given endpoint, never both — each ships the same Sysmon channel, and running both duplicates every event in the detection pipeline.

Run all of the commands below in an Administrator PowerShell session on the endpoint.


Step 1 — Install Sysmon (the collector)

Download Sysmon from Microsoft Sysinternals and extract it:

cd C:\
Invoke-WebRequest -Uri "https://download.sysinternals.com/files/Sysmon.zip" -OutFile Sysmon.zip
Expand-Archive -Path Sysmon.zip -DestinationPath "C:\Program Files\Sysmon"

Add the configuration. Sysmon does almost nothing useful without one — the configuration decides which events are recorded and which noise is filtered out. Logster ships a ready-to-use baseline: download sysmon-config.xml and save it on the endpoint as:

C:\Program Files\Sysmon\sysmon-config.xml

It is a high-signal baseline that records the process, network, and file activity Logster analyzes while filtering out routine Windows noise. It is based on the widely-used community SwiftOnSecurity sysmon-config (licensed CC BY 4.0; attribution is retained in the file header).

Note

The configuration requires Sysmon 13 or newer (the Sysmon.zip download above is always the latest version, so this is satisfied automatically).

Install Sysmon with the configuration:

cd "C:\Program Files\Sysmon"
.\Sysmon64.exe -accepteula -i sysmon-config.xml

Verify the service is running:

Get-Service Sysmon64

It should report Running. Sysmon now starts automatically on every boot.

Tip

To apply an updated configuration later — without reinstalling — use:

cd "C:\Program Files\Sysmon"
.\Sysmon64.exe -c sysmon-config.xml

Step 2 — Ship events with the Splunk Universal Forwarder

Sysmon remains the collector; the Splunk Universal Forwarder (UF) already running on your fleet forwards its events to the appliance's Splunk receiver on TCP port 9997.

This section assumes the UF is already deployed and forwarding to your own indexers. Logster is added as an additional destination, so your indexers keep receiving exactly what they receive today. Nothing in etc\system\local\ is touched — the configuration ships as a Splunk app and deploys through a Deployment Server like any other.

The UF normally speaks Splunk's proprietary S2S protocol, which Logster does not implement. Instead, the Logster output group is configured with sendCookedData = false, which makes the UF stream raw, newline-delimited events over plain TCP — with Sysmon collected as rendered XML (renderXml = 1), each event arrives as a single-line Windows Event XML document that the receiver parses and feeds into the detection pipeline.

The mechanism is _TCP_ROUTING in inputs.conf, which overrides defaultGroup for a single input. Two facts make this work:

  • sendCookedData is per output group. The logster group can be raw while your existing group stays cooked S2S, on the same UF.
  • _TCP_ROUTING is per input. Only the Sysmon channel is redirected; every other input keeps following defaultGroup.

Never point defaultGroup at Logster

Setting defaultGroup = logster under [tcpout], or disabling the UF's internal log monitors with disabled = 1, re-points every input at Logster and stops collecting splunkd.log, metrics.log and license usage — a monitoring outage in your existing Splunk. Route per input, as below.

The UF must run as Local System

Reading the Microsoft-Windows-Sysmon/Operational channel requires it. A UF running under a restricted service account collects nothing from that channel. This is the default for an MSI install (SplunkForwarder service).

1. Find your existing output group

Read it from the effective config rather than guessing:

cd "C:\Program Files\SplunkUniversalForwarder\bin"
.\splunk btool outputs list --debug

Look for defaultGroup under [tcpout]. The examples below call it primary_indexers — substitute your own name everywhere it appears.

2. Install the logster_forwarder app

Logster ships as a Splunk app so nothing in etc\system\local\ is touched and the whole thing deploys through a Deployment Server. A ready-made copy of the files below lives alongside this guide in logster_forwarder/ — substitute CHANGEME and EXISTING_GROUP and drop it straight into etc\apps\. Create:

C:\Program Files\SplunkUniversalForwarder\etc\apps\logster_forwarder\
    default\app.conf
    local\inputs.conf
    local\outputs.conf

default\app.conf:

[install]
state = enabled
is_configured = 0

[package]
id = logster_forwarder
check_for_updates = 0

[ui]
is_visible = 0
label = Logster Forwarder

local\outputs.conf — note what is absent: there is no [tcpout] stanza and no defaultGroup. Adding either would re-point every input at Logster.

[tcpout:logster]
server = <appliance-host>:9997
sendCookedData = false
useACK = false
autoLBFrequency = 86400
maxQueueSize = 64MB
dropEventsOnQueueFull = 5

Set server to your appliance's address.

Note

sendCookedData = false is required — it switches this group from the proprietary S2S protocol to raw TCP, which is what the Logster receiver accepts. Both settings are per group, so your existing group stays cooked S2S. useACK must stay false here: indexer acknowledgement is an S2S feature and is not supported in raw mode.

Keep dropEventsOnQueueFull set

Without it the group's queue blocks indefinitely when the Logster receiver is unreachable, which can apply back-pressure to the shared input pipeline and stall delivery to your existing indexers. Five seconds bounds the blast radius of a Logster outage to Logster.

OpenShift deployments

On the OpenShift deployment the receiver always runs in-cluster but is not exposed through a Route (Routes cannot carry raw TCP). Expose it to out-of-cluster endpoints and get the external server address — a LoadBalancer IP or NodePort — from Install Logster on OpenShift → B7, then use that in place of <appliance-host>:9997 above.

local\inputs.conf:

[WinEventLog://Microsoft-Windows-Sysmon/Operational]
disabled = 0
renderXml = 1
_TCP_ROUTING = logster

# PowerShell Script Block Logging (Event ID 4104) — optional but
# recommended. Requires the policy enabled on the endpoint (step 2b).
[WinEventLog://Microsoft-Windows-PowerShell/Operational]
disabled = 0
renderXml = 1
whitelist = 4104
_TCP_ROUTING = logster

# PowerShell 7 (pwsh) logs to its own channel. On an endpoint without
# PowerShell 7 the channel does not exist: the UF logs one error at
# startup and skips this stanza; every other input keeps working.
[WinEventLog://PowerShellCore/Operational]
disabled = 0
renderXml = 1
whitelist = 4104
_TCP_ROUTING = logster

# Keep the UF's internal logs off the Logster socket — see below.
[monitor://$SPLUNK_HOME\var\log\splunk\splunkd.log]
_TCP_ROUTING = primary_indexers

[monitor://$SPLUNK_HOME\var\log\splunk\metrics.log]
_TCP_ROUTING = primary_indexers

[monitor://$SPLUNK_HOME\etc\splunk.version]
_TCP_ROUTING = primary_indexers

The three internal monitors are not optional

Splunk ships those three stanzas with _TCP_ROUTING = * in etc\system\default\inputs.conf. * means every tcpout group, so they reach the logster group even though only the Sysmon input names it. Measured on a live UF, omitting these overrides sends 35,327 noise lines for every 85 real events — 99.8% waste.

Re-pointing them at your existing group suppresses them for Logster while leaving collection fully intact. Do not use disabled = 1 here: that stops collecting them for your own Splunk too.

To keep shipping Sysmon to your existing indexers as well, list both groups:

_TCP_ROUTING = primary_indexers, logster

The logster_forwarder app also forwards PowerShell Script Block Logging (Event ID 4104) so the console can reason over what scripts actually did. Turn the policy on once per endpoint (an elevated PowerShell):

$k = 'HKLM:\SOFTWARE\Policies\Microsoft\Windows\PowerShell\ScriptBlockLogging'
New-Item -Path $k -Force | Out-Null
Set-ItemProperty -Path $k -Name EnableScriptBlockLogging -Value 1 -Type DWord

If PowerShell 7 (pwsh) is installed, enable its policy too. PowerShell 7 does not read the Windows PowerShell key above. It logs to its own PowerShellCore/Operational channel, which the app also forwards:

$k = 'HKLM:\SOFTWARE\Policies\Microsoft\PowerShellCore\ScriptBlockLogging'
New-Item -Path $k -Force | Out-Null
Set-ItemProperty -Path $k -Name EnableScriptBlockLogging -Value 1 -Type DWord

Leave EnableScriptBlockInvocationLogging unset — it multiplies event volume by two orders of magnitude and adds nothing Logster uses.

Restart the forwarder after any inputs.conf change

The UF does not pick up a newly added event-log input until SplunkForwarder restarts. An added stanza with no restart looks exactly like a working one — the channel is simply never collected.

3. Verify the merged config, then restart

btool shows the effective config and which file each setting came from. Check this before restarting:

cd "C:\Program Files\SplunkUniversalForwarder\bin"
.\splunk btool inputs list --debug  | findstr /I "Sysmon PowerShell ROUTING renderXml"
.\splunk btool outputs list --debug | findstr /I "tcpout server sendCooked"

You should see both tcpout groups with independent sendCookedData, the Sysmon stanza routed to logster, and the three internal monitors routed to your existing group — no remaining _TCP_ROUTING = *.

Restart-Service SplunkForwarder

Config precedence

etc\system\local\ beats any app. If the Sysmon channel or those monitors are already defined in your system\local\inputs.conf, this app's stanzas are ignored — add _TCP_ROUTING to your existing stanzas instead. btool is what tells you which file won.

Generate some activity on the endpoint (start a program, open a website) and confirm events for the endpoint's hostname appear on the Logster Console (http://<app-node>:5001) within a few minutes.

To confirm the PowerShell channel specifically, run any short script on the endpoint, e.g. powershell -NoProfile -Command "Get-Date; Get-ChildItem $env:TEMP | Measure-Object", then check System Health → Ingest → Receiver events by channel on the console: a Microsoft-Windows-PowerShell/Operational band should appear within a minute. (Without the dashboard: curl -s <appliance>:8005/metrics | grep channel_events.) For PowerShell 7, run the same script with pwsh instead of powershell; the receiver metrics then show a channel="PowerShellCore/Operational" line.

Already collecting Sysmon into your own Splunk?

This is the one case that does not resolve cleanly. renderXml is a per-stanza setting and there is one stanza per channel, so a single UF cannot render the same channel as XML for Logster and non-XML for your indexers. The Logster receiver requires XML — non-XML input is discarded as non_xml_noise.

Three options:

  • Switch the channel to renderXml = 1 and re-test your extractions. Your indexers keep receiving Sysmon (via _TCP_ROUTING = primary_indexers, logster), but _raw becomes XML, which can affect existing props, transforms, CIM mappings and dashboards built on the Sysmon TA.
  • Run a second UF instance under a separate SPLUNK_HOME and management port, dedicated to Logster. Clean isolation at the cost of a second agent.
  • Leave your existing collection alone and feed Logster by a different mechanism — for example Winlogbeat on the same host. Note the one-shipper-per-channel caution above.

UF notes and limitations

  • Hostname is taken from the Computer element inside each event's XML (raw mode carries no Splunk metadata such as host/source/sourcetype).
  • Tenant assignment is by hostname via the appliance's splunk_receiver.tenant_map configuration; unmatched hosts land in default_tenant_id. The mapping is unauthenticated — appropriate for a trusted LAN, not for untrusted networks.
  • TLS between the UF and the receiver is off by default — see TLS to the receiver below.
  • Per-event filtering is not possible on a UF. A UF has no index-time props/transforms pipeline, so you cannot drop individual Sysmon event codes on the way out. Filter at the collector instead, in the Sysmon config XML — the better place for it regardless.
  • Only the Sysmon channel above is supported. The receiver enforces a channel allowlist (splunk_receiver.allowed_channels, default Microsoft-Windows-Sysmon/Operational): any other Windows event channel shipped through the same UF is dropped before Kafka and counted under the wrong_channel outcome. Re-pointing the _internal monitors at your existing group in inputs.conf above keeps the UF's own logs off the Logster socket in the first place.

TLS to the receiver

By default the UF ships cleartext to port 9997. Encrypt it whenever the endpoints reach the receiver over anything but a trusted LAN.

On the receiver side first: appliance — set splunk_receiver.tls in service-config.yaml; OpenShift — use splunkReceiver.tls (see Install Logster on OpenShift → B7, step 4). The receiver does not ask for or validate client certificates.

Then, on each endpoint, add TLS settings to [tcpout:logster] in outputs.conf. Pick one of the two configurations below and restart the UF.

Copy the CA that signed the receiver's certificate, plus a client certificate for the UF, into %SPLUNK_HOME%\etc\auth\:

[tcpout:logster]
server = <receiver-host>:9997
sendCookedData = false
useACK = false
useSSL = true
clientCert = $SPLUNK_HOME/etc/auth/logster-client.pem
sslRootCAPath = $SPLUNK_HOME/etc/auth/logster-ca.pem
sslVerifyServerCert = true
sslCommonNameToCheck = <CN of the receiver certificate>

No certificate files needed on the endpoint — the transport is encrypted but the server's identity is not checked:

[tcpout:logster]
server = <receiver-host>:9997
sendCookedData = false
useACK = false
useSSL = true
sslVerifyServerCert = false

The UF logs ERROR SSLCommon - No valid cert found in sslContext once at startup. That is expected in this mode — forwarding still works.

useSSL = true is what actually switches TLS on

sslRootCAPath / sslVerifyServerCert on their own do not enable TLS. Without useSSL, the UF uses its legacy rule — SSL only if clientCert is set — and otherwise keeps sending cleartext into a TLS listener. Nothing on the tcpout path reports an error; the events are simply lost. Confirm the mode in splunkd.log:

Select-String -Path "$env:SPLUNK_HOME\var\log\splunk\splunkd.log" `
  -Pattern "Initializing connection for"
# want: "... for SSL forwarding to <receiver-host>:9997"
# bad:  "... for non-ssl forwarding to <receiver-host>:9997"

clientCert is required to verify the server

Counter-intuitively, the UF cannot build a verifying client SSL context without a clientCert of its own — with sslVerifyServerCert = true and no clientCert, it fails with Can't read ca certificate file / Failed to create SSL context, then falls back to cleartext. Any self-signed client certificate works; the receiver ignores it.

Path gotchas for clientCert / sslRootCAPath

Use $SPLUNK_HOME/... with forward slashes, as above. Absolute Windows paths with backslashes (C:\Program Files\...) are not read reliably from .conf files and produce the same Can't read ca certificate file error — and the same silent cleartext fallback.

Also note the certificate must not have the same Common Name as its CA, and sslCommonNameToCheck must match the receiver certificate's CN/SAN, or the UF rejects the connection with X509 certificate ... did not match any alternate names.

Verify. On the receiver, the connection appears as usual (UF connected) and logster_splunk_receiver_events_total{outcome="produced"} keeps climbing. Repeated TLS handshake failed ... WRONG_VERSION_NUMBER warnings mean an endpoint is still sending cleartext.


Endpoint not appearing on the console?

  • Confirm Sysmon is running on the endpoint and writing to its Microsoft-Windows-Sysmon/Operational log.
  • Confirm exactly one shipper is installed and running (SplunkForwarder or winlogbeat, not both).
  • Confirm the endpoint can reach <appliance-host>:9997, and check docker compose logs -f splunk-receiver on the appliance for a connection and for non_xml_noise / wrong_channel outcomes.
  • Check firewalls between the endpoint and the appliance.
  • PowerShell events missing but Sysmon fine? Confirm Script Block Logging is enabled (registry key above) and that SplunkForwarder was restarted after the PowerShell stanza was added; the Ingest panel's per-channel chart shows whether the channel is arriving at all. For PowerShell 7 scripts, check the separate PowerShellCore policy key. The forwarder only looks for the PowerShellCore/Operational channel when it starts, so restart SplunkForwarder if PowerShell 7 was installed after it started.