Skip to main content

Syslog

Accepts syslog messages sent directly over a TLS TCP connection to the Monad platform.

Requirements​

There are no prerequisites to start using the Syslog input. Simply create a Syslog input and attach it to a pipeline.

Details​

The Syslog input receives messages over a persistent TLS TCP connection. It is well-suited for any system or application that emits syslog output, including Linux hosts, network devices, firewalls, and custom applications. No authentication is required — the pipeline ID is embedded in the destination hostname.

Sending data​

Send syslog messages over TLS to:

<pipeline-id>.l4.monad.com:6514

Replace <pipeline-id> with the ID of your pipeline.

Message framing​

The input follows RFC 6587 TCP framing and accepts either of the two standard framing methods:

  • Octet-counted — each message is prefixed with its byte length followed by a space: <length> <message>
  • Non-transparent (newline-delimited) — each message is terminated with a newline (\n) character

Both formats can be mixed on the same connection; the input automatically detects which framing is in use for each message based on whether it starts with a digit.

Message parsing​

After framing is stripped, the raw message is parsed as one of two syslog formats:

RFC 5424​

RFC 5424 is the modern IETF syslog protocol. Messages follow this structure:

<PRI>VERSION TIMESTAMP HOSTNAME APP-NAME PROCID MSGID [STRUCTURED-DATA] MSG

Example:

<34>1 2024-01-15T10:30:00Z myhost sshd 12345 ID47 [exampleSDID@32473 iut="3" eventSource="Application"] An application event log entry

RFC 3164​

RFC 3164 is the older BSD syslog format. Messages follow this structure:

<PRI>Mmm dd hh:mm:ss HOSTNAME TAG[PID]: MSG

Example:

<34>Jan 15 10:30:00 myhost sshd[12345]: Accepted publickey for user from 192.0.2.1

Detection is based on message structure: RFC 5424 is identified by a version number following the PRI; RFC 3164 is identified by a PRI or a BSD-format timestamp starting with a month abbreviation.

Fallback for unrecognized messages​

If a message cannot be parsed as either RFC 5424 or RFC 3164, it is not dropped. Instead, the entire raw message is placed in the msg field and the event is forwarded as-is.

Output fields​

Each parsed message produces a JSON object with the following fields. Fields with no value (including RFC 5424 - nil values) are omitted.

FieldTypeSourceDescription
priintegerRFC 5424, RFC 3164Priority value encoding facility and severity. Facility = pri / 8, severity = pri % 8.
versionintegerRFC 5424 onlySyslog protocol version (always 1 for RFC 5424).
timestampstringRFC 5424, RFC 3164Timestamp of the event. RFC 5424 uses ISO 8601 format; RFC 3164 uses BSD format (Mmm dd hh:mm:ss).
hostnamestringRFC 5424, RFC 3164Hostname, IP address, or other identifier of the originating device.
app-namestringRFC 5424, RFC 3164Name of the application or process that generated the message. In RFC 3164, this is the TAG field (before any [PID]).
procidstringRFC 5424, RFC 3164Process ID of the sending application, if present.
msgidstringRFC 5424 onlyMessage type identifier used to distinguish event classes.
structured-dataobjectRFC 5424 onlyKey-value pairs from the structured data section, grouped by SD-ID. Each SD-ID is a key whose value is an object of parameter name/value pairs.
msgstringRFC 5424, RFC 3164, fallbackThe free-form message text. For unparseable messages, this contains the entire raw message.

Structured data example​

RFC 5424 structured data like:

[exampleSDID@32473 iut="3" eventSource="Application"][origin ip="192.0.2.1"]

Is represented as:

{
  "structured-data": {
    "exampleSDID@32473": {
      "iut": "3",
      "eventSource": "Application"
    },
    "origin": {
      "ip": "192.0.2.1"
    }
  }
}

Authentication​

The Syslog input does not currently require authentication. The pipeline is identified by the destination hostname (<pipeline-id>.l4.monad.com). Ensure that only trusted sources send to your pipeline endpoint.

Example​

rsyslog​

To forward messages from rsyslog over TLS to Monad, add a rule like the following to your rsyslog configuration:

# /etc/rsyslog.d/99-monad.conf
*.* action(type="omfwd"
           target="<pipeline-id>.l4.monad.com"
           port="6514"
           protocol="tcp"
           StreamDriver="gtls"
           StreamDriverMode="1"
           StreamDriverAuthMode="anon")

syslog-ng​

destination d_monad {
  syslog("<pipeline-id>.l4.monad.com"
    port(6514)
    transport("tls")
    tls(peer-verify(required-untrusted))
  );
};

log { source(s_local); destination(d_monad); };

netcat (testing)​

To send a quick test message using octet-counted framing:

MSG='<34>1 2024-01-15T10:30:00Z myhost testapp 1234 - - Hello from Monad'
echo -n "${#MSG} ${MSG}" | openssl s_client -connect <pipeline-id>.l4.monad.com:6514 -quiet