How to configure syslog-ng

What is syslog-ng configuration file?

The syslog-ng configuration file is composed of distinct, declarative blocks that define the logging architecture of syslog-ng. It includes the following blocks:

Global options

This block defines general configuration parameters that affect the entire syslog-ng daemon, such as time zone settings, memory usage limits, and more.

For instance,

• recv-time-zone() sets the default time zone for messages that arrive at the source.
• log-msg-size() sets the maximum size of the log message that can be handled by syslog-ng.

Source block

The source block defines the input channels and sources from where syslog-ng collects the messages. It includes:

Each source block comprises of multiple source drivers that collect specific inputs.

The s_local block consists of the following drivers:

system(): This driver collects platform specific logs

internal(): This drivers collects logs from syslog-ng itself

The s_network block consists of the following drivers:

network(): This specifies that syslog-ng should open a network socket to accept incoming log data.

syslog(): This specifies that syslog-ng receives IETF-standard (RFC 5424) syslog messages.

The s_file block consists of the following drivers:

file(): It defines the specific file to be opened and read. It follows the syntax, file("/var/log/____").

Sample code:

source s_local { system(); internal(); file("/var/log/application.log" follow_freq(1)); };

The code above defines a source s_local that collects system messages, syslog-ng's internal messages, and follows a custom application log file.

Filter block

The filter block defines conditions based on which the messages are collected. It includes:

Sample code:

filter f_critical { level(crit..emerg) and facility(kern, authpriv); };

The filter block above defines a filter that selects messages with severity levels from critical to emergency, specifically from the kernel or authorization facilities.

Destination block

The destination block defines the output locations for the processed messages. It includes:

The drivers within the destination block are similar to the source block. But here they act as log receivers instead of acting as the sources. It includes the file(), network(), port(), ip(), and transport() drivers.

In cases where the logs are written on a database, the database drivers such as sql() with relevant parameters such as type() and host() that specify the type of the database and the host machine respectively are used.

Sample code:

destination d_remote_siem { tcp("10.0.0.1" port(6514) tls(peer-verify(required))); };

The code above defines the destination driver, d_remote_siem which is a remote SIEM server that will be receiving the logs via encrypted TCP (TLS) at a specific IP and port.

Rewrite block

A rewrite block is used to change specific fields permanently (macros) within a log message.

For instance the set() operation assigns a fixed or calculated value to a field and subst() operation replaces a matching pattern within a field using regular expressions.

Sample code:

rewrite r_host_update { set("CENTRAL_HOST", value("HOST")); };

The code above specifies the rewrite block, r_host_name where the set() operation is used to fix the hostname (HOST) to a specific name, "CENTRAL_HOST".

Template block

The primary role of the template block is to control the output appearance of the logs, without changing the internal log data itself. It is usually defined within the destination block to ensure the format of the log message that is delivered.

Sample code:

template t_filename { template("/var/log/remotes/${HOST}/$YEAR/$MONTH/$DAY/messages.log"); };

In the code above, the template block defines a dynamic filename that uses the sender's hostname and the date as the template for a file saved at the destination.

Log statements

The log statements block is the crucial component that connects all the elements to form a complete log path.

Sample code:

log { source(s_local); filter(f_critical); destination(d_remote_siem); };

This statement takes messages from s_local, applies the f_critical filter, and sends the resulting messages to d_remote_siem.

How to configure syslog-ng

In syslog-ng configuration, the specification of the version number using the @version directive is a mandatory step, and it must appear as the very first non-comment line in the primary configuration file.

After specification of the version number, the configuration is carried out as follows:

Define sources for log collection

For a typical syslog-ng daemon, it is necessary to define two sources, one for local messages and another for remote messages.

@version: <version_number> 
source s_local {
    system();    # Collects native system logs (/dev/log on Linux)
    internal();  # Collects messages generated by syslog-ng itself
};
source s_network {
    # UDP is unreliable but standard (port 514)
    syslog(ip(0.0.0.0) port(514) transport("udp"));
    # TCP is reliable and recommended for critical logs (port 601 or 514)
    syslog(ip(0.0.0.0) port(601) transport("tcp"));
    # TLS is reliable and encrypted port(6514)
syslog( ip(0.0.0.0) port(6514) transport("tls"));
};

Define destinations to store logs

Specify the destination drivers where syslog-ng sends the logs. This can be a local file, a database, or even another remote logging server.

destination d_file_per_host {
    file("/var/log/remote/$HOST/$YEAR-$MONTH-$DAY.log"
        create_dirs(yes)
        owner(root) group(root) perm(0644)
    );
};

Define parsers and filters

These are optional but essential for advanced log management as parsing transforms unstructured log text into structured key-value pairs and filtering helps narrow down the log volume.

# Example: A parser for Apache access logs
parser p_apache {
    apache-accesslog-parser();
};
# Filter for logs with a severity of 'error' or higher
filter f_error {
    level(err..emerg);
};

# Filter for only logs coming from a specific host
filter f_webserver {
    host("webserver01.example.com");
};

Define log paths

This is the step that helps define the entire logging pipeline from source to destination.

log {
    source(s_local);
    filter(f_webserver);
    parser(p_apache);        # Insert parser before destination
    destination(d_file_per_host);
};

Final steps

• Save the configuration file (/etc/syslog-ng/syslog-ng.conf).
• Test the file for syntax errors using the below command:
  Copy

  sudo syslog-ng -s -v

• Restart the service to apply the new configuration using the below command:
  Copy

  sudo systemctl restart syslog-ng

Syslog-ng configuration - Troubleshooting issues

Here are five common troubleshooting issues for syslog-ng configuration:

Firewall block

The firewall often blocks log ports (514/601/6514), preventing remote logs from reaching the server. Therefore, verify the ports are open and confirm the daemon is actively listening on the correct ports.

Syntax errors

The service fails to start or crashes due to syntax errors such as missing semicolons or brackets. Thus, run a configuration check using sudo syslog-ng -s -v before reloading the service to locate the exact error line.

TLS/Certificate failure

Secure TLS connections fail due to incorrect certificate paths in the tls() block or when the syslog-ng user lacks read permission for the certificate and key files. Therefore, verify all file paths and correct file permissions for before configuration.

Lack of write access

In most scenarios, logs are received but not saved in the file because the syslog-ng user lacks write permission to the destination file or the dynamic directory defined in a template. Therefore, ensure that the user has appropriate write permissions for the files and directories.

Lack of source definition

Remote logs are often dropped when the source driver is not listening on the expected port, or when the source is defined but not explicitly linked to a destination via a log statement. Hence it is critical to verify all source and log definitions to ensure a complete log path exists.

Syslog-ng and EventLog Analyzer

Modern log architecture leverages the unique strengths of both syslog-ng and analytics tools like ManageEngine's EventLog Analyzer.

Syslog-ng handles the reliable transport of logs, ensuring no data is lost in transit while also performing essential preprocessing (filtering and collection) at the source.

On the other hand, EventLog Analyzer functions as the central syslog server, indexing and analyzing the clean data it receives. It transforms raw logs into actionable security intelligence through its advanced capabilities. This synergy helps you achieve:

• Guaranteed log integrity: By utilizing syslog-ng's secure transport mechanisms, the logs that arrive at EventLog Analyzer are encrypted and untampered, ensuring the integrity of data for security analysis.
• Effective threat corroboration: EventLog Analyzer correlates individual security events across various devices obtained through syslog-ng for identifying complex, multi-stage attacks.
• Reduced noise: Syslog-ng pre-filters and normalizes logs, allowing EventLog Analyzer's powerful detection engine to focus exclusively on high-value events, leading to faster, more accurate threat detection and fewer false positives.
• Automated reporting: EventLog Analyzer automatically generates pre-built audit reports for regulatory standards such as PCI DSS, HIPAA, SOX, reducing the manual effort and time required to meet compliance obligations.

What's next?