Remote Log collection on Window and Linux

Every enterprise needs to collect and monitor log data from devices across their network to ensure security, troubleshoot operational issues, and conduct forensic analysis on security incidents. For this they might rely on either a log management tool or a SIEM solution. Irrespective of the tool used, collecting logs to a centralized location is more difficult than it appears. From configuring the devices to send log data to central server to ensure security of logs in transit, log collection is as important and tough as any other log management processes.

Predominantly, there are two methods to collect log data—agent-based and agentless. Agent-based log collection requires installation of an agent in every machine that collects and forwards the log data from the device to the central server. When collecting log data from a secured network, agent-based log collection is employed. In other circumstances, this method is not preferred as it is difficult to administer. So, enterprises prefer native log forwarding and at times remote log collection.

When it comes to network devices, Linux/Unix machines, the syslog data can be captured using the log forwarding feature available in the native platform. However, for remotely collecting Windows event logs, the procedure is slightly different.

This page explains the steps needed to remotely collect syslog data using a Syslog server.

How to collect logs remotely using Syslog server?

Collecting syslogs remotely is a fairly simple process involving two steps—configuring the remote server that will centrally collect all log data, and configuring the devices to send log data to the remote server.

Step 1: Configuring the remote server

To configure a syslog server to collect logs remotely,

• Append the following to the /etc/rsyslog.conf file, in the /var/log folder of the server.

  $ModLoad imtcp.so

  $InputTCPServerRun 514

  Here 514 refers to the TCP port number through which the syslog server receives the log data.

• Create a variable template to ensure that the logs collected from different hosts do not get mixed up. Add the following to /etc/rsyslog.conf file:

  $template

  DynamicFile,"/var/log/loghost/%HOSTNAME%/%syslogfacility-text%.log"

  *.* -?DynamicFile

• Add the below entry to /var/logrotate.d/ syslog configuration file to ensure that new log files are a part of the log rotation:

  /var/log/loghost/*/*.log

• Assign a static IP address to the remote server. This helps the devices to contact and constantly send the log data to the remote server.
• Ensure that your firewall allows access to TCP port 514 by adding the following rules :

  # systemctl restart rsyslog

  # firewall-cmd --add-port=514/udp --permanent

  # firewall-cmd --add-port=514/tcp --permanent

  # firewall-cmd --reload

Step 2: Configure the syslog devices

• Add the following to the rules section in/var/rsyslog.conffile

  *.* @@<IP address of the log server>:514

  Here, <IP address of the log server> refers to the static IP address of your Syslog server and 514 is the TCP port number through which the log data will be sent.

How to remotely collect Windows event logs?

There are multiple ways to remotely access and collect windows event logs.

• Using API calls that utilize EvtOpenSession to establish a remote connection and call event log functions.
• Establishing remote sessions through WMI and run WMI tasks for collecting event logs.
• Collecting and accessing event logs through Event Viewer UI on an Active Directory account with permissions to read event logs.

Pre-requisites to remotely collect Windows event log:

To access and collect event logs using Event Viewer UI you need an Active Directory service account with specific permissions to access Windows event logs. These permissions can be granted through Local security policy or Group policy object (GPO) in the domain.

Below are the pre-requisite steps that you need to follow to remotely access and collect Windows event logs.

Creating service accounts and providing the required permissions

• Create a service account and configure it on the remote collector. Another option is to have the account on the collector machine that has proper access so that you can use integrated AD authentication for log collection.
• Add the account to the following built-in domain groups:
  • Event log readers
  • Distributed COM users
• Give the service account 'Manage auditing and security logs' privilege. This can be done by creating a GPO or using the local security policy.
  • Providing privileges using 'local security policy'
    • Navigate through the following: Computer configuration >> Windows Settings >> Security settings >> Local policies >> User rights assignment
    • Under User rights assignment, navigate to Manage auditing and security logs and add the service account to the list.
• If you wish to remotely collect logs through WMI protocol, give this account WMI access by following the below steps:
  • Open 'wmimgmt' and right click -> Properties > Security -> Advanced.
  • Allow the service account to “Execute Methods”, “Provider Write”, “Enable Account”, “Remote Enable”.
• Give registry permissions for this account.
  • Open Regedit -> Local machine ->
    System\CurrentControlSet\ Services\eventlog\Security -> right click -> permissions and add the service account.
• Assign DCOM rights and grant permissions on c:\windows\system32\winevt for the service account.

The service account is now able to read all the logs from any part of the domain through Event Viewer UI. Just a few more steps now.

1. Enabling connectivity: Edit the Windows firewall rules on the machine on which the service account resides
   • Navigate to Inbound rules and enable Remote event log management (RPC)
   • Ensure that the protocol and profile are respectively specified as 'TCP' and 'Domain' respectively
2. Enabling windows collector service: You need to enable the collector service on the remote server for it to receive the log files. To do that, login to the remote server as a local or domain admin and execute the following command in cmd. exe.

wecutil qcin

3. Enabling domain computers to remote connection: Windows Remote Management (WRM) is a protocol that is used for exchanging information across systems in the domain. For remote log collection, you need to enable this protocol on each of the devices to facilitate log data exchange. To enable WRM protocol, login to the source computers as a local or domain administrator and execute the below command.

winrm quick config

4. Enabling subscription on Windows: Subscriptions define the relation between the source device and the collector, the remote server. A collector can receive log data from all devices in the network or from a specific set of devices. To enable subscription of the domain computers on the remote log collector machine, follow the below steps.
   • Navigate through Event Viewer >> Subscriptions >> Actions >> Create Subscription.
   • In the Subscription Properties dialog box,
     • Specify the name for the subscription
     • Provide the description
     • In 'Destination Logs' select 'Forwarded Events'
     • Choose Subscription Type as 'Collector initiated' if the logs are collected by the remote server from the respective sources. In this case, you need a service account with appropriate privileges to collect the logs. Refer to step 5 for details on creating the service account and assigning permissions. If you choose 'Source initiated' the source device will use the native log forwarding techniques to forward the logs to the collector.
     • Click 'Select Computers' and 'Add Domain Computers' in the subsequent dialog box.
     • Type the source computers' name, click on 'Check Names' and if found, click Ok.
     • Click OK to return to 'Subscription Properties'
   • Click Select Events to open query filter.
     • From the 'Logged' drop down, specify the time interval at which the logs have to be collected
     • Select the type of event logs—Critical, Warning, Verbose, Information, and Error— that you wish to collect
     • From the drop down select how you wish to collect logs from source —'By log' or 'By source' —depending on your requirement.
   • Click on 'Advanced Subscription Settings' button to fine tune your log collection. Here, you can specify the User Account that can be used to collect the log data remotely, Event Level Optimization criteria— to minimize bandwidth, latency, or choose normal log collection method, protocol and port used for log collection.