Logs are a record of everything that is happening inside the IT environment of your organization. They're typically a series of timestamped messages that give you firsthand information about all the activities in your network.
Every device and application in the network generates log data, along with NetFlow data, which is used to monitor network traffic. Logs are the main source of input to security information and event management (SIEM) solutions. A SIEM solution, at its root, is a log management platform that also performs security analytics and alerting, insider risk mitigation, response automation, threat hunting, and compliance management.
Log management involves the collection, storage, normalization, and analysis of logs to generate reports and alerts. Log management ensures that the network activity data hidden in logs is converted to meaningful, actionable security information. Log management is a prerequisite for network and security administrators to monitor and secure the network. SIEM logging combines event logs with contextual information about users, assets, threats, and vulnerabilities and processes them using algorithms, rules, and statistics.
Log management is a challenging task. To collect and process log data in real time, regardless of the volume of log data and the number of devices in the network, organizations need a robust log management mechanism. All in all, log management needs to be flexible enough to accommodate all network devices and applications.
Log collection is the first step in log management. A SIEM solution collects logs and events from a diverse set of systems in the network and aggregates them in one place. Logs are typically collected from workstations, servers, domain controllers, network devices, IDSs, IPSs, endpoint security solutions, databases, web servers, public cloud infrastructure, and cloud platforms.
Every network has different systems and environments that generate various log formats, such as event logs, syslogs, and other application logs. Log collectors need to be flexible enough to accommodate all network devices and applications.
Logs can be collected via:
Agent-based log collection requires the deployment of an agent on the devices that generate logs. The agent not only collects and filters the logs, but it also parses and converts them into other formats before forwarding them to the log collection server.
Windows, Unix, and most other systems create logs in areas of the file system that require high-level privileges to view, rotate, or relocate. Agents were developed to collect security-related information from the local system and then convert it to a format suitable for transmission over the network to a central collector. The agents are designed to run in the background with sufficient privileges to monitor and manage the logging subsystem, utilizing only those system resources necessary to collect, process, filter, and send the logs to the SIEM host with minimal overhead.
Agent-based log collection comes in handy for collection of logs across WANs and through firewalls. It also helps in log collection from devices residing in the restricted zones of your network such as DMZs. Using an agent for log collection reduces the CPU utilization of the server and thereby provides more control over the events per second rate. Windows Server, NXLog, and OSSEC are some of the popular agents used for log collection.
The agent can be deployed on any server in the network or sub-net and on all types of operating systems. It is installed as a service in that server. The agent collects the logs remotely, pre-processes the logs, and transfers them to the server in real time and without interruption.
In SIEM solutions, agentless log collection is the predominant method used to collect logs. In dynamic cloud environments, agentless auditing is critical to reduce costs, unlock visibility, and to accelerate the speed of deployment.
There are embedded devices such as routers, printers, switches, and firewalls in which third-party software installation is not supported. In highly regulated systems, installation of additional software is not permitted. In these cases, an agentless log collection approach can be implemented instead, allowing devices to send logs to a remote data collector. One factor that forces the deployment of agents for log collection is the unavailability of an established network connection.
In agentless log collection, the log data generated by the devices is automatically sent to a SIEM server securely, eliminating the need for an additional agent to collect the logs, which reduces the load on the devices.
Between agent-based or agentless log collection, neither is better than the other. The choice should be made considering the needs of the organization. So it is best to have a SIEM solution that offers both agent based and agentless log collection methods.
Log360 is your one-stop solution for all log management and network security challenges. It is an integrated solution that combines EventLog Analyzer, ADAudit Plus, and Cloud Security Plus into a single console to help you manage your network security, Active Directory auditing, and public cloud management. EventLog Analyzer is designed to support both agent based and agentless log collection mechanisms to cater to all devices and applications in the network.
The following table lists some of the important log sources and what methods can be used to collect these logs in Log360.
|Log source||Agent-based log collection||Agentless log collection|
|Core Windows infrastructure|
|Endpoint security solutions|
|Firewalls, NGFWs, IDSs, and IPSs|
|Linux and Unix systems|
|Routers and switches|
You will receive regular updates on the latest news on cybersecurity.
2022 Zoho Corporation Pvt. Ltd. All rights reserved.
You'll be receiving the savings report in your inbox shortly.