Support Get Quote
Log Management

What is log collection?

Aug 08, 2022 3 min read

What is log collection?

Log collection is the process of collecting log data from various log sources within an organization's network and bringing them together in a central location for better analysis.

What are the different types of log collection?

The two most common mechanisms for collecting log data are agentless and agent-based log collection. Some of the other common log collection techniques include API-based log collection, WMI-based log collection, and SNMP traps.

1.Agentless log collection:

In agentless log collection, the logs generated on every device are collected without an agent. The device or application where the log is generated will directly send the log data to a central server. The transmission will be secured using protocols such as TCP and HTTPS.

2.Agent-based log collection:

This log collection mechanism uses an agent that resides within the device. The agent collects and securely sends the log data to a central server. The advantages of using agent-based log collection is that log collection filters can be applied using the agents to limit how much bandwidth the process consumes. Agent-based log collection is usually used in networks within a secure zone and where communication is restricted.

3. API-based log collection:

In this method, an API is used for querying and transferring log data to a secure server. You can also use APIs to collect logs and send them to a third-party log analytics tool to analyze the log data.

4. WMI event logging:

Windows Management Instrument (WMI) event logging is a method used to collect logs from the Windows environment. The Event Tracing for Windows (ETW) is done by WMI event logging. They collect details about events, diagnostic data, errors, and various other activities in your network.

5. SNMP traps:

An SNMP trap is made by an SNMP-enabled device, which is the agent, and sent to a collector. The collector is informed in real time by the SNMP trap whenever an important event happens, primarily collecting events for management and monitoring.

Where are logs collected from?

Typically, there are multiple log sources in a network, making it vital to choose the right sources to monitor. Here are some of the critical resources from which you need to collect and analyze log data:

1. Firewalls

Firewall logs are one of the most common logs to be collected due to the important role firewalls play in securing an organization. The firewall logs give insights into network traffic such as denied connections, allowed connections, configuration changes, and configuration errors, as well as details about the addition and deletion of users and their privilege level changes.

By analyzing firewall logs, admins can discover malicious activity in the network, optimize firewall rules, and strengthen their network's security boundaries.

2. Proxies and other web filters

Logs from proxies and other web filters are made up of the log data from users and applications that use your network. Apart from website requests from users, these logs also capture application and service requests. Proxy logs can extract information like the destination IP, destination port, user agent, device action, and a lot more. Capturing this information provides insights into what's happening in the network.

Organizations can find major issues in their network by monitoring the various user-agent strings and scrutinizing any abnormalities.

3. Windows events

The Windows event log is a complete record of everything that happens in a Windows operating system. Some of the log info collected includes Windows application logs, security and system logs, DNS server logs, Directory Server logs, and File Replication Service logs.

Collecting the Windows event logs ensures that any anomalies or strange behavior is immediately flagged and brought to notice. It ensures better server security, workstation security, and diagnostics for problems with malfunctioning hardware components.

For example, Pass-The-Hash is a popular attack among hackers to gain account access without a password. You will need to look for NTLM Logon type 3 event IDs—i.e. 4624 (success) & 4625 (failure)—for this attack.

Another common trait among hackers is that they try to hide their presence. Looking out for event IDs 104 (event log cleared) and 1102 (audit log cleared) can help you find their presence in your network.

4. Applications

An application log is a file with the information of all the events that occurred within an application. Some of the common components in application logs include context information, timestamps, and log levels.

You can collect logs from web server applications like IIS and Apache, databases including MS SQL and Oracle, DHCP-based applications, and others.

Application logs help you to identify and correct issues related to the performance and security of the applications. It also helps you detect unauthorized file access and data manipulation attempts by users.

A log collection tool can help you collect different types of logs from multiple sources and unify them with ease. Using a comprehensive log collection tool like ManageEngine's EventLog Analyzer can also help you organize and sort through your logs to gain valuable insights about your organization's security posture. Check out the solution's log collection capabilities here.

You may also like


Interested in a
log management

Try EventLog Analyzer
Database platforms

Understanding SQL Server Audit better

Read more
Previous articles
Next articles
Network devices

Critical Windows events: Event ID 6008 - Unexpected system shutdown

Read more

Manage logs, comply with IT regulations, and mitigate security threats.

Seamlessly collect, monitor, and analyze
logs with EventLog Analyzer

Your request for a demo has been submitted successfully

Our support technicians will get back to you at the earliest.

By clicking 'Submit', you agree to processing of personal data according to the Privacy Policy.

  Zoho Corporation Pvt. Ltd. All rights reserved.

Link copied, now you can start sharing