EventLog Analyzer - Performance Optimization Guide

The performance of EventLog Analyzer depends on the machine in which it is deployed. To maximize the performance on any machine, several configuration procedures can be followed as detailed in this guide.

• Calculate network traffic volume
• Java parameters configuration
• Database parameters configuration
• Disk space optimization

Calculate network traffic volume

It is important to understand the volume of traffic generated by your network, as you can configure various parameters on the EventLog Analyzer server to suit the amount of network data it handles. A single standalone server can handle up to 20000 EPS (events per second) for syslogs and 2000 EPS for event logs on a 64-bit machine. On a 32-bit machine, the values are 10000 EPS for syslogs and 1000 EPS for event logs. You can calculate the log flow rates for your network environment using the method described below. Below are the values for the average log size for various types of logs:

• Windows - 2000 bytes
• Linux/syslog - 100 bytes
• AS400 - 1000 bytes

For each log type, consider the average EPS from the devices generating that type of log. The amount of log data generated per second from these devices is thus given by:

By calculating the log flow rate for each log type and adding them, you can arrive at the network flow rate (in bytes/second), that is, the total amount of log data generated by your network per second.

Java parameters configuration

The memory required in Java, based on the network flow rate, is given below.

For 64 bit

Network flow rate (bytes/sec)	Memory required (MB)
	64-bit machine	32-bit machine
500,000	1024	1024
1,000,000	2048	1536
2,000,000	4096	-

To set the memory based on your network's requirement, follow the below steps, based on how EventLog Analyzer is installed on your system.

EventLog Analyzer installed as a service

1. Stop the EventLog Analyzer service.
2. Navigate to <EventLog Analyzer home>/server/conf.
3. Open the file wrapper.conf.
4. Search for wrapper.java.maxmemory.
5. The default value for this parameter is 1024, change it to match your network flow rate. Eg. If your network flow rate is 2,000,000 bytes/second, change the value to wrapper.java.maxmemory=4096.
6. Save the file and restart the EventLog Analyzer service.

EventLog Analyzer installed as an application

1. Shut down EventLog Analyzer.
2. Navigate to <EventLog Analyzer home>/bin.
3. Open the file setCommonEnv.bat (or setCommonEnv.sh in a Linux machine).
4. Search for -Xmx. It is in a line beginning with JAVA_OPTS.
5. The default value for this parameter is 1024, change it to match your network flow rate. Eg. If your network flow rate is 2,000,000 bytes/second, change the value to -Xmx4096m.
6. Save the file and restart EventLog Analyzer.

Database parameters configuration

PostgreSQL parameters configuration

To optimize performance of the PostgreSQL database:

1. Stop EventLog Analyzer.
2. Navigate to <EventLog Analyzer home>/pgsql/data/directory.
3. Open the file postgres_ext.txt.
4. Replace the existing values of the parameters, with the values mentioned below.
5. Save and restart EventLog Analyzer.

Parameter	Comment
shared_buffers=128 MB	Minimum requirement is 128 KB.
work_mem=12 MB	Minimum requirement is 64 KB.
maintenance_work_mem=100 MB	Minimum requirement is 1 MB.
checkpoint_segments=15	Logfile segments minimum 1 and 16 MB each.
checkpoint_timeout=11 minutes	Range: 30 seconds to 1 hour.
checkpoint_completion_target=0.9	checkpoint target duration is 0.0 - 1.0.
seq_page_cost=1.0	This parameter is measured in an arbitrary scale.
random_page_cost=2.0	This parameter is measured in same scale as above.
effective_cache_size=512MB
synchronous_commit=off

MySQL parameter configuration

To optimize performance of the MySQL database:

1. Stop EventLog Analyzer.
2. Navigate to <EventLog Analyzer home>/bin.
3. Open the file startDB.bat (startDB.sh in case of a Linux machine).
4. Replace the existing value of the parameter "--innodb_buffer_pool_size", with a value suited to the RAM size of the machine, as given in the table below. For example, if the RAM size is 8 GB, the parameter should be "--innodb_buffer_pool_size=3000M".
5. Save and restart EventLog Analyzer.

RAM Size	Value
1 GB	Default value (no need to replace)
2 GB	1200M
3 GB	1500M
4 GB	1500M
8 GB	3000M
16 GB	3000M

Disk space optimization

The hard disk space requirement depends on the log volumes generated in your environment. For a high network flow rate you need to have a greater disk space to store and process the logs.

Calculation of the required hard disk space for your environment

EventLog Analyzer automatically scales up to meet the growing number of log sources configured for log collection. The log archive and index folders are the main contributors to the growing size of stored logs. The total disk space required at any time to store the logs generated by your network is the combined size of the archive and index folders.

The archive and index sizes for a specific time period depend on the total volume of raw logs generated during that time period. 

	Default location	Default retention	Retention settings	Compression
Indexed data	<Installation Folder>/EventLog Analyzer/ES/	32 days	To update or change the retention period, navigate to Settings → Admin → Database Retention Settings.	Data which is older than 32 days will be automatically be compressed in the ratio of 1:10.
Archived data	<Installation Folder>/EventLog Analyzer/Archive/	Forever	To update or change the retention period navigate to Settings → Admin → Archive Settings.	Data which is older than 4 days will be automatically compressed in the ratio of 1:10.

Note:

The disk space required to store indexed data per day is almost equal to that of archived data. 

Formula:

Consider that:

V: Log volume per day in GB

A: Retention period of archived data in days

I: Retention period of indexed data in days

The log volume per day can be measured by checking the size of the <ELA Home>/ES/data folder after successfully running the application for one day with all the log sources configured.

The total log volume generated by your network over a certain time period can be given by:

Example:

Consider an environment with the following specifics.

• Log volume per day = 15 GB
• Archived data retention period = 180 days
• Indexed data retention period = 60 days

Result: Approximately 900 GB of disc space is needed to store the logs of this environment. 

The above calculation is approximate. The real values can vary as EPS increases, due to processing differences. The below table specifies the hard disk space required for one day for various log types and EPS levels:

Log Type	EPS	Hard disk space required (GB)
Windows event logs	1000	35
	2000	70
Syslogs	10,000	32
	20,000	64