High Availability (HA) Configuration - Enterprise Edition - MySQL (Central)

Before configuring the HA setup, please ensure that you have read this document completely.

This document contains the following sections;

  1. Prerequisites

  2. Steps to Configure High Availability

    1. Backup

    2. Restore

    3. Installing MySQL as Windows Service

    4. Steps to be Performed in Primary Central server

    5. Steps to be Performed in Standby Central server

    6. Configuring Notifications when Standby Server Takes Over the Role of Primary

  1. Additional Configurations

  2. FAQs

1. Prerequisites

  1. Network connectivity should be reliable between the Primary and Standby Central servers.
  1. The MySQL database will be running on the same server, where the IT360 is running.

  2. Both the Primary and the Standby Central servers should have the same IT360 Build version.

  3. The Primary and Standby Central servers should be in the same time-zone and time.

  4. If license is applied in the Primary Central Server, the same has to be applied in the Standby Central Server also. Click here for the steps to apply the license in both the servers.

  5. Before configuring the HA setup, take a backup of the IT360 database.

Top

2. Steps to Configure High Availability

In order to configure HA, first you need to backup the data from the Primary Central server and restore the same in the Standby Central server. This is to make sure that the data in both the servers are in sync. Next is to install MySQL as a Windows service. After this, its is required to activate MySQL data replication between the Primary and Standby databases. The replication process enables the data from one MySQL database server (the master) to be replicated to one or more MySQL database servers (the slaves). The steps required for data replication are listed below for the Primary and the Standby Central servers, separately.

2.1 Backup

  1. Stop the IT360 service in the Primary Central server, to take a backup.
  1. Ensure that the processes mysqld-nt.exe, java.exe and javaw.exe, related to the IT360's installation directory are not runnning in the Task Manager.

  2. Execute the FailoverBackup_Mysql.bat script, available under the <PrimaryCentralIT360_Home>/bin/ directory, to start the data backup.

  3. Once the backup is over, a directory backup is created under the <PrimaryCentralIT360_Home> directory, and the backup file IT360backup_DateTimeStamp.bak is placed in this directory.

  4. Copy the backup file IT360backup_DateTimeStamp.bak to the Standby Central server (in a directory, e.g. C:\IT360backup_DateTimeStamp.bak). This will be useful while executing the FailoverRestore_Mysql.bat script during restoration.

2.2 Restore

  1. You can restore the data over a Standby installation of IT360.

  2. For this, stop the IT360 service in the Standby Central server.

  3. Ensure that the processes mysqld-nt.exe, java.exe and javaw.exe, related to the IT360's installation directory are not running in the Task Manager.

  4. Execute the FailoverRestore_Mysql.bat script, available under <StandbyCentralIT360_Home>/bin/ directory, with the backup file (this is the file copied from the Primary to the Standby Central server - refer here) name as argument. (Provide the absolute path of the backup file.)

Example: C:\ManageEngine\IT360\bin>FailoverRestore_Mysql.bat , C:\IT360backup_DateTimeStamp.bak.

Top

<IT360_HOME> stands for the IT360 installation directory.  For e.g., if IT360 is installed in D:\ManageEngine, then <IT360_HOME> refers to D:\ManageEngine\IT360.

2.3 Installing MySQL as Windows Service

Note: This process of Installing MySQL as Windows Service has to be carried out in both the Primary and the Standby Central servers.
  1. Note down the mysql port number from <IT360_HOME>/applications/conf/AMServer.properties, against the property am.mysql.portUpdate that MySQL Port number in the <IT360_HOME>/mysql/my-defaults.ini file, as shown below;

    • Include port=<value from am.mysql.port> (e.g., port=32266) as a first entry under [mysqld] section in <IT360_HOME>/mysql/my-defaults.ini file.
  1. Go to <IT360_HOME>/mysql/bin directory.

  2. Execute the below mentioned command to install MySQL as a service. (Note: While installing the MySQL service, please use the absolute path of mysqld-nt.exe as mentioned below to install the MySQL service)

    • <IT360_HOME>/mysql/bin/mysqld-nt.exe --install MySQLIT360 --defaults-file=<IT360_HOME>/mysql/my-defaults.ini file.

Top

2.4 Steps to be Performed in Primary Central Server

Follow the below instructions;

  1. Ensure that the Primary IT360 services are stopped and the mysql process is not runnning, in the Task Manager.

  2. Change DB_HOME in ServiceDesk/startDB.bat file, by following the below steps;

  1. Execute the following batch files. Here, the scripts have to be executed with Administrator privilege. For e.g. on Windows 2008 Server, right click on the Command Prompt icon and Select the "Run as Administrator" option'. Now, follow the below steps;

    1. Execute the script FailoverSetup.bat, under <IT360_HOME>/bin.

    2. The script will prompt you to configure the database details.

    3. Refer the below screenshot for the complete steps:
The Secondary Server / Database mentioned in the below wizard is the Standby Server / Database.

Note: If the User Name and Port differ from the default ones, then configure their values, by referring the <IT360_HOME>/applications/working/conf/database_params.conf file, which is available in the Standby server.
  1. Click here to receive notifications, when the Standby Central server takes over as Primary.

Top

2.5 Steps to be Performed in Standby Central server

Follow the below instructions;

  1. Ensure that the IT360 services are stopped and the mysql process is not runnning, in the Task Manager.

  2. Change DB_HOME in ServiceDesk/startDB.bat file, by following the below steps;

  1. Execute the following batch files. Here, the scripts have to be executed with Administrator privilege. For e.g. on Windows 2008 Server, right click on the Command Prompt icon and Select the "Run as Administrator" option'. Now, follow the below steps;

    1. Execute the script FailoverSetup.bat under <IT360_HOME>/bin.

    2. The script will prompt you to configure the database details.

    3. Refer the below screenshot for the complete steps.
The Secondary Server / Database mentioned in the below wizard is the Standby Server / Database.

Note: If the User Name and Port differ from the default ones, then configure their values, by referring the <IT360_HOME>/applications/working/conf/database_params.conf file, which is available in the Standby server.
  1. Stop user-started MySQLs: Stop the MySQLs running on both the Primary and Standby servers, by following the below steps.
    1. Go to the <IT360_HOME>/bin directory.
    2. Stop DB.bat <PortNo> root, where <PortNo> is the port, in which the database is running.
  2. Start MySQL as Windows Service: Follow the below steps;

    1. Go to services.msc.
    2. Start the MySQLs (MySQLIT360), installed as Services.
  1. Start the Primary server. Make sure that the Primary server startup process is complete and then start the Standby Server.

  2. Click here to receive notifications, when the Standby Central server takes over as Primary.

Top

2.6 Configuring Notifications when Standby Central Server Takes Over the Role of Primary

You will receive notifications, in the form of emails, when the Standby Central server takes over the functions that were being performed by the Primary. For this, you need to manually add the email id (s) in the file: IT360_HOME>\applications\working\conf\FailOver.xml.

Sample entries are as follows;

<FAILOVER>

<Primary HEART_BEAT_INTERVAL="60" />

        <Standby FAIL_OVER_INTERVAL="60" RETRY_COUNT="1">

   <BACKUP ENABLED="TRUE" BACKUP_INTERVAL="600"/>

            <SEND_EMAIL

                SMTP_SERVER="smtp"

                TO_ADDRESS="xyz@zohocorp.com"

                FROM_ADDRESS="admin@zohocorp.com"

                SUBJECT="Primary Server Failed"

                BODY="Primary Server is failed and taken over by the Standby Server"/>

            </Standby>

   </FAILOVER>

Note: Stop and start the ManageEngine IT360 service, for the above changes to take effect.

The optimal value for the RETRY_COUNT is 10.

Top

3. Additional Configurations

Copying SDP attachments files to the new Primary Central Server

  1. Go to the Primary Central server`s installation directory.

  2. Execute backUpData.bat script, available under <PrimaryCentralIT360_Home>/Servicedesk/bin directory, with the argument: backUpData.bat --confiles.

  3. Once the backup is over, a directory confbackup is created (if it is not already available), under <PrimaryCentralIT360_Home>/servicedesk directory and the backup file backup_7513_conbackup_DateTimeStamp.data is placed in this directory.  

  4. Copy that backup_7513_conbackup_DateTimeStamp.data file into the <StandbyCentralIT360HOME>/servicedesk directory.

  5. Rename that .data extension to .zip (e.g., if the file name is backup_7513_conbackup_DateTimeStamp.data, then rename it to backup_7513_conbackup_DateTimeStamp.zip)

  6. Extract the backup_7513_conbackup_DateTimeStamp.zip file, in the <StandbyCentralIT360HOME>/servicedesk directory.
Note: As of now, this is a limitation in our implementation and will be addressed in one of our upcoming hot fix releases.

Top

4. FAQs

  1. When does the MySQL data replication failure occur? What is the recommended solution for the same?

When the network connectivity between the Primary and Standby Central servers breaks, each server will start running as a Standalone Primary server. In this scenario, the Administrator will receive an email notification, indicating the failure in replication. Upon receiving this email, he has to restore the HA mode, by performing the HA configuration, once again.

  1. How to verify that the Mysql replication has started?

    To check whether the DB replication has started for both the Primary and Standby server DBs, do the following;

    1. Open the file <machinename>.err [IT360_HOME/mysql/data], in both the Primary and Standby Central servers.

    2. Check for the entry replication started. If the entry is available, then replication is successfully done.

  1. How to give multiple email ids, while configuring the notifications?

Type in the email ids, separated by commas.

  1. What is the recommended value for the HEART_BEAT_INTERVAL and the FAILOVER_INTERVAL, while configuring the notifications?

It is recommended to have the HEART_BEAT_INTERVAL and the FAILOVER_INTERVAL, above 20 seconds. Having lesser values may lead to unexpected behaviour.

  1. What is the purpose of setting the RETRY_COUNT, while configuring the notifications? Is the value set for it in the above sample entries is fixed or changeable?

In general, the Standby will go through the parameters, specified in the 'FailOver.xml' file, available under the '<IT360_HOME>/applications/working/conf' directory.  By specifying the RETRY_COUNT value in this configuration file, before starting the service, you can instruct the Standby server to try as many times as the count says, before taking over the role of the Primary. 

By default, this attribute is configured as 1, which means that it will try only once, before taking the role as Primary. Subsequent retry attempts will happen with an interval of 60 secs. For instance, if you need the Standby server to wait for 5 mins before taking over the role as Primary, you can set the RETRY_COUNT value as 5. Please note that the optimal value for the RETRY_COUNT is 10.

Top



Copyright © 2013, ZOHO Corp. All Rights Reserved.