AssetExplorer

    Disaster Recovery

    Disaster recovery (DR) ensures availability of AssetExplorer when the region or site hosting the application goes down due to a disaster. It implements a hot-standby system by keeping a secondary server in the standby mode while a primary server runs the application. Moreover, the primary and secondary servers are placed in different geographical regions.

     

    The application runs on the primary server while the secondary server monitors the activity of the primary server. When disaster strikes, the secondary server takes over and hosts the application by replicating the data from the primary server. This ensures application availability for users without any data loss.

     

    Role Required: SDAdmin

     

    Currently, disaster recovery is supported only for servers running on Windows Operating System.

     

    This page discusses disaster recovery in the following topics.

     

    Prerequisites

    Server Requirements

    • Two 64-bit server machines with high network connectivity for setting up AssetExplorer application.

    • Two 64-bit database server machines with high network connectivity.

    • AssetExplorer 64 bit .exe installations (preferred).

    • Servers used for the DR setup must have two-way read-write access for the ManageEngine folder (where AssetExplorer is installed).

     

    AssetExplorer Configuration Requirements

     

    • FOS license must be purchased as an add-on to your application edition. You cannot view HA configurations on the Admin tab without applying FOS license to your application.

    • The database must be externalized from primary and secondary servers but remain accessible to both. Preferably, use MSSQL setup.

    • The database must be available in Always On availability groups (AG) setup, preferably in a different geographical location.

    • The application must be started as a service using Java Service Wrapper.

    • Polling and retry parameters should be the same in both servers.

    • Applications installed on the primary and secondary servers must both run on the same port.

     

    How does disaster recovery work?

    The disaster recovery setup involves the following components:

     

    • Primary Server - The server where the application is hosted.

    • Secondary Server - The backup server in standby mode that monitors the health of the primary server and also replicates files from the primary server periodically.

    • Database Server - The primary and secondary servers are connected to a common database server. By default, the database server is used to monitor the primary server's health via DB heartbeat mechanism. The data from the primary server is stored in the database server. This ensures that no data is lost if the primary server fails.

    • The primary server, secondary server, and database server are connected to a different network.

    • The primary server's status can be monitored in two ways:

      • DB heartbeat Mechanism - Default method for monitoring the health of a peer machine. The primary server updates its status to the database server every minute. If no status update is received, it is assumed that the primary server is unavailable and the secondary server takes over.

      • Ping Mechanism - Requires SSL certificate and can be configured only in the HTTPS protocol. In case of database failure, the application automatically sets up this method.

    • The standby server monitors the health of the primary server and replicates files from the primary server periodically.

    • When the primary server goes down, no status update is sent to the database server. The primary server status is updated as "Down" in the database server.

    • The secondary server recognizes primary server failure and instantly begins the takeover process.

    • The secondary server pulls the latest files from the primary server and starts as the primary server. The time taken to complete the takeover depends on the size of replication data.

    • When the primary server is restored back online, it starts as the standby server and the other server continues to act as the primary server. Thus, the cycle continues.

     

     

     Benefits of Launching AssetExplorer in DR mode

    • Availability of AssetExplorer application at all times.

    • Smooth and automatic take-over of the failed server.

    • Prevents data loss in primary server in the event of server failure.

    • When the primary server goes down and gets restored, it will start in standby mode while the other server continues to act as the primary server. Therefore, it is not essential for administrators to restore DR setup each time a hardware/software failure occurs.

    Setup process

    You can set up your servers for DR configurations in two ways:

    • Approach 1: Set up each server individually

    Install AssetExplorer separately in both servers and configure them separately to the same database server.

    • Approach 2: Set up one server and mirror the configurations to the other server

    Use robocopy to mirror the server. Install the application in the primary server and execute the following command from <AE_home>\bin directory: mirrorSetup.bat <IP address of secondary server>

    On mirroring the application, the folders relevant to AssetExplorer will be copied from the primary server to the secondary server. However, the application will not be started as a service in the secondary server. To start the application as a service,

    • In the command prompt, go to {AE_Home}/bin

    • Execute the following command: sd_service.bat -i

    • The service will be created in the secondary server. You can now start AssetExplorer as a service.

     

     

     Set Folder Permissions

    • Share the AssetExplorer folder from the primary server to secondary server and vice versa.

    • Ensure that the servers have mutual full read/write permission for this folder. You can set the folder access permission to Everyone.

    • To access the shared folder on your server, go to from start menu > run > \\<machineIP>\AssetExplorer. Provide the username and password (if needed) to make sure an IPC connection is established between the machines.

    • If you want to restrict the folder access to one particular user account, set the folder access to that account and open run command (Win+R) 'services.msc'.

    • Search for "ManageEngine AssetExplorer" and go to Properties.

    • Select the Log on tab and choose This account option.

    • Enter login credentials of the domain user to whom the access is given and save it.

    • Start the application.

     Configure Disaster Recovery

    • Navigate to Admin > General Settings > HA Configuration.

    • Switch to Disaster Recovery.

    • Select the Enable DR startup mode check box.

    Use the following pointers to configure the application in DR mode.

     

    Attributes

    Description

    Primary Server IP*

    Enter the IP address of the server where the application is running.

    Secondary Server IP*

    Enter the IP address of the standby server.

    Common Alias Name*

    Specify the domain name using which you can access the application. You can customize the alias name as needed.

    In case of disaster notify to

    Specify the email address to which the notifications must be sent in case of server failure. Notifications regarding disaster self-monitoring alerts, data synchronization alerts, and secondary server takeover alerts will be sent to this email address. You can add multiple email addresses by separating them using commas.

    Mandatory fields

     

    The status of the servers is monitored every 5 minutes by default. In the event of server failure, an email is triggered to the configured email address. You can modify the server's listening time by following the steps below:

    • Navigate to {AE_home} / conf.

    • Open the ha.conf file and find the entry: #peer.status.check.time.period=

    • Uncomment the entry by removing the hashtag

    • Specify the required time in minutes. For example: peer.status.check.time.period=10

    For the notification emails to be sent, the outgoing mail server should be configured for the application. You can customize the notification message from Translations.

    • Click Save and restart the application for the configurations to take effect.

     

     

    After setting up DR configuration, set up HA file replication configuration.

    When the secondary server takes over as the primary server, the configurations in the DR page are not auto-updated accordingly. In other words, DR configurations do not reflect which server is primary/secondary unless manually updated.

     

    Network Modifications

    •  Initially, the primary machine IP is mapped to the domain URL/common alias URL.

    • After takeover, the application will be started automatically on the secondary server but you cannot access the application via the same domain URL/common alias URL.

    • Map the secondary machine IP to the common alias URL/domain URL to access the application.

     

    Upgrade/Restore

    Upgrade DR


    Announcement for Users using AssetExplorer version 6971 and above

    Users upgrading to version 6971 and above are not required to mirror the updates in both the servers as the changes are automatically pushed during the upgrade.

     

    To upgrade your application,

    • Invoke <AE HOME>\bin\shutdown.bat in the primary and secondary server. This will stop AssetExplorer.

    • Upgrade the build in the primary server using PPM. Click here to learn how.

    • Mirror the settings in the secondary server via robocopy by invoking <AE HOME>\bin\mirrorSetup.bat <IP address of secondary server> - pass the secondary server IP as parameter.

    • Invoke <AE HOME>\bin\run.bat in both servers to start AssetExplorer.

    Alternatively, you can also start AssetExplorer as a service.

    Note: Ensure that you back up your files before upgrading your application. Click here to know more. 

    Restore DR

    In the event of upgrade failure, you can restore the application to its previous version by following the steps mentioned below.

    • Invoke <AE HOME>\bin\shutdown.bat in the primary and secondary server. This will stop AssetExplorer.

    • Restore the data in the primary server. Click here to learn how.

    • Mirror the settings in the secondary server via robocopy by invoking <AE HOME>\bin\mirrorSetup.bat <IP address of secondary server> - pass the secondary server IP as parameter.

    • Invoke <AE HOME>\bin\run.bat in both the servers to start AssetExplorer.

    Alternatively, you can also start AssetExplorer as a service.

    After successfully restoring the application, DR will be disabled by default. Enable DR and restart the application again.

    Why is DR disabled on restoring AssetExplorer and an application reboot is required?

    In some cases, the application backup from one machine with DR enabled is restored in another machine. In such cases, the application will fail to start because the IP address will be incorrect in the ha.conf file. To avoid such failures, DR will be disabled by default.

    Administrators can enable DR from Admin > General SettingsHA Configuration and restart the application for the changes to take effect.

    Disable DR 

    To disable DR from the application,

    • Go to Admin > General SettingsHA Configuration

    • Switch to Disaster Recovery and uncheck the Enable DR Startup Mode option.

    • Invoke <AE HOME>\bin\shutdown.bat in the primary and secondary servers. This will stop AssetExplorer.

    • You can then restart the application in the servers individually by invoking <AE HOME>\bin\run.bat command. On restarting, the servers will behave individually as two separate machines.

    Alternatively, you can also start AssetExplorer as a service in each server.

    History   

    The History tab records the summary of operations performed on both DR and FOS configuration.

     

     


     

    Zoho Corp. All rights reserved.