Configuring Failover in OpManager

What is Failover?

Failover is an alternative monitoring instance that is used to ensure your network remains monitored even when your primary monitoring setup goes down. OpManager helps in ensuring uninterrupted monitoring of your network by allowing you to configure a secondary monitoring instance on a separate server.

opmanager failover 2

How does Failover work?

The primary server updates a value called heartbeat in the database. The heartbeat value is a counter that is incremented by the primary server at a specific frequency of time. The secondary server monitors the heartbeat value to check if it is being updated in the specified time interval. When the primary server goes down, it will not be able to update the heartbeat  value in the database. If the heartbeat value in the database is not updated for the last 60 seconds, the primary server is considered to have gone down and the secondary monitoring instance takes over. This secondary server will continue monitoring the network as long as it is up. Meanwhile, if the primary server is up (recovered and restarted), it will take the standby mode and let the secondary server continue monitoring.

The information between the primary and secondary instances are synced periodically, thus ensuring that you don't miss critical monitoring data ( such as device status, traps, syslog messages etc., ) when your primary OpManager instance goes down.

What are the prerequisites?

  • Apply the failover add-on: Apply the Failover - Hot Standby Engine (MSSQL only) add-on in your primary instance. You can purchase the add-on for Professional Edition from here and for OpManager Plus from here.
  • Have the database in a separate server: Ensure that the database for your OpManager installation is setup in a separate server and not the same server in which the primary or secondary OpManager instance is installed (MSSQL setup preferred).
  • Create a shared folder in a separate server: Some data in OpManager are stored in files which are present in the local directory. When failover is configured, instead of a local directory, these files are stored in a shared folder that is accessible by both primary and secondary servers . This ensures that there is no data loss when the secondary server takes over the monitoring process.
    Create a folder in a separate server  and share it with both the primary and secondary servers. Ensure that both primary and secondary servers have access to the shared folder with write permission.
    ( Note: The server in which the folder is created should be in the same domain in which your primary and secondary servers are configured. Also, the server in which the folder is created should not be the same server in which the primary or secondary instance is configured). Click here to learn how to share a folder with both primary and secondary instances.
  • Have a virtual IP address: A Virtual IP address is a common IP address that is shared by both primary and secondary server on the same subnet. When the one server goes down, the virtual IP points to the other server. (This feature is available only for OpManager versions 12.5.140 and above.)
  • Hardware and software requirements
    • Both the primary and secondary instances should be installed in Windows systems. 
    • The same version of OpManager should be installed in both servers.
    • Both primary and secondary OpManager services should have the same port and protocol ( http / https ).
    • Both primary and secondary servers should have the same time and time zone.
    • Both primary and secondary servers should have the same hardware configurations.
  • Network requirements
    • Both primary and secondary servers should have a static IP address.
    • The virtual IP should be static and in IPv4 format.
    • The primary server and secondary server should be able to resolve each other's host name and IP address.
    • The IP and virtual IP of both the primary and secondary servers should belong to the same subnet.
    • Both the servers should have high connectivity and bandwidth.
    • The primary, secondary and the server in which the shared folder is created should all be in the same domain.
    • The Syslogs, SNMP traps and Flows are forwarded to the virtual IP address.

How to configure failover instance in OpManager?

    In your primary instance, go to Settings -> General Settings  -> Failover Details and enter the following details:
    opmanager failover 1
    • Secondary Server IP: The IP address or host name of your secondary server.
    • Shared folder path: The path to the empty shared folder created in a separate server. This is generally of the form \\<Server_Name_or_IP>\<Share_Name>.
      Note: Ensure that the empty folder is shared with both primary and secondary servers. Click here to learn how to share the folder with primary and secondary servers.
    • Virtual IP: The virtual IP address. Refer to the pre-requisites to know more about Virtual IP Address.
    • Email address (optional): Receive notifications on failover self monitoring alerts, data synchronization alerts and secondary server takeover alerts. You can specify the email recipients to whom the notifications must be sent. You can specify multiple recipients by separating each email address by a comma.
    Save the details and perform the following steps in the primary and secondary servers:

    opmanager failover 2

    In the primary server:

    • Stop OpManager service.
    • Share the <OpManagerHome> folder to the secondary server. Click here to learn how.
    • Open command prompt / terminal with administrator priviliges, navigate to <OpManagerHome>\bin and execute the following command:
           Clone_primary_server.bat
    • Start the OpManager service.

    In the secondary server:

    • Download the Configure_failover_server.bat file and move it to the folder where you wish to have your secondary instance configured. (Ex: C:\Program Files\ManageEngine)
    • Open Command prompt as administrator, navigate to the <Location> and execute Configure_Failover_Server.bat. 
    • Share the OpManagerHome folder to the primary server. Click here to learn how.
    • Start the secondary OpManager instance.

Note:

  • The option to configure failover with a virtual IP address is only available for OpManager versions 12.5.140 and above.
  • OpManager does not provide any kind of database failover support. It only provides application level failover support.
  • Always start the secondary instance after the primary instance is completely started.
  • The approximate time taken for the secondary server to completely takeover the primary will be 3-4 minutes. There may be a minor loss of data in few SNMP traps, syslogs or flow received during that period.
  • The Syslogs, SNMP Traps, Flow should be forwarded to the virtual IP address.

Upgrading the failover setup: While upgrading your OpManager service, it is enough to apply the PPM for the primary setup. The secondary server will be updated automatically.