7.3 Restoring a Failed Database Service

If the Database Service component fails but the other components are still running, you can power off the other components, redeploy the Database Service from backup, and then restart the other components in the proper order. We strongly recommend that you cluster the Database Service component and maintain regular backups. For more information, see the following topics:

IMPORTANT:Secure API Manager uses a master/master database setup to provide enhanced database performance due to high availability. This means that if a primary database fails and cannot simply be restarted, the secondary database must become the primary database. After that, you can add a secondary database to provide backup and high availability again.

In the event of database failure, the first step is to determine whether the appliance-name/var/opt/microfocus/sapim/mount/mf-sapim-postgres/ directory is still available on a Database Service node.

  • If the directory is available, you can follow the steps below to restore the Database Service from the backup, beginning with the primary database node.

  • If you do not have this directory, it means that all databases failed without backup information. If all nodes have failed and you cannot access any database through the command line, you must completely remove and recreate the database cluster.

IMPORTANT:Assuming you have clustered and backed up the Database Service component, you must recover the primary database node before any other nodes. Complete the following steps to restore the primary database node, then restore the other database nodes.

To restore the Database Service from a backup:

  1. Power off all of the other components and delete the Database Service component appliance from VMware.

  2. Deploy an appliance for the Database Service component with the same networking configuration as it had before the failure. For more information, see Deploying the Secure API Manager Appliances.

    IMPORTANT:You must use the same network configuration for this appliance as it had before the failure, otherwise the restoration fails. The directory contains the databases for your system which includes the network settings.

  3. On the appliance that will become the Database Service component, paste the backup directory appliance-name/var/opt/microfocus/sapim/mount/mf-sapim-postgres/ to the new appliance.

    The Deployment Manager creates and populates this directory when you deploy a Database Service component. When the Deployment Manager detects that the directory already exists, it will not overwrite the information in the directory and it maintains all of the configuration information and APIs in the database.

  4. On the appliance that will become the Database Service component, paste the following two files that you included in your backup plan to the appliance-name/var/opt/microfocus/sapim directory:

    • deployment-reference.json

    • createPostgresContainer.sh

  5. Restart the Deployment Manager web application on the appliance that will become the Database Service component by issuing the following command at the appliance command prompt:

    systemctl restart vabase-jetty
  6. From the command prompt in the appliance-name/var/opt/microfocus/sapim directory, execute the following two shell scripts in the order listed:

    ./createPostgresContainer.sh 
    ./sapim-postgres-start.sh
  7. Redeploy the Database Service component using the Deployment Manager, ensuring that you use the same database user name and password for the database. When saving the configuration, select Save configuration and deploy only this appliance. For more information, see Deploying the Secure API Manager Components.

  8. (Conditional) If you have additional database nodes to redeploy, complete Step 2 through Step 7 for each additional node.

  9. Power on the other Secure API Manager components in the proper order, ensuring that each component is up and communicating before starting the next component. For more information, see Restarting Secure API Manager in the NetIQ Secure API Manager 1.1 Administration Guide.

IMPORTANT:We strongly recommend that you cluster the Database Service component and make regular backups of the nodes in the cluster. If all Database Service nodes fail and you have not implemented a backup plan to capture a snapshot of the persistent database files, then the system is completely gone and you must recreate your entire system.