Versions Compared

Old Version 1

changes.mady.by.user Nathaniel Halbrooks

Saved on

compared with

New Version Current

changes.mady.by.user Kapil Gupta

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

Note

This information on this page applies only to PBX versions 15.x and above. For version 14.x or lower please see: Warm Spare Setup

Table of Contents

Introduction

...

The Warm Spare setup on PBX 15+ has changed from the way it used to be in older versions.  
We are now using the Filestore ,  API and Backup & Restore modules for the Warm spare Backup configuration, so we need to ensure that all 3 mentioned modules are present in both Primary and Secondary server.

...

the process needs to send data from the primary to the warmspare. We can either use FTP or SSH protocol to perform this task using Filestore module (SSH is the recommended path) 

SSH Configuration 

In this section, we will describe how to configure "SSH" to use for Warm Spare setup. 
First, we will set up shared keys between the two servers so they can communicate across SSH on port 22. PBX 15+ onward there is no need of generating SSH Keys for the Primary server. Backup & Restore module will take care of generating SSH Keys by itself. But we have to copy the Primary Server SSH Keys to Secondary server to ensure that Primary can communicate with Secondary server easily.
We can use any one of the following 2 methods to copy the SSH keys to Secondary (Warm spare) server. 

 

  1. Manually copy the SSH Keys to Secondary Server

"FreePBX GUI - > Admin → Backup & Restore → Global Settings" has server SSH keys which we can copy to Secondary server manually. (note some browsers may not let you copy this data)

You may copy this key to spare/Secondary server manually to /root/.ssh/authorized_keys file. Create a file called "authorized_keys" (if not present) and add your Public Key in that file. If that file already exists just add your Key to the end of the file (make sure each key is separated by a new line!)

...

  1. Using SSH CLI command to copy Primary SSH keys to Secondary server

(preferred method)

Note

Please note if you are doing this from a fresh install and have never visited the backup and restore module these steps will FAIL.

Before proceeding go to the backup and restore module in the GUI , this triggers creation of keys referenced below.

Once in the module click on the global settings tab and confirm the key is present as shown above

Login to your Primary server with an SSH client such as PuTTy, SecureCRT, or other SSH client.

We will copy the key to the Secondary server with the help of  the following command so that the primary server can SSH to the secondary server without needing a password.  

At the primary server Linux CLI prompt type: sudo -u asterisk ssh-copy-id -i  /home/asterisk/.ssh/id_rsa.pub root@SecondaryServerIP and enter the password when prompted. 

Info

APPLICATION NOTES

Make sure you replace the SecondaryServerIP with the IP Address of your Secondary PBX. (use IP and not a hostname that may be common to both primary and warmspare; if fqdns are desired create 3 records the common name , specific name for primary and specific name for warm spare - ie mypbx.company.com , mypbx1.company.com , mypbx2.company.com)

If the Firewall is configured, pay attention to creating the right rule allowing the two servers to talk to each other.

If this command completes without error, you are ready to test:

At the prompt type: ssh -i /home/asterisk/.ssh/id_rsa root@SecondaryServerIP
If all went well, you should now be logged in to the Secondary Please refer to Setting up the SSH key section to set the SSH association between primary and warmspare server.

Filestore Module Setup 

Once SSH Setup done, we have to configure the Filestore module for the SSH.

Please follow the guide Filestore Module#SSH to perform the SSH configuration on the primary server.

...

We have to enable FTP protocol on the Warm Spare (Secondary) server by following System Admin - Provisioning Protocols#ProvisioningProtocols-Enable/DisableFTP 

Info

filestore FTP

If you have selected the FTP filestore, make sure that your FTP username and password are not removed from the Warm Spare Server
during the Restore process(System Admin→Provisioning Protocols) to overcome this situation, you can use the same FTP username and password
on the Warm Spare Server as the Primary Server. Or, you can create a separate FTP user for the Filestore module to access the server.

...

Once FTP Setup as mentioned as above done then we have to configure "Filestore" module for the FTP.

Please follow the guide Filestore Module#FTP to perform the FTP configuration i.e. Add Warm Spare (Secondary) server.

...

  1. Create the Application Machine to Machine App

    Image Removedimage2022-9-22_13-49-44.pngImage Added

  2. Allow the scope(gql:backup) to access the backup module and then "Add Application".

    Image Removedimage2022-9-22_13-50-40.pngImage Added

  3. Copy the Access details to notepad or text editor, you will need this info when we setup the replication job on the Primary server.

    Image Removedimage2022-9-22_13-52-7.pngImage Added

Info

SSL Certificates

If you are using self Singed Certificates, DO NOT use HTTPS in the below URLs on primary server Which can result this error (cURL error 60: Issuer certificate is invalid. )

1. Token URL
2. Authorization URL
3. GraphQL URL
4. Rest URL

Info

HTTP / HTTPS ports

When the PBX WebUI is configured to use non-standard ports for HTTP (80) / HTTPS (443), then you need to include the port number in the URLs.
For example when port 2001 is configured for HTTP the Token URL will look like this:

http://192.168.3.249:2001/admin/api/api/token

How to setup a Warm Spare Backup job

Build your backup job as you normally would however make sure you are adding your Warm Spare server as one of the backup Storage Locations (note you can select multiple Storage Locations here)Image Removed

...

Enable the "Warm Spare" option.

...

Once we enable the "Warm spare" option. Now you can see the Warmspare configuration options as shown below.

...


Warm Spare Server Input fields

Image Removed
image2019-10-24_13-10-52.pngImage Added

The Warm Spare settings will show up once "Enable" is set to "Yes"

Image Removed
image2021-4-27_14-26-24.pngImage Added

Image Removed

image2019-10-24_14-40-34.pngImage Added

Should NAT settings from primary system be restored to the Warm Spare system?

Image Removed
image2019-10-24_14-41-7.pngImage Added

Should Bind Address settings on the primary system be restored to the Warm Spare Server ?

Image Removed
image2019-10-24_14-47-39.pngImage Added

Should DNS settings on the primary system be restored to the Warm Spare Server ?

Image Removed
image2019-10-24_14-48-12.pngImage Added

Should we run "Apply Configs" on the Warm Spare Server after a restore is completed?

Image Removed
image2019-10-24_14-50-43.pngImage Added

The Warm Spare Server which is added in the Filestore module (FTP/SSH)

Image Removed

image2019-10-24_14-51-57.pngImage Added

Warm Spare server Access Token URL

Image Removed

image2019-10-24_14-53-54.pngImage Added

Client ID generated on the Warm Spare Server API

Image Removed
image2019-10-24_14-54-33.pngImage Added

Client Secret generated on the Warm Spare Server API

Image Removed
image2019-10-24_14-55-17.pngImage Added

GraphQl URL Generated on the Warm Spare Server API

Image Removed
image2019-10-24_14-56-18.pngImage Added

This is the Access Token which will be generated on the Warm Spare Server

Image Removed
image2019-10-24_14-57-15.pngImage Added

By clicking on this button, you can TEST the connectivity between the servers

Image Removedimage2021-4-27_14-29-57.pngImage Added

Exculde Certman while restoring the backup

If this option is set to yes then certificate will not be restored which require HTTPS config to be rebuild manually with spare server certificate.

Image Removedimage2021-4-27_14-31-7.pngImage Added

Exclude Trunks

While restring the backup exculde the trunks from primary server

...

Warm spare job output will be something like below and at the end "Restore Status" will display the response from the API (which triggered the restore process in warm spare server).

...

Info

IMPORTANT Operational notes:

  • be certain to omit the backup module from the warm spare job's list of modules to backup and restore;  otherwise the job will be replicated to the warm spare in an active state and will attempt to run causing general chaos and mayhem.

  • if you see warm spare job failures related to a truncated key or connectivity issues  insure that the IP of both the primary itself and the warm spare are defined in firewall as trusted ON THE PRIMARY... if not during the restore those items configured in firewall are on the warm spare are overwritten by that which is restored from the primary and if the primary's IP is not defined as trusted on itself , when those rules are replicated to the warm spare access from the primary to the warm spare can be impacted causing the job to fail 

  • Sipstation users need to have a primary and a warm spare trunk group; trunks are to be individually configured on the respective deployment with the primary trunk group set up to failover to the warm spare trunk group. Additionally within the warm spare options, "Exclude Trunks" must be flagged as YES and the sipstation module must be omitted from the warm spare job.