Digital Card Self Test

The following guidelines apply only for E1/T1 Sangoma cards. You will need to be logged into the Linux CLI for the entire process

In short, the load test will digitally loopback all your ports on all your Sangoma cards and run a test over a specific time interval, where you can them monitor the system health for sanity.

 Below is a diagram of how the load test works.  The Wanpipe driver is configured for API mode, where an application called BERT is used to run the load test through the attached Sangoma hardware.

Load test scenario

Below are the steps required in order to run a load test

1. Backup your complete Wanpipe configuration so that you can restore your production setup. (you do not need to backup your DAHDI or Asterisk configuration as the production test is simply API application for Wanpipe.

   1. cd /etc/

   2. tar cfz wanpipe-backup.tgz wanpipe/

   3. mv wanpipe-backup.tgz /root
      

   4. your entire wanpipe backup is now located in /root called wanpipe-backup.tgz

2. Download required scripts into your wanpipe directory (use ‘wget’ to download to autodownload directly into your CLI)

   1. cd /etc/wanpipe/scripts/

   2. wget https://ftp.sangoma.com/linux/script/loop.sh

   3. wget https://ftp.sangoma.com/linux/script/mpd.sh
      

3. Make the scripts executable by typing:

   1. chmod +x loop.sh

   2. chmod +x mpd.sh

      * If these scripts already exist in /etc/wanpipe/scripts directory, disregard step 2 and continue.

4. The following command auto-detects all your ports on all your installed Sangoma cards in the system and enabled MASTER clocking on each port.  MASTER (internal) clocking mode is required in order to put the card(s) in digital loopback mode.

   1. For T1: wancfg_tdmapi --fe_media=T1 --fe_clock=MASTER –silent

   2. For E1: wancfg_tdmapi --fe_media=E1 --fe_clock=MASTER –silent

5. Start the Sangoma Wanpipe driver

   1. Type: wanrouter start

   2. Configure hardware in digital loopback mode

      1. cd /etc/wanpipe/scripts

      2. ./loop.sh     #set all spans to digital loopback

      3. ./mpd.sh     #set hardware echo canceller into power down mode, if you have one)

   3. Run the production test (BERT)

      1. cd /etc/wanpipe/api/libsangoma/examples/hdlc_test

      2. type: make  #compile hdlc test

         1. * if you run into errors here, please download make and gcc

      3. ./tdm_hdlc_test.sh <span> <chan> <timeout in seconds>
         example for an A104 4-port E1 configuration:
         ./tdm_hdlc_test.sh “1 2 3 4” all 10

         This will run production test on all 4 ports, on “all” channels for 10 seconds
         Increase the timeout to a larger interval if you with to monitor system using some utilities (i.e. 300 seconds)

Monitoring server while running load test

While the result of the load test is important, it is also important to run a few commands during the load test to monitor.

1. In a separate terminal screen run the ifconfig command and monitor for ‘overruns’.  If the ‘overruns’ counters for your Wanpipe ports increment  during the load test, this means that your server is not able to efficiently handle the data sent from the ports to your kernel.

   The following command will provide you a live output of ‘ifconfig’ so that you can monitor overruns in real time:

   watch –n 1 “ifconfig |grep “overruns””

   The above output shows low amount of overruns.  This is perfectly fine during the first few seconds as everything is trying to sync up.  If the counters continue to increment every second, there is a problem.

   If you experience overruns while running the Wanpipe load test, your entire issue is system related (and not the lines plugged into the system). Run the steps indicated on this page: PRI Audio Issue
   

2. Run the Linux vmstat command to monitor the system CPU usage during the performance test.  Run the following line to watch the output in real time

   -> vmstat –n 1

   

   The numbers in the red outline above indicate the entire CPU (all cores) idle, per second.  You do not want your server idle CPU usage to go below 60% during the production test.  Keep in mind that you will possibly be adding other applications to this system, so anything lower than 60% system idle may cause load issues in the future.  However, if you are not adding any application, and your system item goes down to 20%, your system may perform fine. Simply monitor the rest of the items listed here to conclude system sanity.
    

3. Monitor the /var/log/messages file during production test.  If there are any issues relating to hardware, overruns, NMI, and physical layer, they will display in this file.  If there are any messages that present themselves that appear somewhat concerning, please contact Sangoma support at support.sangoma.com or visit our online documentation for troubleshooting at wiki.sangoma.com

If all the above steps product successful output, you can now stop the load test and stop monitoring the above commands. Stop the wanpipe driving with ‘wanrouter stop’ command and now restore your Wanpipe configuration (that we saved initially)  and overwrite the current /etc/wanpipe directory.