Troubleshooting DSP failures

Introduction and Purpose

The purpose of this guide is to provide list of reason of DSP Failure and how to troubleshoot the same.

VEGA GUI shows DSP Related information under "Media" section of "Status" Tab (i.e. Status -> Media) as shown below:

Open

Now, Media section basically has two different sections:

1. DSP Status

2. DSP Information

DSP Status

DSP status show the status of DSP on VEGA i.e. whether DSP is  Active state or DSP state is FAILED as shown below:

Open

DSP Information

DSP Information will provide all the necessary information with respect to DSP channels status etc if active else provide the reason for DSP Failure.

DSP information when DSP Status is Active as shown below:

DSP information when DSP Status is FAILED as shown below:

DSP Failure Reason

There can be number of reason of DSP Failure as stated below:

1. Hardware Failure (HWFAIL)

2. Unavailable (UNAVAIL)

3. DSP Image download failure (SWFAIL)

4. DSP stopped responding (SWFAIL)

5. DSP disabled (UNAVAIL)

6. DSP fatal runtime error (SWFAIL)

NOTE: UNAVAIL stands for Unavailable, HWFAIL stands for Hardware Failure and SWFAIL stands for Software Failure

Troubleshoot DSP Failure

As explained DSP Failure may be due to multiple reason. Now if order to troubleshoot DSP failure there are two steps:

Step 1) Check if DSP Failure is due to DSP upgrade Required?

If DSP is Failed due to DSP Upgrade then in such case DSP Upgrade button with a help button will appear on to VEGA GUI in front of DSP Status as shown below:

In order to get further information with respect to the same please refer to DSP Upgrade

To confirm DSP Failure is due because mismatch of DSP version please goto  "CLI Command" Option as present with in "Advanced" Tab and type "dsp show version" (i.e. Advanced -> CLI Command) as shown below:

Now, Check if the output of above command has different "VEGA Supported DSP Firmware Version" and "VEGA Octasic Chip Version" then we are sure the the cause of DSP Failure is mismatch of DSP version
and DSP upgrade is required as shown below:

NOTE: The same command can be executed on VEGA CLI in order to get the same output.

Step 2) Take appropriate action

Now if the DSP Failure is due to mismatch of DSP version i.e. DSP upgrade is required then please follow the steps as stated here: DSP Upgrade Procedure

It might be possible that the DSP Upgrade itself get failed. 

TroubleShoot DSP Upgrade Procedure Failure

The possible reason of DSP Upgrade Failures are:

1. Invalid DSP Firmware uploaded

2. Checksum mismatch

3. Improper Space present on VEGA in order to upload DSP

Invalid DSP Firmware / CheckSum Mismatch

In case of Invalid DSP Firmware Uploaded and Checksum mismatch  the UI should display proper information on to VEGA screen this means that that either the DSP Firmware has been uploaded on to VEGA or the DSP Firmware is not properly copied on to VEGA.

DSP Failure due to no space available on to VEGA

It is possible that the DSP Upgrade Procedure will get Failed with no proper information present on to the VEGA GUI. In such case please follow below steps:

• Try to uploaded DSP Firmware manually using SCP operation on to VEGA /tmp/ folder via 2022 port (either using winscp from windows or using scp command from linux)
  one can enable 2022 port on to VEGA as provided in Advanced Debugging link

• Now if the scp if failed due to space not available on to vega then check if there is any core file present in / folder or any invalid file present within VEGA on VEGA then remove the same and try upgrade procedure again. If still the DSP Upgrade procedure is failed the please contact sangoma support.

If DSP Failure is is not because of DSP Upgrade required then follow below steps:

• Logon to vega console and start pcap capture as stated here: How to Take pcap capture on VEGA

• Now, issue below command on vega cli:
  admin>shell vegad restart
  
  

• Once this is done and vega is up again i.e after 3 to 4 minutes stop the pcap trace

• Now, raise a ticket with the pcap attached  sangoma support support.sangoma.com and the issue details.