IMG 1010 - DSP Auto Recovery

 

 

 

DSP functionality on the IMG 1010 and IMG 1004 determines the number of VoIP channels that can be utilized. If one or more DSP's fail, the number of VoIP channels available is decreased. This adds a new set of issues like not having enough resources available for the traffic being sent through the system. To alleviate the issues, the DSP's initiate an automatic reboot of the failed DSP to bring it back to its running state. After three reboots attempts with no success, the DSP is rendered DEAD and cannot be brought back to its running state unless a total system reboot is instituted. Feature F-6388 DSP Auto Recovery takes the reboot one step further. Feature F-6388 DSP Auto Recovery will wait a set interval after the third reboot and then restart the reboot process. The reboot process continues till the failed DSP either goes back to its running state or in the case of a hardware failure, the module is replaced.

Example

Below is an example of the DSP Auto Recovery feature. In the example, one of the DSP's on VoIP Module 0 goes Out Of Service. The system utilizes the Auto Recovery feature to bring the DSP back into service.

  1. One of the DSP's on VoIP Module 0 fails and some of the IP channels will report back as failed or Out Of Service. The system recognizes the DSP has failed and initiates a series of three reboots to the DSP to try to bring it back into service.

  2. On the first reboot, a DSP Reset Alarm is sent to the EventView application. The DSP after the first reboot did not come back into service so the IMG 1010/1004 reboots the DSP a second and third time. If the DSP does not come into service after the third reboot, the DSP is rendered DEAD and a DSP Dead Alarm is sent to EventView.

  3. Since the DSP did not come back into service after the third reboot, F-6388 DSP Auto Recovery is initiated and a timer is started.

  4. The DSP Auto Recovery timer will count down from 1 minute. Once the DSP Auto Recovery Timer expires the reboot process described above will once again be initiated. In this example, the 2nd series of reboots fail to bring the DSP back into service. After the third reboot in this process, the Auto Recovery Timer is started again but this time the timer is doubled to 2 minutes.

  5. The DSP Auto Recovery timer will now count down from 2 minutes. Once the DSP Recovery timer expires the reboot process described above will once again be initiated. The reboot process again fails to bring the DSP back into service and the DSP Auto Recovery Timer is re-initiated at double the previous timer value which in this case is 4 minutes.

  6. This process continues as displayed in the table below.

DSP Auto Recovery boot process

DSP Auto Recovery boot process

DSP goes into a failure state for whatever reason. The IMG initiates three reboots

  • Reboot 1 – DSP Reset Alarm sent

  • Reboot 2 – (DSP still in failed state)

  • Reboot 3 – (DSP still in failed state) - DSP Dead Alarm is sent to EventView.

DSP Auto Recovery Timer is started. The timer is initially set at 1 minute. DSP Auto Recovery Timer Expires after 1 minute and the reboot process is initiated again. A DSP Recovery Retry Alarm is sent to EventView.

  • Reboot 1 – DSP Reset Alarm sent

  • Reboot 2 – (DSP still in failed state)

  • Reboot 3 – (DSP still in failed state) – DSP Dead Alarm is sent to EventView.

DSP Auto Recovery Timer is started. The timer is initially set at 2 minutes. DSP Auto Recovery Timer Expires after 2 minutes and the reboot process is initiated again. A DSP Recovery Retry Alarm is sent to EventView.

  • Reboot 1 – DSP Reset Alarm sent

  • Reboot 2 – (DSP still in failed state)

  • Reboot 3 – (DSP still in failed state) – DSP Dead Alarm is sent to EventView.

DSP Auto Recovery Timer is started. The timer is initially set at 4 minutes. DSP Auto Recovery Timer Expires after 4 minutes and the reboot process is initiated again. A DSP Recovery Retry Alarm is sent to EventView.

  • Reboot 1 – DSP Reset Alarm sent

  • Reboot 2 – (DSP still in failed state)

  • Reboot 3 – (DSP still in failed state) – DSP Dead Alarm is sent to EventView.

DSP Auto Recovery Timer is started. The timer is initially set at 8 minutes. DSP Auto Recovery Timer Expires after 8 minutes and the reboot process is initiated again. A DSP Recovery Retry Alarm is sent to EventView.

  • Reboot 1 – DSP Reset Alarm sent

  • Reboot 2 – (DSP still in failed state)

  • Reboot 3 – (DSP still in failed state) – DSP Dead Alarm is sent to EventView.

DSP Auto Recovery Timer is started. The timer is initially set at 16 minutes. DSP Auto Recovery Timer Expires after 16 minutes and the reboot process is initiated again. A DSP Recovery Retry Alarm is sent to EventView.

  • Reboot 1 – DSP Reset Alarm sent

  • Reboot 2 – (DSP still in failed state)

  • Reboot 3 – (DSP still in failed state) – DSP Dead Alarm is sent to EventView.

DSP Auto Recovery Timer is started. The timer is initially set at 32 minutes. DSP Auto Recovery Timer Expires after 32 minutes and the reboot process is initiated again. A DSP Recovery Retry Alarm is sent to EventView.

  • Reboot 1 – DSP Reset Alarm sent

  • Reboot 2 – (DSP still in failed state)

  • Reboot 3 – (DSP still in failed state) – DSP Dead Alarm sent is sent to EventView.

DSP Auto Recovery Timer is started. The timer is initially set at 32 minutes. DSP Auto Recovery Timer Expires after 32 minutes and the reboot process is initiated again. A DSP Recovery Retry Alarm is sent to EventView.

  • Reboot 1 – DSP Reset Alarm sent

  • Reboot 2 – (DSP still in failed state)

  • Reboot 3 – (DSP still in failed state) – DSP Dead Alarm sent is sent to EventView.

At this point the reboot process will continue at 32 minute intervals. If the DSP finally comes into service a Clear DSP Reset Alarm and Clear DSP Dead Alarm is sent to EventView.

Alarms

In EventView, a new alarm is added. The new alarm is labeled DSP Recovery Retry and is set as an Informational Alarm. This alarm is sent to the EventView application after the DSP Auto Recovery Timer has expired and the reboot process is initiated.

Additional Information

  • Auto Recovery is always enabled. No configuration is required.

  • The Auto Recovery is initiated automatically once the DSP is rendered DEAD or Out Of Service.

  • If all 4 DSP's on one DSP Module are continuously resetting then the IMG automatically initiates a DSP Module reset. If the DSP Module is reset 3 times without coming back into service, the module is declared DEAD and need to be fixed or replaced.

 

Return to Documentation Home I Return to Sangoma Support