Dialogic DMG1000/2000 Media Gateways Logging Primer

1 Summary
2 Introduction
3 How much logging can be collected
4 How to enable debug
5 Debug message formatting
6 Debug values

Summary

A guide to DMG1000/2000 Media Gateways logging features and functionality

Introduction

The Dialogic DMG1000 Media Gateway Series and the Dialogic DMG2000 Media Gateway Series both contain an extremely granular firmware debugging system that allows the user to enable debug messages down to a very low level. Such a system provides the ability to debug very specific issues without having to enable large amounts of data and then search through voluminous logs.

This technote details the available trace keys, what subsystem they are related to, what relevance they have and a small example of what their debug looks like.

How much logging can be collected

The amount of logging that can be collected by either gateway model, varies due to hardware differences. Log storage on the gateway is done by default via a circular buffer with a limited size. New entries are added to the end of the buffer and old entries are pushed off the top.

DMG1000 Series Gateways will store 1.5 MB of debug data.
DMG2000 Series Gateways will store 25MB of debug data.

How to enable debug

Firmware debug can be enabled by either the web interface or the Telnet maintenance interface. Note: Debug can also be enabled by the serial maintenance interface, but this method is NOT recommended as the amount of data collected can overflow the serial interface resulting in missed debug data making it difficult to find problems.

To enable debug via the web interface, do the following:

Log into the gateway via a web browser
Navigate to the Diagnostics page via the main menu on the left of the browser window
Select the Trace Capture tab, where you will be presented with a page containing an array of check boxes that you will use to enable your debug settings.

When logging is enabled via this method, your only option is to have the data collected to the local buffer for download via the web interface.

The debug mask is configured by placing a check mark at the intersection of the firmware subsystem and the level of debug that you want. Once you complete your selections, click the apply masks button to set the trace mask and then click the button to start the traces. When you are ready to gather your trace file, click the button to stop the traces and then right click on the trace.log link and select the 'Save Target As' to save your trace file to a local hard disk for review. It is recommended that you follow the right click option instead of just clicking on the link since right-clicking will ensure that you will have the option to save the file first. If the file opens automatically in a viewing application because of a file association (Windows® Notepad, for example)

To enable debug via the Telnet interface: Log into the gateway via a telnet client; this provides you the option of either allowing the data to be saved to the circular buffer on the gateway or to have it streamed to the telnet console for capture by the client application. If you choose the first option, you will still need to collect the debug output via the web interface as described above. If you select the second option, you will need to use a telnet client that allows capturing the output to a file (the Windows® command line telnet client cannot do this but HyperTerminal or Symantec ProComm Plus will) and you will also need to stay connected for the duration of your trace collection. When you are done entering in your trace mask selections, you will need to exit from the command prompt by using the 'exit' command before the traces will be fully started.

The debug mask is configured by using the 'trace' command followed by the arguments you want to enable. For example, if you want to turn on traces just to see the SIP VoIP protocol messages for a call and to let the data be streamed to the terminal window you would use the following sequence (bold type is input form the user):

PIMG-Admin> trace voip prot on
OK
PIMG-Admin> trace on
Trace messages to console started
PIMG-Admin> exit
Good-Bye
.
.
.

From this point on the traces will appear on the terminal screen and need to be captured by the telnet client application for subsequent review.

For cases where you may want logs running in a permanent fashion while in a production environment or in which you need to gather debug data over an extended period of time, you can also have the bug output written out over a network connection to a syslog server. This can be configured via the Traps and Alarms section of the Gateway Advanced settings tab, where you can specify the server name and what data you wish to be logged to that server.

It is important to understand the implications of logging to a syslog server before enabling this configuration. If your network connection is bandwidth-constrained, then you may want to be careful with regard to the amount of data you enable in this logging since the amount of data being logged has a direct impact on the amount of network bandwidth that will be consumed by this logging. It is often preferable to leave syslogging to specific errors and events while reserving diagnostic logging for conditions when you are troubleshooting a specific problem.

Debug message formatting

The debug data that is collected is written out in a standardized human-readable format. The details of the trace lines is shown below:

Column 1	Column 2	Column 3	Column 4
623:08:964	[Tel-1 ]	Event	Lamp 46:CallApp0:0 FLASH

Column #1 contains a timestamp in the format of minutes:seconds:milliseconds. This represents the time since the gateway was rebooted and will wrap around once the 999:59:999 time is reached.

Column #2 contains the firmware subsystem from which the debug message was emitted from. If the message is related to a specific channel then the text will be followed by a 1-based channel number.

Column #3 contains the debug message level that was selected in the steps above.

Column #4 contains the actual debug message text.

The next section contains a list of valid values for the text seen in both columns 2 and 3.

Debug values

NOTE

The data that follows may contain trace samples that have been adjusted for size or for line length. Where data has been removed this will be indicated by an ellipsis (...).

What follows below is a listing of important debug trace masks, what they capture, when they are best used, and to what models they are relevant. Where notable, samples of trace data will be shown to help point out important details that may help the end user in troubleshooting issues in the field.

Trace masks not included in this technote are eliminated because they either do not provide any relevant data to the end user and / or are typically used by engineers in a lab environment.

Firmware Area	Tel
Description	Debug from the telephony interface used by the 1000 series digital station set gateways and the T1 interface of the 2000 series gateways. This debug includes all actions taken on the physical interface including hook state transitions, display and indicator updates, key presses and digital station set protocol.
Relevant Dialogic Gateway Models	Dialogic 1000 Media Gateways Series models DMG1008LSW, DMG1008DNIW, DMG1008RLMDNIW, and DMG1008MTLDNIW Dialogic 2000 Media Gateways Series models DMG2030DTI, DMG2060DTI, and DMG2120DTI
Relevant Levels	prot - OOn the Dialogic 1000 Media Gateways Series models, this provides the digital station set protocols in raw (but compressed) form. The digital station set protocol is not readable by anyone but Dialogic engineering and support staff, so this should not be enabled unless under their direction while troubleshooting an issue. Depending upon the integration being traced, this can emit large amounts of data that is often not needed during the course of troubleshooting a problem. On the Dialogic 2000 Media Gateways Series models, this is used in a robbed bit mode since this debug provides useful information related to bit transitions. So if you are dealing with call control issues on a T1 robbed bit interface, this debug is required. event - Provides data on the physical events that can take place durring the processing of calls such as hook state changes, display and indicator updates, button presses, etc. This data is required to troubleshoot any problems related to call control or call integration so it should almost always be enabled. When in doubt enable this trace level. code - Data showing the actions taken and how they relate to actions taken in the gateway application level. This debug can have some relevant use, for example, when trying to troubleshoot call control issues as it shows how the tel layer transitions through states based on the phyisical actions/event of this interface as well as from commands from the gateway application above it.
When Used	This firmware area is one of the most common to turn on, and is helpful in nearly all troubleshooting sessions because it provides very useful state information that helps monitor the state and actions taken while processing calls. In the sample below note the following: In example 1, the tel event entry on line 09 shows the raw display text of the virtual phone on channel 7. In example 1, the tel prot entry contains the compressed digit station set protocol in this example. In example 1, the tel event entries at several lines are showing key presses (PRS) and releases (REL) as the gateway application uses the virtual phone to perform an action (in this case a message waiting request). In example 1, the tel event entries on several lines show indicator events (EVT-Lamp) as the virtual phone transitions through its various actions. In example 2, the tel code and tel prot debug shows the actions taken to perform a double hook flash on a T1 robbed bit line. Note that you can see the bit transitions in the prot debug and the actions taken on the code lines. Note also how you can see the timing of each hook flash by observing the time stamps for each debug entry. In example 3, the tel code entries here show the parsing and processing of an ISDN setup message to collect the various calling and called numbers and name sections.
Sample	Example 1 (Avaya digital station set debug) 01 143:31:044 [Tel-7 ] Event Key PRS 60:CallApp0 02 143:31:098 [Tel-7 ] Event Voice On Speaker 2 03 143:31:100 [Tel-7 ] Event EVT-Lamp 18:Speaker:1 OFF->STEADY 04 143:31:182 [Tel-7 ] Event EVT-Lamp 60:CallApp0:0 OFF->STEADY 05 143:31:184 [Tel-7 ] Event Key PRS 27:Feature6 06 143:31:380 [Tel-7 ] Event EVT-Lamp 27:Feature6:0 OFF->STEADY 07 143:31:480 [Tel-7 ] Event Key PRS 8:3 08 143:31:520 [Tel-7 ] Code NIM_CB_DISPLAYSTABLE 1 09 143:31:522 [Tel-7 ] Event 2:40:80 \|a= \| 10 143:31:580 [Tel-7 ] Event Key RLS 8:3 11 143:31:680 [Tel-7 ] Event Key PRS 7:2 12 143:31:680 [Tel-7 ] Prot PIMGXX021240D89878DA4AB00214D53F9DD8D276AC8...B8013DB774 13 143:31:680 [Tel-7 ] Prot PIMGXX020300FA42605 14 143:31:780 [Tel-7 ] Event Key RLS 7:2 15 143:31:884 [Tel-7 ] Event Key PRS 9:4 16 143:31:980 [Tel-7 ] Event Key RLS 9:4 Example 2 (T1 robbed bit debug) `01 875:45:392 [Tel-2 ] Code FlashHook (500 ms) 02 875:45:392 [Tel-2 ] Code tenimDoHook(on) 03 875:45:392 [Tel-2 ] Prot CAS Signaling Tx: x0 04 875:45:872 [Tel-2 ] Code tenimDoHook(off) 05 875:45:872 [Tel-2 ] Prot CAS Signaling Tx: xF 06 875:46:352 [Tel-2 ] Code FlashHook (500 ms) 07 875:46:352 [Tel-2 ] Code tenimDoHook(on) 08 875:46:352 [Tel-2 ] Prot CAS Signaling Tx: x0 09 875:46:832 [Tel-2 ] Code tenimDoHook(off) 10 875:46:832 [Tel-2 ] Prot CAS Signaling Tx: xF` Example 3 (ISDN protocol parsing) 01 255:08:400 [Tel-1 ] Event l4_appl() N_STAT_IN connid:65535 cause[17:00] datalen:16 02 255:08:400 [Tel-1 ] Event _isdnLiOnConnectionStatus() FACILITY_IND [17:00] 03 255:08:400 [Tel-1 ] Code Invoke Facility [0xA1]: 02 02 C2 CD 02 01 29 03 05 00 80 00 00 00 04 255:08:400 [Tel-1 ] Code invokeId: [49869] 05 255:08:400 [Tel-1 ] Code opcode: [41] 06 255:08:400 [Tel-1 ] Event l4_appl() N_CALL_PRES_IN connid:27 cause[00:00] datalen:129 07 255:08:400 [Tel-1 ] Code _isdnLiOnConnectionIndication() isdn conn:27 tdm:12 08 255:08:400 [Tel-1 ] Code _isdnLiAssignTdmChannel(12) 09 255:08:400 [Tel-1 ] Code calling party number 10 255:08:400 [Tel-1 ] Code numbering plan: [Private:0x9] 11 255:08:400 [Tel-1 ] Code type of number: [L2 Regional:0x10] 12 255:08:400 [Tel-1 ] Code pres ind [Allowed:0x0] 13 255:08:400 [Tel-1 ] Code screening ind: [Network:0x3] 14 255:08:400 [Tel-1 ] Code number digits: [1234567] 15 255:08:400 [Tel-1 ] Code called party number 16 255:08:400 [Tel-1 ] Code numbering plan: [Private:0x9] 17 255:08:400 [Tel-1 ] Code type of number: [L0 Regional:0x40] 18 255:08:400 [Tel-1 ] Code number digits: [8700] 19 255:08:400 [Tel-1 ] Code _isdnLiOnSuppServicesIndication(27) 20 255:08:400 [Tel-1 ] Code ss_element: [Facility:1] 21 255:08:400 [Tel-1 ] Code ss_identifer: [callingName:30:0] 22 255:08:400 [Tel-1 ] Code ss_pdu: [Invoke:1] 23 255:08:400 [Tel-1 ] Code name: [CALLER NAME] 24 255:08:400 [Tel-1 ] Code _isdnLiOnSuppServicesIndication(27) 25 255:08:400 [Tel-1 ] Code ss_element: [Facility:1] 26 255:08:400 [Tel-1 ] Code ss_identifer: [cmnInform:35:1] 27 255:08:400 [Tel-1 ] Code ss_pdu: [Invoke:1]

Firmware Area	VoIP
Description	Debug from the SIP stack used by the Dialogic 1000 Media Gateway Series models and Dialogic 2000 Media Gateway Series models listed below. This debug includes all actions taken on the physical interface, including hook state transitions, display and indicator updates, key presses and digital station set protocol.
Relevant Gateway Models	Dialogic 1000 Media Gateways Series models DMG1008LSW, DMG1008DNIW, DMG1008RLMDNIW, and DMG1008MTLDNIW Dialogic 2000 Media Gateways Series models DMG2030DTI, DMG2060DTI, and DMG2120DTI
Relevant Levels	prot - Provides the SIP protocol messages as they are sent and received by the gateway. This is a required debug for issues related to call control or integration as it is the sole messaging layer of this information on the IP layer between the gateway and IP endpoints when using the SIP protocol. code - This data, since it is more internal code related, is typically only enabled under direction of Dialogic support. The debug emitted by this level can help solve some issues but for the most part is not directly useful to the end user directly.
When Used	This firmware area is one of the most common to turn on and is helpful in nearly all troubleshooting sessions because it provides the entire SIP signaling data involved for a call. In the sample below note the following things: The VoIP prot entry starting on line 01 and continuing down to line 29 shows the full invite going out from the gateway to the IP endpoint in response to an inbound TDM call. Note the direction of the arrow is pointing to the left (<----). This direction indicates a SIP message going from the gateway to the IP endpoint. An error in the other direction (---->) indicates a SIP message arriving at the gateway from the IP endpoint. Thsi allows you to track the SIP message transactions as they happen. The VoIP prot entry starting on line 01 and continuing down to line 29 shows not only the full Invite but also the full SDP data sent to the IP endpoint. This helps you to debug possible issues related to media negotiation.
Sample	Example 1

Firmware Area	Adept
Description	Debug from the ADEPT parsing engine used by the Dialogic 1000 Media Gateway Series models and Dialogic 2000 Media Gateway Series models listed below. This debug includes all output shown as the engine reads in and attempts to parse out either digital displays or analog in band DTMF integration streams. This debug is not relevant to T1/E1 ISDN, QSIG, or serial integrations, so it need not be enabled on those integrations.
Relevant Gateway Models	Dialogic 1000 Media Gateways Series models DMG1008LSW, DMG1008DNIW, DMG1008RLMDNIW, and DMG1008MTLDNIW Dialogic 2000 Media Gateways Series models DMG2030DTI, DMG2060DTI, and DMG2120DTI
Relevant Levels	code - This debug provides the actions taken by the parsing engine related to call integration. This debug is required to troubleshoot call integration related issues on the relevant integrations.
When Used	This debug is required to troubleshoot call integration-related issues on the relevant integrations. In the sample below note the following things: In example 1, the entries on lines 1 through 6 show the raw display of a digital set call on a Mitel PBX as the parsing engine ses the data. Note that when seeing this data in the tel traces it will look differently because at that layer there is no concept of rows and columns, only raw text data. This trace level formats that raw data properly before running the parsing rules. In example 1, the entries on lines 7 through 13 show the engine running comparisons on each rule in the configuration looking for a latch. Rules that do not match are marked as failed. In example 1, the entries on lines 14 through 25 show the results of a rule match and how the engine has used the rule to take the display text apart and break it up into its components to be used to integrate the call. In example 2, the entries on line 2 show the raw DTMF integration digits from an in band analog call as the parsing engine see the data.
Sample	Example 1 (Mitel Digital Station Set) Example 2 (Analog Inband Integration)

Firmware Area	SI
Description	Debug from the serial integration interface that shows the raw data between the serial source and the gateway as well as how that data is parsed apart and used to both integrate calls and to turn on and off message waiting indicators. This trace key only appears in the web interface if the gateway is currently configured to use the serial port for integration.
Relevant Dialogic Gateway Models	Dialogic 1000 Media Gateways Series models DMG1008LSW and DMG1008DNIW (NEC integrations only) Dialogic 2000 Media Gateways Series models DMG2030DTI, DMG2060DTI, and DMG2120DTI
Relevant Levels	prot - This debug provides the raw serial data between the gateway and the serial data source. The data is presented in raw ASCII form and is written to the trace log as hexadecimal numbers. code - This debug provides insight into how the raw data is used once the gateway receives it. These traces will show how the packets are seen and taken apart once they are judged as valid by the firmware.
When Used	This debug is required to troubleshoot any call integration related issues when serial integration is being used. In the sample below note the following things: In example 1, the entries on lines 1 through 36 show the raw serial protocol (NEC MCI in this example) of an inbound call. In example 1, the entries on lines 37 and 38 show that the serial packet was judged as complete and valid by the firmware and will be used for integration. In example 1, the entries on lines 39 through 41 show the results of the firmware after it has gone through the packet gathering its integration data. In example 2, the entries show a packet of SMDI data as it has been passed from the serial master gateway over IP to the serial slave gateway for use in call integration. Note how the debug shows the data in completely parsed format and how it has identified this as a direct call for LTN #4 and also shows the 7-digit calling party,
Sample	Example 1 (NEC MCI protocol) Example 2 (SMDI protocol)

Firmware Area	SIIP
Description	Debug showing the serial over IP link status of the serial master and serial slave gateways. This trace key only appears in the web interface if the gateway is currently configured to use the serial port for integration.
Relevant Dialogic Gateway Models	Dialogic 1000 Media Gateways Series models DMG1008LSW and DMG1008DNIW (NEC integrations only) Dialogic 2000 Media Gateways Series models DMG2030DTI, DMG2060DTI, and DMG2120DTI
Relevant Levels	code - This debug provides insight into how the raw data is used once the gateway receives it. These traces will show how the packets are seen and taken apart once they are judged as valid by the firmware.
When Used	This debug is required to troubleshoot issues where the link between master and slave become suspect. In the sample below note the following: The entries on lines 1 through 4 show the heartbeat between a serial slave and a serial master. Thsi heartbeat is used to maintain the data link between the two gateways.
Sample	Example 1

Firmware Area	DspCpi
Description	Debug from the DSP Media and call progress API that shows all tone detection and packetizing activity on the gateway.
Relevant Dialogic Gateway Models	Dialogic 1000 Media Gateways Series models DMG1008LSW, DMG1008DNIW, DNG1008RLMDNIW, and DMG1008MTLDNIW
Relevant Levels	stat - This debug provides a purely statistical view of the DSP interface. Data shown here includes audio levels as detected on the circuit interface as well as RTP status reports that get emitted on a timed schedule showing RTP statistics of the established media connection. code - This debug provides insight into how the DSP sees the audio energy on the line, and can show relevant details such as DTMF detections, sampling of the audio when looking for call progress tones, and even the conversion of RFC2833 RTP events into TDM-side audio. event - This debug shows when the DSP interface starts and stops specific internal processes as part of tone detection and when specific tone and cadence templates have been matched.
When Used	This debug (at the code and event levels) is required when troubleshooting any audio related issues. It will help to locate when a tone template is not aligned with the actual audio on the channel or if a DTMF digit is being missed on the TDM side of the gateway or an RFC2833 event is suspected of being missed on the IP side of the gateway. In the sample below note the following things: In example 1, the entries on lines 1 through 12 show the call progress detection engine starting up at the beginning of an IP to TDM call. The loading and starting of each of the configured tone temples will be seen here. In example 1, the entries on line 17 through 23 show the detection of a tone comming onto the TDM channel. Note that the frequency components as well as the associated RTP packet time data is shown here. In example 1, the entry on line 50 tells us that the tone being seen has been matched by its tone template and cadence time as the configured dial tone template so this tone event has been raised. The upper layers of firmware will now see this as a detected tone. In example 2, the entries on lines 1 through 7 show a DTMF 1 being detected on the TDM side of the gateway and the corresponding RFC2833 RTP event being sent out to the IP side of the call. In example 2, the entries on lines 8 through 13 show the same DTMF tone being detected as stopping so the RTP event is completed. In example 3 we see the entries emitted when the gateway detects an RFC2833 sequence representing the digit string '12' being sent to the IP side of the gateway. These messages will result in a DTMF tone being generated in the voice path of the TDM side of the gateway. In example 4 we see a typical RTP statistics message. When enabled these are emitted roughly every 30 seconds in an effort to provide you with updated statistics regarding the health and use of the RTP stream.
Sample	Example 1 (Detection of dialtone after goingoff hook) Example 2 (DTMF detection on TDM side of the gateway) Example 3 (RFC2833 event detected on IP interface) Example 4 (RTP status messages)

Firmware Area	TelDrv
Description	Debug from the ISDN interface that shows all raw D-Channel messages.
Relevant Dialogic Gateway Models	Dialogic 2000 Media Gateways Series models DMG2030DTI, DMG2060DTI, and DMG2120DTI
Relevant Levels	prot - This debug provides a just the raw D-Channel data as it is sent and received. code - This debug provides insight into how the D-Channel messages are parsed and used by the application on the gateway.
When Used	This debug (at the code and prot levels) is required when troubleshooting any TDM side call control or call id data issues on ISDN system. In the sample below note the following things: In example 1, the entries on line 1 shows an inbound call arriving at the gateway. Note that the arrow (<-) signifies the direction of the message. This is a setup message is from the circuit network side into the gateway. In example 1, the entries on lines 2 through 12 show the firmware parsing through the setup message and breaking it up into logical sections for subsequent use in gathering call data from the setup message and establishing the TDM call with the network. To make this part of the trace even clearer it is a good idea to also enable tel traces as that trace shows this process even further and highlights the process if breaking up IE sections to gather call id data.
Sample	Example 1 (Arrival of a TDM to IP call over ISDN)

Firmware Area	CallLog
Description	Debug that shows each entry that gets made into the gateways call log.
Relevant Dialogic Gateway Models	Dialogic 1000 Media Gateways Series models DMG1008LSW, DMG1008RLMDNIW, and DMG1008MTLDNIW Dialogic 2000 Media Gateways Series models DMG2030DTI, DMG2060DTI, and DMG2120DTI
Relevant Levels	event - This will show each entry made into the log and is useful to help tie a specific call to a specific area of debug data.
When Used	In the sample below note the following: In the example the entry on line 1 shows the calls number in the series. In the example the entry on line 2 shows the direction of the call, in this case this was a call that originated from the IP side of the gateway. In the example the entries on lines 3 through 4 show the start and stop times of the call. Thsi helps you determine how long the call existed for inside the gateway. In the example the entry on line 5 provides data for the inbound call request. In this case this was an MWI Clear request from an IP application on IP address 172.16.228.7 and the circuit destination was extension 3476. In the example the entry on line 6 showed that the last port of an 8 port gateway (these ports are 0 based) was selected by the firmware to process the MWI. In the example the entry on line 7 shows that the call ended normally and without failures.
Sample	Example 1 (Arrival of a TDM to IP call over ISDN)

Dialogic Media Gateways (DMG)