Summary:	ASTERISK-20727: Certified Asterisk - while DPMA module is loaded - Core Reload on CLI triggers crash
Reporter:	Jordan McQuown (jmcquown)	Labels:
Date Opened:	2012-11-26 08:38:49.000-0600	Date Closed:	2013-01-18 14:31:42.000-0600
Priority:	Major	Regression?
Status:	Closed/Complete	Components:
Versions:	1.8.11.1	Frequency of Occurrence	Frequent
Related Issues:
Environment:	PBIAF - Purple - 1.7.5.6, Centos 5 X86_64, Freepbx 2.10, DPM module - 1.8.11_1.3.0	Attachments:	( 0) backtrace.txt ( 1) backtrace.txt ( 2) modules.conf.txt ( 3) resdigiumphone.conf
Description:	When freepbx issues a core reload or a manual core reload is issued asterisk crashes with "segfault at 0000000000000150 rip 0000000000483b50 rsp 0000000045f97d90 error 4" or similiar. I cannot reproduce when res_digium_phone.so is unloaded. I've tried debugging using methods described including valgrind. However I could not get the module to load when I recompiled with DONT_OPTIMIZE, DEBUG_THREADS, MALLOC_DEBUG.
Comments:	By: Jordan McQuown (jmcquown) 2012-11-26 09:17:44.812-0600 This didn't seem to happen with the original release we tried res_digium_phone-1.8.11_1.0.1 and 1.8-cert2 combination. By: Rusty Newton (rnewton) 2012-11-27 18:02:32.322-0600 Jordan, can you provide a backtrace as described here https://wiki.asterisk.org/wiki/display/AST/Getting+a+Backtrace despite being unable to load with the described compiler flags? Can you also provide a full log with VERBOSE and DEBUG set to level 5 each, captured during the core reload?  Please attach modules.conf as well. By: Rusty Newton (rnewton) 2012-11-27 18:10:46.397-0600 Can you provide the output of  the Asterisk CLI command "digium_phones show version" ? By: Jordan McQuown (jmcquown) 2012-11-28 17:23:45.027-0600 Here is the digium_phones show version. pbx1*CLI> digium_phones show version Digium Phone Module for Asterisk Version 1.8.11_1.3.0 By: Jordan McQuown (jmcquown) 2012-11-28 17:23:59.227-0600 [EDIT by Rusty Newton - Moved modules.conf output to modules.conf.txt attachment. Comment fields are for comments.] By: Jordan McQuown (jmcquown) 2012-12-02 15:10:34.020-0600 backtrace By: Rusty Newton (rnewton) 2012-12-06 17:21:32.788-0600 What is the exact Asterisk RPM package (for Asterisk itself) you are using? By: Jordan McQuown (jmcquown) 2012-12-06 17:25:53.164-0600 the distribution is pbx in a flash. However asterisk has been compiled from source. By: Rusty Newton (rnewton) 2012-12-06 17:29:46.736-0600 Thanks. Compiled from http://downloads.asterisk.org/pub/telephony/certified-asterisk/releases/certified-asterisk-1.8.11-cert8.tar.gz ? By: Jordan McQuown (jmcquown) 2012-12-06 17:34:11.532-0600 Yes, that is correct. By: Jordan McQuown (jmcquown) 2012-12-10 21:26:12.009-0600 Here is another backtrace By: Rusty Newton (rnewton) 2012-12-13 17:59:31.740-0600 We haven't been able to reproduce this with a quick try or two. Can you describe the exact PBIAF distro version you are using, and attach your res_digium_phone.conf and sip.conf (sanitized please)? Were you ever able to reload? Did this start happening in a particular version of the module or Asterisk? By: Jordan McQuown (jmcquown) 2012-12-13 18:46:07.509-0600 PIAF version - 1.7.5.6 It seems the issue didn't appear until after the initial digium_phones release and running asterisk 1.8cert2. after that we went to cert 5 and starting seeing periodic crashes on reloads. I've found it doesn't always happen but if the system has res_digium_phones loaded and a few digium phones connected it happens around 50% of the time. I'll get you the requested configs later tonight or early tomorrow AM. By: Malcolm Davenport (mdavenport) 2012-12-14 15:56:47.922-0600 I'm on PIAF 2.0.6.2.4 64-bit, because it was the current version, on a VirtualBox instance. I installed "brown" and opted for FreePBX 2.10.  It gave me Asterisk 1.8.11-cert8. I did the install-digiphones.  That installed a really old res_digium_phone, so I downloaded the current res_digium_phone 1.3.0 for x86_64 and put the .so in place and restarted amportal.   I used the FreePBX Digium phones module that came with install-digiphones.  Someone was asking me separately about using a newer version, so I grabbed the SRPM off our download site, installed it, unpacked it into the digium_phones freepbx module directory and then through the FreePBX module admin updated it.  I'm at version 2.10.0.7.tar.gz. I've got two phones tied to the system now and I've done hundreds of "core reloads" without issue. I'll leave it to someone else to look at backtraces, but towards reproducing, I'd like to know what versions of everything else and what platform (base system?  VirtualBox?  proxmox?) you're on. By: Jordan McQuown (jmcquown) 2012-12-14 16:26:13.517-0600 I'm on pbx purple - 1.7.5.6 - i've been updating from source to the current version since certified asterisk was released. Along with that updating the res_digium_phone version. I've also manually updated to freepbx 2.10. However we do not use the digium supplied provisioning module as it had limitations so we wrote our own provisioning script. We have 12 digium phones registered to the system using the DPMA module. The hardware is a whitebox supermicro server DPM module - 1.8.11_1.3.0 downloaded from asterisk.org I'm not real sure what to say except I cannot reproduce the behavior without res_digium_phones being reloaded. We did not have this issue prior to installing certified-asterisk-1.8cert5 It seems to trigger more frequently when doing a digium_phones reconfigure all and then core reload. By: Jordan McQuown (jmcquown) 2012-12-14 16:26:40.578-0600 res_digium_phone.conf By: Malcolm Davenport (mdavenport) 2012-12-17 08:48:32.713-0600 That file's got two general sections and includes a host of other files, including one which says it provides general section options, which may step on the general section options that you define in that file, and individual phone .cfg files. You're also putting bad parameters into the definition for the [102] line type - registration address and alt registration address, etc. You should probably go back to the wiki docs (https://wiki.asterisk.org/wiki/display/DIGIUM/DPMA+Configuration) anad make sure you're putting valid stuff in the right places and then open a case with our Support people. By: Rusty Newton (rnewton) 2012-12-18 08:15:39.214-0600 I'm going to second Malcolm's suggestion that you contact support http://www.digium.com/en/users/support-create-a-case/ to have them look closer at configuration. In the meantime, we'll see if someone can look at the backtrace.  Make sure to point tech support at this issue number. By: Malcolm Davenport (mdavenport) 2013-01-18 14:31:42.678-0600 I'm suspending this issue until there's new data.  We've got a new DPMA release, 1.4, where we fixed a couple of issues related to problems where the configuration was bad, i.e. you've got two general sections.  Ping me if you upgrade to DPMA 1.4 and still see it and you've worked with Support to get your config cleaned up. By: Jordan McQuown (jmcquown) 2013-01-18 15:46:56.776-0600 Malcolm, Thanks for following up. Unfortunately I did try the new 1.4 module last night and followed the WIKI to clean up our configs to no avail. You can close this issue as I'll soon no longer have access to this box and will be converting these phones over to xml provisioning prior to my departure.