Summary:	ASTERISK-25239: Asterisk 13.4.0 crashes
Reporter:	Christian Beyerlein (cbeyerlein)	Labels:
Date Opened:	2015-07-09 02:43:12	Date Closed:	2015-07-11 17:38:12
Priority:	Major	Regression?
Status:	Closed/Complete	Components:
Versions:	13.4.0	Frequency of Occurrence	Frequent
Related Issues:
Environment:		Attachments:	( 0) backtrace.txt
Description:	We are observing regular crashes on Asterisk 13.4.0 serving PJSIP calls and forwarding them to libss7. Call load is approximately 5 calls per second. System was running fine for two weeks with this load, and now started to crash regularly. Re-compiled with DONT_OPTIMIZE and got a coredump. Attaching backtrace.
Comments:	By: Asterisk Team (asteriskteam) 2015-07-09 02:43:14.852-0500 Thanks for creating a report! The issue has entered the triage process. That means the issue will wait in this status until a Bug Marshal has an opportunity to review the issue. Once the issue has been reviewed you will receive comments regarding the next steps towards resolution. A good first step is for you to review the [Asterisk Issue Guidelines|https://wiki.asterisk.org/wiki/display/AST/Asterisk+Issue+Guidelines] if you haven't already. The guidelines detail what is expected from an Asterisk issue report. Then, if you are submitting a patch, please review the [Patch Contribution Process|https://wiki.asterisk.org/wiki/display/AST/Patch+Contribution+Process]. By: Christian Beyerlein (cbeyerlein) 2015-07-09 02:47:09.770-0500 Further information about Environment: Dell R610 Linux fravocmg30 2.6.32-504.16.2.el6.x86_64 #1 SMP Wed Apr 22 06:48:29 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux pjproject-2.3-0.digium3.1_centos6.x86_64 Asterisk 13.4.0 libss7 2.0.0 dahdi 2.10.1 By: Rusty Newton (rnewton) 2015-07-09 09:03:38.861-0500 [~jcolp] looked over the crash and determined that you are using the older PJSIP version that is configured to crash on debug assertions. The solution here is to use a newer version of PJSIP, but we don't have a newer package yet. Until we get a newer RPM package, you could install from source the latest PJSIP and that should prevent the crash from happening. http://www.pjsip.org/download.htm Thanks! By: Christian Beyerlein (cbeyerlein) 2015-07-09 09:54:27.637-0500 Can you give any ETA when new PJSIP release will be available as RPM? Can older release be fixed to not "crash on debug assertions"? If there is a patch to disable this "feature", we could try to rebuild the existing RPM packages, but packaging the new version would probably be quite some effort for us. By: Asterisk Team (asteriskteam) 2015-07-09 09:54:28.454-0500 This issue has been reopened as a result of your commenting on it as the reporter. It will be triaged once again as applicable. By: Christian Beyerlein (cbeyerlein) 2015-07-10 09:15:35.807-0500 I have diffed your pjproject version 2.3-digium3 against upstream 2.3 and found no differences besides missing files. Furthermore, I have diffed upstream 2.3 against 2.4 and found no changes in debug assertion handling. I don't see how using 2.4 should change anything regarding the case we are hitting. Can you advise? By: Joshua C. Colp (jcolp) 2015-07-10 09:22:36.322-0500 There is no patch to pjproject itself, whether certain assertions are treated as fatal or not is part of the flags given to it when compiling. The current configure flags used by the install_prereq script are as follows: {noformat} ./configure CFLAGS="-DNDEBUG -DPJ_HAS_IPV6=1" --enable-shared --with-external-speex --with-external-gsm --with-external-srtp --disable-sound --disable-resample {noformat} This builds without debug support (which makes certain assertions not assert) and enables IPv6 support. By: Mark Michelson (mmichelson) 2015-07-10 09:24:14.326-0500 I think there was a miscommunication here. I think when Rusty said "old version with debug assertions" he means that the package you are using was created from a version of PJProject with debug assertions present. Since we don't have newer packages, the only available option for now is to download PJProject from source and compile without debug assertions. As far as an ETA on a new package, I'm not certain about that. By: Christian Beyerlein (cbeyerlein) 2015-07-10 09:47:01.901-0500 I think I got it, as the pjproject-2.3-0.digium3.1_centos6.src.rpm in the pjproject.spec there is only stated {noformat} %{configure} --enable-shared --with-external-speex --with-external-gsm --with-external-srtp --disable-sound --disable-resample {noformat} I will try to rebuild the project RPM with changed configure settings. By: Rusty Newton (rnewton) 2015-07-11 17:37:51.398-0500 That sounds like a plan. I'm going to close this out since we don't have a bug to address here. If you have any further questions about building pjproject or the RPM packages you may ask on the asterisk-users or asterisk-dev mailing lists.