[Home]

Summary:ASTERISK-26017: dial: Crash when executing pre-dial handler
Reporter:Mateusz (mmazur)Labels:crash
Date Opened:2016-05-12 03:09:44Date Closed:
Priority:MajorRegression?
Status:Open/NewComponents:
Versions:13.9.1 Frequency of
Occurrence		Occasional
Related
Issues:
Environment:Attachments:( 0) 2016-06-29backtrace.txt
( 1) backtrace.txt
( 2) backtrace-27-may.txt
( 3) newest_backtrace.txt
Description:Hi,
in our company we are using an pre-dial macro and in the very last step in this macro ( Return() ) we occasionally get an segmentation fault.

We have checked three version of Asterisk.

Out handler:
{code}
exten => outbound,1,NoOp()
same => n,Set(CHANNEL(userfield)=${CHANNEL(userfield)}|calls_id:${ARG1})
same => n,CELGenUserEvent(DIALED_AT)
same => n,Return()
{code}

I have attached an backtrace.
Comments:By: Asterisk Team (asteriskteam) 2016-05-12 03:09:45.550-0500

Thanks for creating a report! The issue has entered the triage process. That means the issue will wait in this status until a Bug Marshal has an opportunity to review the issue. Once the issue has been reviewed you will receive comments regarding the next steps towards resolution.

A good first step is for you to review the [Asterisk Issue Guidelines|https://wiki.asterisk.org/wiki/display/AST/Asterisk+Issue+Guidelines] if you haven't already. The guidelines detail what is expected from an Asterisk issue report.

Then, if you are submitting a patch, please review the [Patch Contribution Process|https://wiki.asterisk.org/wiki/display/AST/Patch+Contribution+Process].
By: Mateusz (mmazur) 2016-05-12 03:12:50.459-0500

Attached attachment with real dst number replaced with "dst_number" string.
By: Rusty Newton (rnewton) 2016-05-12 09:09:47.470-0500

Thank you for the crash report. However, we need more information to investigate the crash. Your backtrace has many values optimized out. Please follow the linked instructions to get an appropriate backtrace. While doing this, if possible test in the latest version of 13 (13.9.0 at this moment).


Please provide:

1. A backtrace generated from a core dump using the instructions provided on the Asterisk wiki [1].
2. Specific steps taken that lead to the crash.
3. All configuration information necesary to reproduce the crash.

Thanks!

[1]: https://wiki.asterisk.org/wiki/display/AST/Getting+a+Backtrace


By: Asterisk Team (asteriskteam) 2016-05-26 12:00:01.641-0500

Suspended due to lack of activity. This issue will be automatically re-opened if the reporter posts a comment. If you are not the reporter and would like this re-opened please create a new issue instead. If the new issue is related to this one a link will be created during the triage process. Further information on issue tracker usage can be found in the Asterisk Issue Guidlines [1].

[1] https://wiki.asterisk.org/wiki/display/AST/Asterisk+Issue+Guidelines
By: Mateusz (mmazur) 2016-05-31 06:55:36.485-0500

Hi,
we have compiled 13.1 cert version with DONT_OPTIMIZE. I have attached backtrace.

We have CentOS 6.5.

Average load is 11000 calls per day. Something about half part of this statistic is outbound calls where we using this macro.
This outbound calls are initialized by callfiles.  

We have idea that our problem could be connected or similar to ASTERISK-25603, but we are not sure about it.

I have attached new backtrace.

Please, let me know if you want some more informations.

Best regards,
Mateusz.

By: Asterisk Team (asteriskteam) 2016-05-31 06:55:36.758-0500

This issue has been reopened as a result of your commenting on it as the reporter. It will be triaged once again as applicable.
By: Rusty Newton (rnewton) 2016-06-17 15:47:17.143-0500

{quote}
While doing this, if possible test in the latest version of 13 (13.9.0 at this moment).
{quote}

Are you able to test in the latest version of vanilla Asterisk? 13.9.1, or the latest Git branch of 13?

I see you say that you are using Certified Asterisk. Do you have a support agreement with Digium; are you already working Digium support?
By: Mateusz (mmazur) 2016-06-30 02:33:56.120-0500

We have compiled Asterisk 13.9.1 version with DONT_OPTIMIZE. I have attached backtrace named 2016-09-29backtrace.txt

We don't have support from Digium.

Please, let me know, if I can give you some more information.
By: Richard Mudgett (rmudgett) 2016-06-30 11:39:02.716-0500

Your backtrace appears to contain a memory corruption. We need one or both of the following items to continue investigation of the issue:
1. Valgrind output. See https://wiki.asterisk.org/wiki/display/AST/Valgrind for instructions on how to use Valgrind with Asterisk.
2. MALLOC_DEBUG output. See https://wiki.asterisk.org/wiki/display/AST/MALLOC_DEBUG+Compiler+Flag for instructions on how to use the MALLOC_DEBUG option.

Note that MALLOC_DEBUG and Valgrind are mutually exclusive options. Valgrind output is preferable, but will be more system resource intensive and may be difficult to get on a production system. In such a case, you may have better luck getting the necessary output from MALLOC_DEBUG.

The crash happened while we were getting a channel snapshot.  Since it happened inside a memory allocation, memory corruption is almost a certainty.
By: Mateusz (mmazur) 2016-07-04 01:59:48.717-0500

We have compiled Asterisk 13.9.1 with MALLOC_DEBUG flag and we have all commands from document that you provide but in log we have only:

1467384199 - New session
1467384576 - New session
1467435606 - New session
1467522007 - New session
1467608416 - New session
1467614323 - New session
1467614398 - New session

and nothing more.

We tried to send segfault signal to Asterisk to check that this will append something more to log but it doesn't change anythink. Permissions for this file are valid.

In debug we can see "[Jul  4 08:39:58] VERBOSE[26016] astmm.c: Asterisk Malloc Debugger Started (see /var/log/asterisk/mmlog))" line.

Is this ok? Or in this log we should have more information?
By: Richard Mudgett (rmudgett) 2016-07-05 10:00:47.757-0500

Yes, you have MALLOC_DEBUG configured correctly.  The mmlog file is where MALLOC_DEBUG writes information about memory violations that it finds.  It will always write that one line every time asterisk is started.