[Home]

Summary:ASTERISK-28337: Regular segmentation faults
Reporter:Jeroen Hermans (jhermans)Labels:pjsip
Date Opened:2019-03-15 06:44:20Date Closed:2020-01-14 11:13:55.000-0600
Priority:MajorRegression?
Status:Closed/CompleteComponents:Core/General
Versions:13.22.0 Frequency of
Occurrence		Frequent
Related
Issues:
Environment:OS: Red Hat Enterprise Linux 7.5-0.11 Asterisk 13.22.0 PJSIP: 2.7.2 RAM: 6Gb VMware 4 coresAttachments:( 0) core.plusovpbx3.danego.net-2019-03-15T09-15-29+0100-brief.txt
( 1) core.plusovpbx3.danego.net-2019-03-15T09-15-29+0100-full.txt
( 2) core.plusovpbx3.danego.net-2019-03-15T09-15-29+0100-locks.txt
( 3) core.plusovpbx3.danego.net-2019-03-15T09-15-29+0100-thread1.txt
Description:The Asterisk process keeps producing a segmentation fault a few times a day. I have attached the coredump information of one of the segmentation faults.
Comments:By: Asterisk Team (asteriskteam) 2019-03-15 06:44:21.596-0500

Thanks for creating a report! The issue has entered the triage process. That means the issue will wait in this status until a Bug Marshal has an opportunity to review the issue. Once the issue has been reviewed you will receive comments regarding the next steps towards resolution.

A good first step is for you to review the [Asterisk Issue Guidelines|https://wiki.asterisk.org/wiki/display/AST/Asterisk+Issue+Guidelines] if you haven't already. The guidelines detail what is expected from an Asterisk issue report.

Then, if you are submitting a patch, please review the [Patch Contribution Process|https://wiki.asterisk.org/wiki/display/AST/Patch+Contribution+Process].

Please note that once your issue enters an open state it has been accepted. As Asterisk is an open source project there is no guarantee or timeframe on when your issue will be looked into. If you need expedient resolution you will need to find and pay a suitable developer. Asking for an update on your issue will not yield any progress on it and will not result in a response. All updates are posted to the issue when they occur.
By: Jeroen Hermans (jhermans) 2019-03-15 06:47:53.064-0500

Created according to https://wiki.asterisk.org/wiki/display/AST/Getting+a+Backtrace
By: Sean Bright (seanbright) 2019-03-15 09:34:50.809-0500

Re-attached backtraces as text files
By: Benjamin Keith Ford (bford) 2019-03-15 13:17:26.299-0500

Unfortunately this backtrace isn't very helpful, but from what I can gather, it seems to point at the codec_silk module. Since you mentioned you were following the wiki documentation on how to get a backtrace, I'm assuming you compiled with BETTER_BACKTRACES and DONT_OPTIMIZE?

Also, can you provide more details on what is going on when the crash occurs? Is during high traffic? Just one phone call? Maybe just one specific phone / endpoint? Are there any calls going on at all? Any more information you can provide would help!
By: Jeroen Hermans (jhermans) 2019-03-15 16:05:01.377-0500

Hello Benjamin, Asterisk was compiled with:
Build Options:               DONT_OPTIMIZE, COMPILE_DOUBLE, BETTER_BACKTRACES, OPTIONAL_API

The crashes occur at moment with moderate traffic (~30 calls). There is no error in the logs. The problems started when because of queue-problems we switched to pjsip. So there are two main items that are changed: queuing from ringall to leastcalled and from chan_sip to chan_pjsip. Now i donnot believe either of them is the real cause of the problems.
Now i am looking at the logfiles and i see a lot of:
[2019-03-15 21:57:00] ERROR[22319][C-00000012]: pbx_functions.c:610 ast_func_read: Function SIP_HEADER not registered
[2019-03-15 21:57:00] ERROR[22319][C-00000012]: pbx_functions.c:653 ast_func_read2: Function SIP_HEADER not registered
[2019-03-15 21:57:00] ERROR[22319][C-00000012]: res_pjsip_header_funcs.c:454 func_read_header: This function requires a PJSIP channel.
[2019-03-15 21:57:00] ERROR[22319][C-00000012]: res_pjsip_header_funcs.c:454 func_read_header: This function requires a PJSIP channel.

Surely this should not be there (dialplan problems probably), but i also donnot believe this can cause a segmentation fault in Asterisk.

The error in the logs that i am a little concerned about is:
[2019-03-15 21:56:59] WARNING[22306][C-00000012]: taskprocessor.c:895 taskprocessor_push: The 'subp:ast_channel_topic_all-000003fd' task processor queue reached 500 scheduled tasks again.
[2019-03-15 21:56:59] WARNING[22302][C-00000012]: taskprocessor.c:895 taskprocessor_push: The 'subm:ast_channel_topic_all-cached-000000ce' task processor queue reached 500 scheduled tasks again.

Not sure what these queues are and why they are filling up.
I really hope you can help me with this as this is actually a reinstalled PBX after the first one had exactly the same problems.
By: Benjamin Keith Ford (bford) 2019-03-18 11:04:27.018-0500

None of these should be causing segfaults, just spitting out error messages and warnings and moving on. "ast_func_read" is getting nothing back when trying to lookup a custom function, hence the "Function ___ not registered" error. "func_read_header" either cannot find the channel, or the channel in question is not a PJSIP channel since it checks for "PJSIP/" at the start of the channel name, hence the "This function requires a PJSIP channel" error message. As for the taskprocessor, this one in particular is dealing with subscriptions to topics, and reaching the number to trigger a high volume message is something that can be expected in high load systems. Hopefully that clears up those messages for you.
As for the segfault itself, can you attach some log files so we can see what's going on with Asterisk? If you know the time when the crash occurred, this can also help. Also, if you know what route your channels are taking in your dialplan, providing that information (extensions.conf and the path the channel takes through it) would help too.
By: Asterisk Team (asteriskteam) 2019-04-02 12:00:00.986-0500

Suspended due to lack of activity. This issue will be automatically re-opened if the reporter posts a comment. If you are not the reporter and would like this re-opened please create a new issue instead. If the new issue is related to this one a link will be created during the triage process. Further information on issue tracker usage can be found in the Asterisk Issue Guidlines [1].

[1] https://wiki.asterisk.org/wiki/display/AST/Asterisk+Issue+Guidelines