[Home]

Summary:ASTERISK-25366: Segmentation fault - in ast_manager_build_channel_state_string_prefix at manager_channels.c:417
Reporter:Michel R. Vaillancourt (jkl5group)Labels:
Date Opened:2015-09-02 10:44:11Date Closed:
Priority:CriticalRegression?
Status:Open/NewComponents:Core/ManagerInterface
Versions:13.5.0 Frequency of
Occurrence		Frequent
Related
Issues:
is related toASTERISK-28810 Segmentation fault in ast_manager_build_channel_state_string_prefix
Environment:Digital Ocean Droplet / CentOS release 6.7Attachments:( 0) backtrace.01sept2015-01.txt
( 1) backtrace.01sept2015-02.txt
( 2) backtrace.01sept2015-03.txt
( 3) backtrace.01sept2015-04.txt
( 4) backtrace.01sept2015-05.txt
( 5) backtrace.01sept2015-06.txt
( 6) backtrace.01sept2015-07.txt
( 7) backtrace.05sep2015-01.txt
( 8) backtrace.05sep2015-02.txt
( 9) backtrace.05sep2015-03.txt
(10) backtrace.28aug2015.txt
(11) backtrace.29aug2015.txt
(12) backtrace.30aug2015.txt
(13) log_extract.txt.gz
Description:We are getting almost daily SegFaults like this:

{noformat}
Core was generated by `/usr/sbin/asterisk -f -vvvg -c'.
Program terminated with signal 11, Segmentation fault.
#0  0x000000000055e5d7 in ast_manager_build_channel_state_string_prefix (snapshot=0x0, prefix=0x6c769b "")
   at manager_channels.c:417
{noformat}

Backtraces to follow
Comments:By: Asterisk Team (asteriskteam) 2015-09-02 10:44:12.719-0500

Thanks for creating a report! The issue has entered the triage process. That means the issue will wait in this status until a Bug Marshal has an opportunity to review the issue. Once the issue has been reviewed you will receive comments regarding the next steps towards resolution.

A good first step is for you to review the [Asterisk Issue Guidelines|https://wiki.asterisk.org/wiki/display/AST/Asterisk+Issue+Guidelines] if you haven't already. The guidelines detail what is expected from an Asterisk issue report.

Then, if you are submitting a patch, please review the [Patch Contribution Process|https://wiki.asterisk.org/wiki/display/AST/Patch+Contribution+Process].
By: Michel R. Vaillancourt (jkl5group) 2015-09-02 10:47:58.036-0500

Series of back traces - 30 Aug is "optimized", others are not.
By: Rusty Newton (rnewton) 2015-09-03 18:36:25.711-0500

Can you attach a debug log for a few traces? Preferably a debug log showing a few minutes before and right up to the moment of crash.

If possible, having VERBOSE and DEBUG log channel types both turned up to 5 would be great.  If you can't get it with DEBUG due to load or some other issue at least turn up VERBOSE and have ERROR,NOTICE,WARNING.

https://wiki.asterisk.org/wiki/display/AST/Collecting+Debug+Information
By: Michel R. Vaillancourt (jkl5group) 2015-09-04 08:22:32.194-0500

Log extracts from "full" log files, as requested
By: Michel R. Vaillancourt (jkl5group) 2015-09-04 08:23:16.612-0500

@Rusty - let me know if this is what you need or not ...
By: Rusty Newton (rnewton) 2015-09-05 10:57:51.552-0500

[~jkl5group] In the log you posted - was Asterisk dumping each time it uncleanly ended?

Also, do you have any idea why the "built by" user changes on some startups?

{noformat}
Line 1: full-20150830-0-[Aug 27 04:31:06] Asterisk 13.5.0 built by jkl5tech @ switch2.ansspc.com on a x86_64 running Linux on 2015-08-19 22:29:01 UTC
Line 212197: full-10393399405-[Sep  3 07:59:21] Asterisk 13.5.0 built by root @ switch2.ansspc.com on a x86_64 running Linux on 2015-09-02 15:37:48 UTC
Line 222031: full-10394323767-[Sep  3 07:59:44] Asterisk 13.5.0 built by root @ switch2.ansspc.com on a x86_64 running Linux on 2015-09-02 15:37:48 UTC
Line 232457: full-10395313851-[Sep  3 08:13:56] Asterisk 13.5.0 built by jkl5tech @ switch2.ansspc.com on a x86_64 running Linux on 2015-09-03 12:10:02 UTC
{noformat}

By: Michel R. Vaillancourt (jkl5group) 2015-09-07 21:52:56.432-0500

@Rusty Newton - I didn't specifically correlate...  I noticed the heap of cores and then went back through the logs with an extraction tool.  However, the customer is noticing calls being dropped en-masse at least once a day, and more often than not, that corresponds with a core-dump / restart by Asterisk.

The changing build names are because I'm inconsistent with if I "sudo my_build_script" or "sudo su -" and then run the build commands.  It's been "me" handling this for my client, either way.
By: Michel R. Vaillancourt (jkl5group) 2015-09-08 08:02:37.906-0500

New server, different hosting provider went into production on Friday ... three seg faults in the same day, same apparent issue:

Core was generated by `/usr/sbin/asterisk -f'.
Program terminated with signal 11, Segmentation fault.
#0  0x0000000000556a43 in ast_manager_build_channel_state_string_prefix (snapshot=0x0, prefix=0x634e5b "")
   at manager_channels.c:417
417             if (snapshot->tech_properties & AST_CHAN_TP_INTERNAL)


By: Rusty Newton (rnewton) 2015-09-08 16:06:13.494-0500

Thanks for the additional info. Did you get these crashes in a previous 13.X version? What were you using before 13.5.0 ?
By: Michel R. Vaillancourt (jkl5group) 2015-09-08 17:01:07.295-0500

I'm not sure.  The customer was having a raft of other issues that I've been working through, vis-a-vis stability.  Now that we've got most of the "obvious stuff" cleaned up, I started noticing the SegFault issue -- partially because we now start Asterisk to dump core on SegFault, which it wasn't doing before.  So, it's possible this was happening before, but we weren't seeing it.
By: Vasilii Rogin (roginvs) 2017-11-08 10:16:17.539-0600

Had same crash on 13.18.0 version:

Program terminated with signal SIGSEGV, Segmentation fault.
#0  ast_manager_build_channel_state_string_prefix (snapshot=0x0, prefix=prefix@entry=0x5ffc6f "") at manager_channels.c:474
474             if (snapshot->tech_properties & AST_CHAN_TP_INTERNAL) {
[Current thread is 1 (Thread 0x7fcd6b3f7700 (LWP 6012))]