Summary: | ASTERISK-25366: Segmentation fault - in ast_manager_build_channel_state_string_prefix at manager_channels.c:417 | ||||
Reporter: | Michel R. Vaillancourt (jkl5group) | Labels: | |||
Date Opened: | 2015-09-02 10:44:11 | Date Closed: | |||
Priority: | Critical | Regression? | |||
Status: | Open/New | Components: | Core/ManagerInterface | ||
Versions: | 13.5.0 | Frequency of Occurrence | Frequent | ||
Related Issues: |
| ||||
Environment: | Digital Ocean Droplet / CentOS release 6.7 | Attachments: | ( 0) backtrace.01sept2015-01.txt ( 1) backtrace.01sept2015-02.txt ( 2) backtrace.01sept2015-03.txt ( 3) backtrace.01sept2015-04.txt ( 4) backtrace.01sept2015-05.txt ( 5) backtrace.01sept2015-06.txt ( 6) backtrace.01sept2015-07.txt ( 7) backtrace.05sep2015-01.txt ( 8) backtrace.05sep2015-02.txt ( 9) backtrace.05sep2015-03.txt (10) backtrace.28aug2015.txt (11) backtrace.29aug2015.txt (12) backtrace.30aug2015.txt (13) log_extract.txt.gz | ||
Description: | We are getting almost daily SegFaults like this:
{noformat} Core was generated by `/usr/sbin/asterisk -f -vvvg -c'. Program terminated with signal 11, Segmentation fault. #0 0x000000000055e5d7 in ast_manager_build_channel_state_string_prefix (snapshot=0x0, prefix=0x6c769b "") at manager_channels.c:417 {noformat} Backtraces to follow | ||||
Comments: | By: Asterisk Team (asteriskteam) 2015-09-02 10:44:12.719-0500 Thanks for creating a report! The issue has entered the triage process. That means the issue will wait in this status until a Bug Marshal has an opportunity to review the issue. Once the issue has been reviewed you will receive comments regarding the next steps towards resolution. A good first step is for you to review the [Asterisk Issue Guidelines|https://wiki.asterisk.org/wiki/display/AST/Asterisk+Issue+Guidelines] if you haven't already. The guidelines detail what is expected from an Asterisk issue report. Then, if you are submitting a patch, please review the [Patch Contribution Process|https://wiki.asterisk.org/wiki/display/AST/Patch+Contribution+Process]. By: Michel R. Vaillancourt (jkl5group) 2015-09-02 10:47:58.036-0500 Series of back traces - 30 Aug is "optimized", others are not. By: Rusty Newton (rnewton) 2015-09-03 18:36:25.711-0500 Can you attach a debug log for a few traces? Preferably a debug log showing a few minutes before and right up to the moment of crash. If possible, having VERBOSE and DEBUG log channel types both turned up to 5 would be great. If you can't get it with DEBUG due to load or some other issue at least turn up VERBOSE and have ERROR,NOTICE,WARNING. https://wiki.asterisk.org/wiki/display/AST/Collecting+Debug+Information By: Michel R. Vaillancourt (jkl5group) 2015-09-04 08:22:32.194-0500 Log extracts from "full" log files, as requested By: Michel R. Vaillancourt (jkl5group) 2015-09-04 08:23:16.612-0500 @Rusty - let me know if this is what you need or not ... By: Rusty Newton (rnewton) 2015-09-05 10:57:51.552-0500 [~jkl5group] In the log you posted - was Asterisk dumping each time it uncleanly ended? Also, do you have any idea why the "built by" user changes on some startups? {noformat} Line 1: full-20150830-0-[Aug 27 04:31:06] Asterisk 13.5.0 built by jkl5tech @ switch2.ansspc.com on a x86_64 running Linux on 2015-08-19 22:29:01 UTC Line 212197: full-10393399405-[Sep 3 07:59:21] Asterisk 13.5.0 built by root @ switch2.ansspc.com on a x86_64 running Linux on 2015-09-02 15:37:48 UTC Line 222031: full-10394323767-[Sep 3 07:59:44] Asterisk 13.5.0 built by root @ switch2.ansspc.com on a x86_64 running Linux on 2015-09-02 15:37:48 UTC Line 232457: full-10395313851-[Sep 3 08:13:56] Asterisk 13.5.0 built by jkl5tech @ switch2.ansspc.com on a x86_64 running Linux on 2015-09-03 12:10:02 UTC {noformat} By: Michel R. Vaillancourt (jkl5group) 2015-09-07 21:52:56.432-0500 @Rusty Newton - I didn't specifically correlate... I noticed the heap of cores and then went back through the logs with an extraction tool. However, the customer is noticing calls being dropped en-masse at least once a day, and more often than not, that corresponds with a core-dump / restart by Asterisk. The changing build names are because I'm inconsistent with if I "sudo my_build_script" or "sudo su -" and then run the build commands. It's been "me" handling this for my client, either way. By: Michel R. Vaillancourt (jkl5group) 2015-09-08 08:02:37.906-0500 New server, different hosting provider went into production on Friday ... three seg faults in the same day, same apparent issue: Core was generated by `/usr/sbin/asterisk -f'. Program terminated with signal 11, Segmentation fault. #0 0x0000000000556a43 in ast_manager_build_channel_state_string_prefix (snapshot=0x0, prefix=0x634e5b "") at manager_channels.c:417 417 if (snapshot->tech_properties & AST_CHAN_TP_INTERNAL) By: Rusty Newton (rnewton) 2015-09-08 16:06:13.494-0500 Thanks for the additional info. Did you get these crashes in a previous 13.X version? What were you using before 13.5.0 ? By: Michel R. Vaillancourt (jkl5group) 2015-09-08 17:01:07.295-0500 I'm not sure. The customer was having a raft of other issues that I've been working through, vis-a-vis stability. Now that we've got most of the "obvious stuff" cleaned up, I started noticing the SegFault issue -- partially because we now start Asterisk to dump core on SegFault, which it wasn't doing before. So, it's possible this was happening before, but we weren't seeing it. By: Vasilii Rogin (roginvs) 2017-11-08 10:16:17.539-0600 Had same crash on 13.18.0 version: Program terminated with signal SIGSEGV, Segmentation fault. #0 ast_manager_build_channel_state_string_prefix (snapshot=0x0, prefix=prefix@entry=0x5ffc6f "") at manager_channels.c:474 474 if (snapshot->tech_properties & AST_CHAN_TP_INTERNAL) { [Current thread is 1 (Thread 0x7fcd6b3f7700 (LWP 6012))] |