| Summary: | ASTERISK-23325: Asterisk is Running but not processing anything | ||||
| Reporter: | Antonis Psaras (apsaras) | Labels: | |||
| Date Opened: | 2014-02-18 11:07:56.000-0600 | Date Closed: | 2014-02-27 17:37:40.000-0600 | ||
| Priority: | Major | Regression? | |||
| Status: | Closed/Complete | Components: | |||
| Versions: | 1.8.25.0 | Frequency of Occurrence | |||
| Related Issues: |
| ||||
| Environment: | Dell 420, No Card, Xeon, 4Gb RAM, SAS RAID 10, Centos 10, 64bit, Asterisk 1.8.25 | Attachments: | ( 0) asterisk-confs.zip ( 1) asterisk-threads.tgz ( 2) backtrace-threads.txt ( 3) backtrace-threads-20140220.txt ( 4) core-show-locks.txt ( 5) full.tgz | ||
| Description: | During normal operation, asterisk stops responding to any SIP, AGI or AMI message. I am able to log on to CLI and execute commands but all other functionality is down. I am able to connect to AMI but I do not get any event and I am not able to execute any command.
Attached is the show threads and the backtrace commanda (gdb -ex "thread apply all bt" --batch /usr/sbin/asterisk `pidof asterisk` > /tmp/backtrace-threads.txt) | ||||
| Comments: | By: Antonis Psaras (apsaras) 2014-02-18 11:09:44.016-0600 show threads gdb output By: Rusty Newton (rnewton) 2014-02-18 12:45:55.007-0600 Can you also provide the output of "core show locks" and if possible, a VERBOSE and DEBUG log (turned up to level 5 if possible) showing what is going on right before the issue occurs and during. Additionally, if possible, recompile with DONT_OPTIMIZE and try to reproduce, as some useful values may be optimized out. By: Antonis Psaras (apsaras) 2014-02-18 13:27:31.527-0600 I tried to run core show locks but there is no such command. I have a full log with debug enabled. I am attaching the log 3 minutes before the crash util the asterisk manual restart. I will also recompile with DONT_OPTIMIZE and send the debug as soon as I reporduce it. By: Antonis Psaras (apsaras) 2014-02-19 11:34:06.137-0600 I just uploaded a new backtrace with DONT_OPTIMIZE. By: Rusty Newton (rnewton) 2014-02-19 14:56:39.779-0600 Thanks for the new backtrace, I'll see if someone can take a look. Regarding "core show locks", you'll need the DEBUG_THREADS compiler flag enabled. [Relevant wiki page|https://wiki.asterisk.org/wiki/display/AST/CLI+commands+useful+for+debugging#CLIcommandsusefulfordebugging-coreshowlocks]. Sorry that we forgot to mention that! We thought you already had it set for some reason or another. Essentially, you'll want DONT_OPTIMIZE, BETTER_BACKTRACES enabled for the best output for your backtrace, and then DEBUG_THREADS since this is likely a deadlock and the "core show locks" output would be really helpful. Just be aware you'll likely have a performance hit on your system due to DEBUG_THREADS and DONT_OPTIMIZE. Though, that may not matter to you since you are reproducing a crash. By: Rusty Newton (rnewton) 2014-02-19 15:06:36.133-0600 Antonis, Richard Mudgett looked at the backtrace and unfortunately, it is only so helpful and not definitive, we will still need the "core show locks" output to investigate further. Thanks! By: Antonis Psaras (apsaras) 2014-02-20 06:09:57.750-0600 New show locks By: Antonis Psaras (apsaras) 2014-02-20 06:10:40.976-0600 New Back Trace By: Antonis Psaras (apsaras) 2014-02-24 14:21:14.582-0600 Hello. Is there any news about this issue? By: Rusty Newton (rnewton) 2014-02-26 15:33:21.953-0600 Hi! You didn't hit "Send Back" or "Enter Feedback" so the issue was not moved back into a state where I would readily see it. Richard Mudgett has looked at the new output and feels the issue could relate to ASTERISK-21228 or be a duplicate. Are you using pbx_realtime, i.e. dialplan realtime switches? By: Antonis Psaras (apsaras) 2014-02-27 00:17:33.624-0600 Yes we use realtime with odbc and postgresql as a back end By: Antonis Psaras (apsaras) 2014-02-27 00:20:54.496-0600 But we do not have extentions .conf on db as the other issue mentions. By: Rusty Newton (rnewton) 2014-02-27 16:59:03.311-0600 Would it be possible for you to attach your extconfig.conf, extensions.conf, sip.conf, manager.conf and res_odbc.conf , of course scrubbed of any personal info or passwords? By: Antonis Psaras (apsaras) 2014-02-27 17:30:12.097-0600 Attached the configuration files requested By: Matt Jordan (mjordan) 2014-02-27 17:35:53.213-0600 Uhm... your entire dialplan is in the database: {noformat} [DialIn] Switch => Realtime/DialIn [default] Switch => Realtime/DialIn [DefaultPlan] Switch => Realtime/DefaultPlan [AllCalls] include => hints Switch => Realtime/AllCalls [LocalMobileCalls] include => hints Switch => Realtime/LocalMobileCalls [LocalCalls] include => hints Switch => Realtime/LocalCalls [Restrict] Switch => Realtime/Restrict [CtiIsdnDialOut] Switch => Realtime/CtiIsdnDialOut [AllCallsRec] include => hints Switch => Realtime/AllCallsRec [AllCallsRec_393100] include => hints Switch => Realtime/AllCallsRec_393100 [AllCallsRec_393110] include => hints Switch => Realtime/AllCallsRec_393110 [AllCallsRec_393120] include => hints Switch => Realtime/AllCallsRec_393120 [Restrict_1] Switch => Realtime/Restrict_1 [AllCallsRec_1] include => hints Switch => Realtime/AllCallsRec_1 [SipTrunks] Switch => Realtime/SipTrunks [AllCallsNI] Switch => Realtime/AllCallsNI [AllCallsNI_898] Switch => Realtime/AllCallsNI_898 {noformat} This is the same issue as ASTERISK-21228. By: Matt Jordan (mjordan) 2014-02-27 17:37:40.137-0600 As this is a duplicate of ASTERISK-21228, I'm going to go ahead and close it out as such. Keep in mind that {{pbx_realtime}} is an extended support module. Development support for it would typically come from the Asterisk Developer community, and response times may reflect that. You can work around the issue you are currently facing by using non-realtime extensions. You can still use {{func_odbc}} to store pertinent information in a database; as that does not involve usage of realtime {{switch}} statements, it does not fall victim to this problem. | ||||