[Home]

Summary:ASTERISK-24571: Deadlock when asterisk is loading and CLI command "sip reload" is executed
Reporter:Badalian Vyacheslav (slavon)Labels:
Date Opened:2014-12-01 06:07:10.000-0600Date Closed:2014-12-31 04:53:24.000-0600
Priority:MajorRegression?
Status:Closed/CompleteComponents:
Versions:11.14.1 Frequency of
Occurrence
Related
Issues:
is related toASTERISK-24587 Invalid free in valgrind output
Environment:Attachments:( 0) locks.txt
( 1) locks2.txt
( 2) locks3.txt
( 3) vgdb.txt
Description:Found then tested in valgrind.
We use FreePBX. On valgrind asterisk have long load.
If asterisk load extentions file (loading about 5 sec) and do sip reload - asterisk go to deadlock.
If we kiil asterisk - ports stay openned. Only reboot free ports (may be you must add REUSE flag?)
Comments:By: Badalian Vyacheslav (slavon) 2014-12-01 06:08:43.354-0600

in valgrind output
{code}
==36543== Thread 2:
==36543== Jump to the invalid address stated on the next line
==36543==    at 0x39B642B6D8: ???
==36543==    by 0x5F: ???
==36543==    by 0x444896: _child_handler (asterisk.c:1652)
==36543==  Address 0x39b642b6d8 is not stack'd, malloc'd or (recently) free'd
==36543==
==36543==
==36543== Process terminating with default action of signal 11 (SIGSEGV): dumping core
==36543==  Bad permissions for mapped region at address 0x39B642B6D8
==36543==    at 0x39B642B6D8: ???
==36543==    by 0x5F: ???
==36543==    by 0x444896: _child_handler (asterisk.c:1652)

{code}

By: Matt Jordan (mjordan) 2014-12-02 13:29:33.934-0600

I'm not sure what this is showing, but I'm not sure it shows the issue you're describing.

If there was a crash - which appears to be the case - then please get a backtrace.

By: Matt Jordan (mjordan) 2014-12-02 13:29:46.828-0600

Thank you for your bug report. In order to move your issue forward, we require a backtrace[1] from the core file produced after the crash. Also, be sure you have DONT_OPTIMIZE enabled in menuselect within the Compiler Flags section, then:

make install

After enabling, reproduce the crash, and then execute the backtrace[1] instructions. When complete, attach that file to this issue report.

[1] https://wiki.asterisk.org/wiki/display/AST/Getting+a+Backtrace



By: Badalian Vyacheslav (slavon) 2014-12-02 19:38:14.952-0600

{code}
[2014-12-03 02:15:01] VERBOSE[42294] asterisk.c:     -- Remote UNIX connection disconnected
[2014-12-03 02:30:01] VERBOSE[7248] asterisk.c:     -- Remote UNIX connection
[2014-12-03 02:30:01] VERBOSE[43654] chan_sip.c: Previous SIP reload not yet done
[2014-12-03 02:30:01] VERBOSE[43654] asterisk.c:     -- Remote UNIX connection disconnected
[2014-12-03 02:45:01] VERBOSE[7248] asterisk.c:     -- Remote UNIX connection
[2014-12-03 02:45:01] VERBOSE[44771] chan_sip.c: Previous SIP reload not yet done
[2014-12-03 02:45:01] VERBOSE[44771] asterisk.c:     -- Remote UNIX connection disconnected
[2014-12-03 03:00:01] VERBOSE[7248] asterisk.c:     -- Remote UNIX connection
{code}

backtrace i will attach bellow



By: Badalian Vyacheslav (slavon) 2014-12-02 19:40:37.370-0600

backtrace attached

1. bt
2. bt full
3. thread apply all bt
4. thread apply all bt full

By: Badalian Vyacheslav (slavon) 2014-12-02 19:40:51.695-0600

All info given

By: Badalian Vyacheslav (slavon) 2014-12-02 19:50:52.100-0600

Attacked show locks

// deleted...

By: Badalian Vyacheslav (slavon) 2014-12-03 10:05:21.733-0600

Tested on clear 11.14.1 with patch {{ASTERISK-24472-null-4.diff}} from {{Joshua Colp}} from ASTERISK-24472
Core show locks attached

By: Badalian Vyacheslav (slavon) 2014-12-04 06:24:26.444-0600

< 1000 calls. 1 concurent call. Core show locks attached.

By: Rusty Newton (rnewton) 2014-12-04 09:37:39.454-0600

To be clear - you ran Asterisk in GDB, while Asterisk was loading you executed a "sip reload" and then Asterisk freezes, which is when you collected the attached backtrace?

Other than the potential deadlock, is there also a crash occurring?


By: Badalian Vyacheslav (slavon) 2014-12-04 10:40:57.008-0600

No...

All variant {{runned in valgrind}}. Not in GDB. GDB dumped from GDB -> taget vgdb -> bt full

{{locks.txt}} - patched with my patch. 3 concurent calls. after sip reload on load.
{{locks2.txt}} - patched with {{your}} ws patch. 3 concurent calls. no sip reload (turned off)...
{{locks3.txt}} - patched with {{your}} ws patch. 1 concurent call. no sip reload (turned off)...

Looks like our test robot do deadlock...
Is simple js script that
1. Call to 766
2. Listen MOH (15 sec), Asterisk do Hangup (astersik side drop call)
3. Sleep 1 sec
4. Go to 1

By: Rusty Newton (rnewton) 2014-12-17 08:53:13.234-0600

Please recompile and try gathering debug again with the fix in revision 429539 (in the 11 branch).

https://code.asterisk.org/code/changelog/asterisk?cs=429539




By: Badalian Vyacheslav (slavon) 2014-12-31 04:53:24.360-0600

11.15.0 work without deadlocks