[Home]

Summary:ASTERISK-14433: [patch] crash in bridging api
Reporter:Marcus Hunger (fnordian)Labels:
Date Opened:2009-07-08 07:00:10Date Closed:2010-06-02 08:34:11
Priority:CriticalRegression?No
Status:Closed/CompleteComponents:Core/General
Versions:Frequency of
Occurrence
Related
Issues:
Environment:Attachments:( 0) bridging.patch
Description:There's a racecondition in smart_bridge_operation(). I do not understand this function completely, but it seems like one of it's purposes is to change a bridge's technology. For doing so it stops the bridge->thread before working on it. Stopping is done by setting a flag and sending a signal. What's missing there imho is a pthread_join to assure the thread is really gone.

I noticed crashes when doing transfers from and to the bridges and I guess one of the reasons is this bug:

(gdb) bt full
#0  0x0000000000000024 in ?? ()
No symbol table info available.
#1  0x00002aaab677483d in softmix_bridge_thread (bridge=0x2aaab7819e88)
   at bridge_softmix.c:270
sc = (struct softmix_channel *) 0x2aaab780a7d0
bridge_channel = (struct ast_bridge_channel *) 0x0
buf = {0 <repeats 320 times>}
timeout = -1
timer = (struct ast_timer *) 0x2aaab7833370
timingfd = 40
#2  0x000000000043ae34 in bridge_thread (data=<value optimized out>) at bridging.c:381
bridge = (struct ast_bridge *) 0x2aaab7819e88
res = <value optimized out>
__PRETTY_FUNCTION__ = "bridge_thread"
#3  0x00000000004fb96c in dummy_start (data=<value optimized out>) at utils.c:968
__cancel_buf = {__cancel_jmp_buf = {{__cancel_jmp_buf = {46912711515904, 0, 0,
       0, 1089483408, 1089486848, 1089483072, 5224786}, __mask_was_saved = 0}},
 __pad = {0x40f03200, 0x0, 0x0, 0x0}}
__cancel_arg = (void *) 0x40f03960
not_first_call = <value optimized out>
ret = <value optimized out>
#4  0x00002b56877dff1a in start_thread () from /lib/libpthread.so.0
No symbol table info available.
ASTERISK-1  0x00002b56873245d2 in clone () from /lib/libc.so.6
No symbol table info available.
ASTERISK-2  0x0000000000000000 in ?? ()
No symbol table info available.
(gdb) p *bridge->technology
$14 = {name = 0x2aaab1ada807 "multiplexed_bridge", capabilities = 2,
 preference = AST_BRIDGE_PREFERENCE_HIGH,
 create = 0x2aaab1ad9bc0 <multiplexed_bridge_create>,
 destroy = 0x2aaab1ad9fc0 <multiplexed_bridge_destroy>,
 join = 0x2aaab1ad9a90 <multiplexed_bridge_join>,
 leave = 0x2aaab1ad97d0 <multiplexed_bridge_leave>,
 suspend = 0x2aaab1ad9740 <multiplexed_bridge_suspend>,
 unsuspend = 0x2aaab1ad96b0 <multiplexed_bridge_unsuspend>, compatible = 0,
 write = 0x2aaab1ad9370 <multiplexed_bridge_write>, fd = 0, thread = 0, poke = 0,
 formats = 1073741823, suspended = 0, mod = 0x9c9a80, entry = {next = 0x2aaaaedaf300}}

The crash is in bridge_softmix, working on a bridge which thinks it's bridge_multiplexed.
Comments:By: Leif Madsen (lmadsen) 2009-08-31 08:27:40

Can you please provide a backtrace with DONT_OPTIMIZE enabled in menuselect? It will make the backtrace you're providing much more useful.

Additionally, do you have any particular scenarios that you can provide in order to allow us to attempt to reproduce?

Thanks!
By: Leif Madsen (lmadsen) 2009-09-22 08:50:04

Closed due to no response from reporter. Please reopen if you're able to provide the requested information. Thanks!
By: Marcus Hunger (fnordian) 2009-09-27 05:54:56

sorry for replying so latly. i just checked out the current 1.6.2 branch to confirm this issue. here's a DONT_OPTIMIZE backtrace. the last action i tried before the segfault was removing a channel from a 3-party-bridge.

regards, marcus

(gdb) bt
#0  0x0000000000000050 in ?? ()
#1  0x00002aaab681e83d in softmix_bridge_thread (bridge=0xe188f8) at bridge_softmix.c:270
#2  0x000000000043b084 in bridge_thread (data=<value optimized out>) at bridging.c:370
#3  0x00000000004fd4fc in dummy_start (data=<value optimized out>) at utils.c:968
#4  0x00002b128a21df1a in start_thread () from /lib/libpthread.so.0
ASTERISK-1  0x00002b1289d625d2 in clone () from /lib/libc.so.6
ASTERISK-2  0x0000000000000000 in ?? ()
(gdb) bt full
#0  0x0000000000000050 in ?? ()

No symbol table info available.
#1  0x00002aaab681e83d in softmix_bridge_thread (bridge=0xe188f8) at bridge_softmix.c:270
   sc = (struct softmix_channel *) 0xe3e3f0

   bridge_channel = (struct ast_bridge_channel *) 0x0
   buf = {-1208, -1432, -1032, -936, -664, -872, -1304, -664, -240, -80, -160, -536, 288, 1016, 1000, 128, 600, 1928, 1432, 616, 16, 336, 744, 432, 192, -584, -472, 256, 32, -96, -448, -32, 304, 144, 488, 48, -696, -80, 600, -16, -416, -680, -664, -128, -648, -416, 48,
 528, 808, 528, 744, 560, 112, 1224, 600, -728, -552, -432, -32, -592, -1032, -672, -856, -32, -112, -904, -744, -224, 624, 792, -128, -112, 560, 696, 480, 48, -32, -64, -208, -384, -304, -680, 48, 320, -504, -792, -984, -208, -128, -304, -224, -272, 664, 872, 336, 144, 488,
 1080, 952, 464, 368, 272, 560, 544, 48, 64, 192, 552, 400, 288, -192, -568, -48, 400, -224, -1432, -1240, -272, -368, -480, -192, -48, 320, 448, -336, -368, -160, 240, 808, 272, -368, -384, 96, 32, -320, 48, 96, -112, 112, 208, -336, -776, -416, -856, -1208, -1464, -1032,
 -192, -176, -352, -96, 632, 1528, 1992, 1432, 1112, 1064, 1400, 1224, 568, 0 <repeats 160 times>}
   timeout = -1
   timer = (struct ast_timer *) 0xe48fa0
   timingfd = 15
#2  0x000000000043b084 in bridge_thread (data=<value optimized out>) at bridging.c:370
   bridge = (struct ast_bridge *) 0xe188f8

   res = <value optimized out>
   __PRETTY_FUNCTION__ = "bridge_thread"
#3  0x00000000004fd4fc in dummy_start (data=<value optimized out>) at utils.c:968
   __cancel_buf = {__cancel_jmp_buf = {{__cancel_jmp_buf = {14901408, 0, 0, 0, 1082880656, 14814256, 1082880320, 5231842}, __mask_was_saved = 0}}, __pad = {0x408b7200, 0x0, 0x2b1289eccb08, 0x2b1289eccb10}}
   __cancel_arg = (void *) 0x408b7960

   not_first_call = <value optimized out>
   ret = <value optimized out>
#4  0x00002b128a21df1a in start_thread () from /lib/libpthread.so.0

No symbol table info available.
ASTERISK-1  0x00002b1289d625d2 in clone () from /lib/libc.so.6

No symbol table info available.
ASTERISK-2  0x0000000000000000 in ?? ()
No symbol table info available.
By: Marcus Hunger (fnordian) 2009-10-07 10:49:25

Hi,

doing a pthread_join on the stopped bridge->thread in smart_bridge_operation fixes the crash.
By: Leif Madsen (lmadsen) 2009-10-20 08:47:17

Marking this as Ready for Review as the reporter has supplied a patch which resolves the issue. Thanks!
By: Leif Madsen (lmadsen) 2009-10-20 08:48:25

Hey Russell, I've marked this as needing to be resolved for 1.6.2.1. Please let me know if you feel this should be bumped up to 1.6.2.0. Thanks!
By: Peter Holik (peterh) 2009-12-03 15:27:49.000-0600

I also got a segfault if a channel hangs up from a 3-party-bridge.

This patch solved the problem for me - please include.
By: Digium Subversion (svnbot) 2010-06-02 08:32:22

Repository: asterisk
Revision: 266877

U   trunk/main/bridging.c

------------------------------------------------------------------------
r266877 | pabelanger | 2010-06-02 08:32:22 -0500 (Wed, 02 Jun 2010) | 10 lines

pthread_join to assure the thread is really gone

(closes issue ASTERISK-14433)
Reported by: fnordian
Patches:
     bridging.patch uploaded by fnordian (license 110)
Tested by: lmadsen, fnordian, peterh

Review: https://reviewboard.asterisk.org/r/679/

------------------------------------------------------------------------

http://svn.digium.com/view/asterisk?view=rev&revision=266877
By: Digium Subversion (svnbot) 2010-06-02 08:34:09

Repository: asterisk
Revision: 266878

_U  branches/1.6.2/
U   branches/1.6.2/main/bridging.c

------------------------------------------------------------------------
r266878 | pabelanger | 2010-06-02 08:34:09 -0500 (Wed, 02 Jun 2010) | 17 lines

Merged revisions 266877 via svnmerge from
https://origsvn.digium.com/svn/asterisk/trunk

........
 r266877 | pabelanger | 2010-06-02 09:32:22 -0400 (Wed, 02 Jun 2010) | 10 lines
 
 pthread_join to assure the thread is really gone
 
 (closes issue ASTERISK-14433)
 Reported by: fnordian
 Patches:
       bridging.patch uploaded by fnordian (license 110)
 Tested by: lmadsen, fnordian, peterh
 
 Review: https://reviewboard.asterisk.org/r/679/
........

------------------------------------------------------------------------

http://svn.digium.com/view/asterisk?view=rev&revision=266878