[Home]

Summary:ASTERISK-17948: [patch] missing ast_channel_lock() in func_odbc.c function acf_fetch()
Reporter:Archie Cobbs (archie172)Labels:
Date Opened:2011-05-31 18:24:53Date Closed:2012-01-24 11:07:01.000-0600
Priority:MajorRegression?No
Status:Closed/CompleteComponents:Functions/func_odbc
Versions:1.8.4 Frequency of
Occurrence
Related
Issues:
Environment:Attachments:( 0) 19394.patch
Description:We occasionally see Asterisk threads getting hung/deadlocked while attempting an ODBC call to the database.

Visual inspection revealed a possible cause, which is a failure to lock the channel before invoking ast_channel_datastore_find() in acf_fetch(). See attached patch.

Although we are using and obsolete version of Asterisk (1.6.0), this bug exists in all versions of Asterisk up through trunk.

This is just a hunch and completely unverified. However, it should take only 5 seconds to confirm or reject this bug (I'm not an Asterisk locking expert so I'm sure myself).

If this bug is real, I would very much appreciate any insight into whether it would possibly cause the symptoms we are seeing, i.e., channels deadlocking while attempting ODBC queries.



****** ADDITIONAL INFORMATION ******

--- func_odbc.c.orig 2011-03-12 14:08:19.000000000 -0600
+++ func_odbc.c 2011-05-31 18:15:20.000000000 -0500
@@ -790,7 +790,9 @@
struct ast_datastore *store;
struct odbc_datastore *resultset;
struct odbc_datastore_row *row;
+ ast_channel_lock(chan);
store = ast_channel_datastore_find(chan, &odbc_info, data);
+ ast_channel_unlock(chan);
if (!store) {
pbx_builtin_setvar_helper(chan, "ODBC_FETCH_STATUS", "FAILURE");
return -1;
Comments:By: Richard Mudgett (rmudgett) 2012-01-23 13:30:16.674-0600

Debugging deadlocks: Please select DEBUG_THREADS and DONT_OPTIMIZE in the Compiler Flags section of menuselect. Recompile and install Asterisk (i.e. make install).  This will then give you the console command "core show locks." When the symptoms of the deadlock present themselves again, please provide output of the deadlock via:

# asterisk -rx "core show locks" | tee /tmp/core-show-locks.txt
# gdb -se "asterisk" <pid of asterisk> | tee /tmp/backtrace.txt
gdb> bt
gdb> bt full
gdb> thread apply all bt

Then attach the core-show-locks.txt and backtrace.txt files to this issue. Thanks!

Accessing the channel datastore list without locking the channel is bad.  It can potentially lead to memory leaks, corrupt heap, and circular list links.   However, without the deadlock debugging information, there is no proof that it is causing your problem.
By: Richard Mudgett (rmudgett) 2012-01-23 14:15:34.708-0600

Posted a patch to fix the locking issue you found on reviewboard:
https://reviewboard.asterisk.org/r/1687/