[Home]

Summary:ASTERISK-25176: Asterisk segfault on reload res_odbc.so
Reporter:Marty Riedling (mriedling)Labels:
Date Opened:2015-06-19 13:56:58Date Closed:
Priority:MajorRegression?
Status:Open/NewComponents:Resources/res_odbc
Versions:13.1.0 13.4.0 Frequency of
Occurrence		Constant
Related
Issues:
Environment:CentOS 7.0Attachments:
Description:Asterisk-13.1-cert2 (as well as 13.4) segfault on a reload res_odbc.so
With a backtrace of:
{noformat}
(gdb)
#0  0x00007f402ca959cb in mysql_get_character_set_info () from /usr/lib64/mysql/libmysqlclient.so.18
#1  0x00007f402cf8e9e5 in myodbc_do_connect () from /usr/lib64/libmyodbc5.so
#2  0x00007f402cf8edd1 in MySQLConnect () from /usr/lib64/libmyodbc5.so
#3  0x00007f402fb9b3a3 in SQLConnect () from /lib64/libodbc.so.2
#4  0x00007f402fdfb1c4 in odbc_obj_connect (obj=0x7f3fc4005158) at res_odbc.c:1553
#5  0x00007f402fdfa655 in _ast_odbc_request_obj2 (name=0x7f3fc4005580 "asteriskcdrdb", flags=..., file=0x7f402fdfc434 "res_odbc.c",
   function=0x7f402fdfd700 <__PRETTY_FUNCTION__.16193> "odbc_register_class", lineno=1043) at res_odbc.c:1375
#6  0x00007f402fdfab02 in _ast_odbc_request_obj (name=0x7f3fc4005580 "asteriskcdrdb", check=0, file=0x7f402fdfc434 "res_odbc.c",
   function=0x7f402fdfd700 <__PRETTY_FUNCTION__.16193> "odbc_register_class", lineno=1043) at res_odbc.c:1448
#7  0x00007f402fdf9066 in odbc_register_class (class=0x7f3fc4005578, preconnect=-1) at res_odbc.c:1043
#8  0x00007f402fdf8981 in load_odbc_config () at res_odbc.c:927
#9  0x00007f402fdfc0cc in reload () at res_odbc.c:1821
#10 0x000000000052bbf6 in ast_module_reload (name=0x7f3fc4006c8e "res_odbc.so") at loader.c:918
#11 0x00000000004c90aa in handle_reload (e=0x887e98 <cli_cli+3000>, cmd=-4, a=0x7f40440b56f0) at cli.c:294
#12 0x00000000004d0a8a in ast_cli_command_full (uid=-1, gid=-1, fd=16, s=0x7f40440b59c8 "module reload res_odbc.so") at cli.c:2719
#13 0x00007f3f90e5d713 in cli_alias_passthrough (e=0x1b441e8, cmd=-4, a=0x7f40440b6360) at res_clialiases.c:148
#14 0x00000000004d0a8a in ast_cli_command_full (uid=0, gid=0, fd=16, s=0x7f40440b6640 "reload res_odbc.so") at cli.c:2719
#15 0x00000000004d0bc6 in ast_cli_command_multiple_full (uid=0, gid=0, fd=16, size=19, s=0x7f40440b6a90 "reload res_odbc.so") at cli.c:2742
#16 0x0000000000450db2 in netconsole (vconsole=0x8b0680 <consoles>) at asterisk.c:1521
#17 0x00000000005e0e40 in dummy_start (data=0x7f40280008f0) at utils.c:1232
#18 0x00007f4049146df5 in start_thread () from /lib64/libpthread.so.0
#19 0x00007f40484ec1ad in clone () from /lib64/libc.so.6
{noformat}
Comments:By: Rusty Newton (rnewton) 2015-06-19 15:56:33.888-0500

Thank you for the crash report. However, we need more information to investigate the crash. Please provide:

1. A backtrace generated from a core dump using the instructions provided on the Asterisk wiki [1].
2. Specific steps taken that lead to the crash.
3. All configuration information necesary to reproduce the crash.

Thanks!

[1]: https://wiki.asterisk.org/wiki/display/AST/Getting+a+Backtrace


By: Rusty Newton (rnewton) 2015-06-19 15:56:45.809-0500

We require additional debug to continue with triage of your issue. Please follow the instructions on the wiki [1] for how to collect debugging information from Asterisk. For expediency, where possible, attach the debug with a '.txt' file extension so that the debug will be usable for further analysis.

Thanks!

[1] https://wiki.asterisk.org/wiki/display/AST/Collecting+Debug+Information


By: Rusty Newton (rnewton) 2015-06-19 15:57:48.204-0500

In addition to the new trace and debug, please provide versions of odbc and libmysql.
By: Asterisk Team (asteriskteam) 2015-07-04 12:00:20.818-0500

Suspended due to lack of activity. Please request a bug marshal in #asterisk-bugs on the IRC network irc.freenode.net to reopen the issue should you have the additional information requested. Further information on issue tracker usage can be found in the Asterisk Issue Guidlines [1].

[1] https://wiki.asterisk.org/wiki/display/AST/Asterisk+Issue+Guidelines
By: Tzafrir Cohen (tzafrir) 2015-09-01 14:20:01.323-0500

I have the same issue on a Centos 7 system with Asterisk 13.1-cert2. I get a similar backtrace.

The immediate cause of the crash:  mysql_get_character_set_info() assumes that the MYSQL struct is initialized, and specifically has a valid charset.

mysql->charset is initialized in mysql_init(). However, on the second time it is called, mysql_init exits immediately: it begins with:
{code}
  1722   if (mysql_server_init(0, NULL, NULL))
  1723     return 0;
{code}
mysql_server_init, if called the second time, will call my_thread_init() instead of the rest of the initializations. my_thread_init() bails out immediately:
{code}
   309   if (!my_thread_global_init_done)
   310     return 1; /* cannot proceed with unintialized library */
{code}

So it seems that a call to my_thread_global_init() is missing somewhere. Well, that's the low-level part.

Relevant packages:

mariadb-5.5.44-1.el7_1.x86_64
mysql-connector-odbc-5.2.5-6.el7.x86_64

By: Rusty Newton (rnewton) 2015-09-14 14:19:45.102-0500

[~tzafrir] can you describe your usage of res_odbc with Asterisk, what Asterisk is doing with it and also attach a debug log showing the moments right before the crash? That may help the developers track this down a bit further.
By: JoshE (n8ideas) 2015-09-15 18:45:45.838-0500

We've also seen this as an issue, among other issues with CentOS 7.  The crash - not surprisingly - also happens on Asterisk 11.

We reverted all Asterisk boxes to CentOS 6 to work around.
By: Rusty Newton (rnewton) 2015-09-16 08:39:53.868-0500

[~n8ideas] are you using different library and connector versions on CentOS 6 vs CentOS 7? Can you post what versions you were using on both?