[Home]

Summary:ASTERISK-26261: Segfault with libmysqlclient.so
Reporter:Paul Gorman (harttyit)Labels:
Date Opened:2016-08-02 12:11:18Date Closed:2016-08-02 12:26:45
Priority:MajorRegression?
Status:Closed/CompleteComponents:Resources/res_odbc
Versions:13.8.2 Frequency of
Occurrence		Frequent
Related
Issues:
duplicatesASTERISK-26041 odbc: Asterisk crashed repeatedly - ODBC connector
duplicatesASTERISK-26235 res_odbc: Crash when disconnecting
Environment:Debian 8 (Jessie)Attachments:( 0) gdb.txt
Description:Asterisk crashed several time per day. This did not appear to correspond to any particular event in the logs or console debug output.

I got a backtrace of one of the crashes. Based that, I recompiled Asterisk without any odbc options. So far, it hasn't crashed again.

Aug  1 16:13:40 gab kernel: [257092.796830] asterisk[23968]: segfault at 0 ip 00007fdae6a896b7 sp 00007fdb2c8d1df8 error 4 in libmysqlclient.so.18.0.0[7fdae6a58000+2b8000]

#0  _int_malloc (av=0x7f1994fce620 <main_arena>, bytes=544) at malloc.c:3389
#1  0x00007f1994ca5020 in __GI___libc_malloc (bytes=544) at malloc.c:2891
#2  0x00007f19751cad74 in my_malloc () from /usr/lib/x86_64-linux-gnu/libmysqlclient.so.18
#3  0x00007f19756d8d9b in my_SQLAllocEnv () from /usr/lib/x86_64-linux-gnu/odbc/libmyodbc.so
#4  0x00007f197c8057cc in ?? () from /usr/lib/x86_64-linux-gnu/libodbc.so.2
#5  0x00007f197c807847 in SQLConnect () from /usr/lib/x86_64-linux-gnu/libodbc.so.2
#6  0x00007f197ca64cbb in odbc_obj_connect (obj=0x2caa808) at res_odbc.c:813
#7  0x00007f197ca646bd in _ast_odbc_request_obj2 (name=0x4115038 "asteriskcdrdb", flags=..., file=0x7f1926290a3b "cdr_adaptive_odbc.c", function=0x7f1926291800 <__PRETTY_FUNCTION__.14687> "odbc_log", lineno=411) at res_odbc.c:722
#8  0x00007f197ca64725 in _ast_odbc_request_obj (name=0x4115038 "asteriskcdrdb", check=0, file=0x7f1926290a3b "cdr_adaptive_odbc.c", function=0x7f1926291800 <__PRETTY_FUNCTION__.14687> "odbc_log", lineno=411) at res_odbc.c:738
#9  0x00007f192628dfb0 in odbc_log (cdr=0x2f9a730) at cdr_adaptive_odbc.c:411
#10 0x000000000049ef72 in post_cdr (cdr=0x2f9a730) at cdr.c:3271
#11 0x00000000004a07c9 in cdr_detach (cdr=0x2d07d00) at cdr.c:3568
#12 0x0000000000497f4c in cdr_object_dispatch (cdr=0x407d1d8) at cdr.c:1199
#13 0x000000000049b3e6 in handle_channel_cache_message (data=0x0, sub=0x2f12218, message=0x7f198463af48) at cdr.c:2129
#14 0x00000000005c9277 in router_dispatch (data=0x2f12168, sub=0x2f12218, message=0x7f198463af48) at stasis_message_router.c:201
#15 0x00000000005b89c5 in subscription_invoke (sub=0x2f12218, message=0x7f198463af48) at stasis.c:433
#16 0x00000000005b954e in dispatch_exec_async (local=0x7f19931acda0) at stasis.c:702
#17 0x00000000005d37ae in ast_taskprocessor_execute (tps=0x2f122f8) at taskprocessor.c:848
#18 0x00000000005d1ead in default_tps_processing_function (data=0x2f11328) at taskprocessor.c:183
#19 0x00000000005e7437 in dummy_start (data=0x2f11fe0) at utils.c:1237
#20 0x00007f1995f980a4 in start_thread (arg=0x7f19931ad700) at pthread_create.c:309
#21 0x00007f1994d1187d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111
#0  _int_malloc (av=0x7f1994fce620 <main_arena>, bytes=544) at malloc.c:3389
       p = 0x2d27b90
       nb = 560
       idx = 35
       bin = 0x7f1994fce898 <main_arena+632>
       victim = 0x2d27b90
       size = <optimized out>
       victim_index = <optimized out>
       remainder = <optimized out>
       remainder_size = <optimized out>
       block = <optimized out>
       bit = <optimized out>
       map = <optimized out>
       fwd = <optimized out>
       bck = 0x0
       errstr = 0x0
       __func__ = "_int_malloc"
#1  0x00007f1994ca5020 in __GI___libc_malloc (bytes=544) at malloc.c:2891
       ar_ptr = 0x7f1994fce620 <main_arena>
       victim = 0x0
       __func__ = "__libc_malloc"
#2  0x00007f19751cad74 in my_malloc () from /usr/lib/x86_64-linux-gnu/libmysqlclient.so.18
No symbol table info available.
#3  0x00007f19756d8d9b in my_SQLAllocEnv () from /usr/lib/x86_64-linux-gnu/odbc/libmyodbc.so
No symbol table info available.
#4  0x00007f197c8057cc in ?? () from /usr/lib/x86_64-linux-gnu/libodbc.so.2
No symbol table info available.
#5  0x00007f197c807847 in SQLConnect () from /usr/lib/x86_64-linux-gnu/libodbc.so.2
No symbol table info available.
#6  0x00007f197ca64cbb in odbc_obj_connect (obj=0x2caa808) at res_odbc.c:813
       res = 0
       err = 68243512
       mlen = 0
       msg = "\300\304\032\223\031\177\000\000\000\000\000\000\000\000\000\000`\200Η\031\177\000\000[\367R", '\000' <repeats 13 times>, "\260\247\312\002\000\000\000\000v\312_\000\000\000\000\000\320\313_\000\000\000\000\000\233\311_\000\000\000\000\000'\002\000\000\001\000\000\000\001\000\000\000\000\000\000\000\260\247\312\002\000\000\000\000 \305\032\223\031\177\000\000\001\270E\000\000\000\000\000dE\246|\031\177\000\000\343\313_\000\000\000\000\000\233\311_\000\000\000\000\000v\002\000\000\000\000\000\000\362&\246|\031\177\000\000\030", '\000' <repeats 15 times>, "Y\310E\000\000\000\000\000\260\247\312\002\000\000\000\000\350\247\312\002\000\000\000\000P\305\032\223\031\177\000"
       state = "\320@\322\002\000\000\000\000\000"
       con = 0x2ba9e20
       negative_cache_expiration = 0
       __PRETTY_FUNCTION__ = "odbc_obj_connect"
#7  0x00007f197ca646bd in _ast_odbc_request_obj2 (name=0x4115038 "asteriskcdrdb", flags=..., file=0x7f1926290a3b "cdr_adaptive_odbc.c", function=0x7f1926291800 <__PRETTY_FUNCTION__.14687> "odbc_log", lineno=411) at res_odbc.c:722
       obj = 0x2caa808
       class = 0x2d6d9c8
       __PRETTY_FUNCTION__ = "_ast_odbc_request_obj2"
#8  0x00007f197ca64725 in _ast_odbc_request_obj (name=0x4115038 "asteriskcdrdb", check=0, file=0x7f1926290a3b "cdr_adaptive_odbc.c", function=0x7f1926291800 <__PRETTY_FUNCTION__.14687> "odbc_log", lineno=411) at res_odbc.c:738
       flags = {flags = 0}
#9  0x00007f192628dfb0 in odbc_log (cdr=0x2f9a730) at cdr_adaptive_odbc.c:411
       first = 1
       tableptr = 0x4115000
       entry = 0xcc400000000
       obj = 0x2caa808
       sql = 0x3c8abc0
       sql2 = 0x3cba640
       tmp = 0x609e95 <__PRETTY_FUNCTION__.17236> "post_cdr"
       colbuf = "\000INLEY II,GREG\000\000\061@207.148.217.210:5060&PJSIP/392/sip:392@207.148.217.210:1025&P\000\060\307\032\223\031\177\000\000@6\373\002\000\000\000\000x6\373\002\000\000\000\000\230\066\373\002\000\000\000\000\360\307\032\223\031\177\000\000\273\304E\000\000\000\000\000o\314_\000\000\000\000\000`\315_\000\000\000\000\000\004\002\000\000\253\001\000\000[\314_\000\000\000\000\000\000\000\000\000\001", '\000' <repeats 11 times>, "\200\310\032\223\031\177\000\000w\334I", '\000' <repeats 13 times>...
       colptr = 0x0
       stmt = 0x0
       rows = 0
       __PRETTY_FUNCTION__ = "odbc_log"
#10 0x000000000049ef72 in post_cdr (cdr=0x2f9a730) at cdr.c:3271
       mod_cfg = 0x2f11428
       __PRETTY_FUNCTION__ = "post_cdr"
       i = 0x40d7500
#11 0x00000000004a07c9 in cdr_detach (cdr=0x2d07d00) at cdr.c:3568
       newtail = 0x400f790
       curr = 0
       mod_cfg = 0x2f11428
       __PRETTY_FUNCTION__ = "cdr_detach"
       submit_batch = 0
#12 0x0000000000497f4c in cdr_object_dispatch (cdr=0x407d1d8) at cdr.c:1199
       mod_cfg = 0x2f11428
       __PRETTY_FUNCTION__ = "cdr_object_dispatch"
       pub_cdr = 0x2d07d00
#13 0x000000000049b3e6 in handle_channel_cache_message (data=0x0, sub=0x2f12218, message=0x7f198463af48) at cdr.c:2129
       cdr = 0x407d1d8
       mod_cfg = 0x2f11428
       __PRETTY_FUNCTION__ = "handle_channel_cache_message"
       update = 0x7f198421fe10
       old_snapshot = 0x7f1984046a00
       new_snapshot = 0x0
       uniqueid = 0x7f19842d2ff4 "1470079457.1002"
       name = 0x7f19842d2fd2 "PJSIP/gb_trunk-000003e0"
       it_cdr = 0x0
#14 0x00000000005c9277 in router_dispatch (data=0x2f12168, sub=0x2f12218, message=0x7f198463af48) at stasis_message_router.c:201
       router = 0x2f12168
       route = {message_type = 0x2eed218, callback = 0x49b0d2 <handle_channel_cache_message>, data = 0x0}
#15 0x00000000005b89c5 in subscription_invoke (sub=0x2f12218, message=0x7f198463af48) at stasis.c:433
       __PRETTY_FUNCTION__ = "subscription_invoke"
#16 0x00000000005b954e in dispatch_exec_async (local=0x7f19931acda0) at stasis.c:702
       sub = 0x2f12218
       message = 0x7f198463af48
#17 0x00000000005d37ae in ast_taskprocessor_execute (tps=0x2f122f8) at taskprocessor.c:848
       local = {local_data = 0x2f12218, data = 0x7f198463af48}
       t = 0x7f19841deb40
       size = 2
       __PRETTY_FUNCTION__ = "ast_taskprocessor_execute"
#18 0x00000000005d1ead in default_tps_processing_function (data=0x2f11328) at taskprocessor.c:183
       listener = 0x2f11328
       tps = 0x2f122f8
       pvt = 0x2fb2ee0
       sem_value = -1826959664
       res = 0
       __PRETTY_FUNCTION__ = "default_tps_processing_function"
#19 0x00000000005e7437 in dummy_start (data=0x2f11fe0) at utils.c:1237
       __cancel_buf = {__cancel_jmp_buf = {{__cancel_jmp_buf = {0, 1876681208568311110, 0, 139747897802848, 0, 139747818919680, 1876681208559922502, -2001734496472699578}, __mask_was_saved = 0}}, __pad = {0x7f19931acef0, 0x0, 0x0, 0x0}}
       __cancel_routine = 0x44f61d <ast_unregister_thread>
       __cancel_arg = 0x7f19931ad700
       __not_first_call = 0
       ret = 0x0
       a = {start_routine = 0x5d1e17 <default_tps_processing_function>, data = 0x2f11328, name = 0x27731d0 "default_tps_processing_function started at [  200] taskprocessor.c default_listener_start()"}
#20 0x00007f1995f980a4 in start_thread (arg=0x7f19931ad700) at pthread_create.c:309
       __res = <optimized out>
       pd = 0x7f19931ad700
       now = <optimized out>
       unwind_buf = {cancel_jmp_buf = {{jmp_buf = {139747818919680, -2001733863470174906, 0, 139747897802848, 0, 139747818919680, 1876681208566213958, 1876666628945964358}, mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 0x0, 0x0}, data = {prev = 0x0, cleanup = 0x0,
             canceltype = 0}}}
       not_first_call = <optimized out>
       pagesize_m1 = <optimized out>
       sp = <optimized out>
       freesize = <optimized out>
       __PRETTY_FUNCTION__ = "start_thread"
#21 0x00007f1994d1187d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111
No locals.
Comments:By: Asterisk Team (asteriskteam) 2016-08-02 12:11:19.513-0500

Thanks for creating a report! The issue has entered the triage process. That means the issue will wait in this status until a Bug Marshal has an opportunity to review the issue. Once the issue has been reviewed you will receive comments regarding the next steps towards resolution.

A good first step is for you to review the [Asterisk Issue Guidelines|https://wiki.asterisk.org/wiki/display/AST/Asterisk+Issue+Guidelines] if you haven't already. The guidelines detail what is expected from an Asterisk issue report.

Then, if you are submitting a patch, please review the [Patch Contribution Process|https://wiki.asterisk.org/wiki/display/AST/Patch+Contribution+Process].
By: Rusty Newton (rnewton) 2016-08-02 12:27:32.924-0500

Closing this out as a duplicate of ASTERISK-26041 and ASTERISK-26235 , please upgrade to Asterisk 13.10 or greater or upgrade UnixODBC.