[Home]

Summary:ASTERISK-30502: asterisk crashes when loading res_odbc
Reporter:Philip de Orleans (falves111)Labels:
Date Opened:2023-04-24 12:56:04Date Closed:
Priority:MinorRegression?No
Status:Triage/NewComponents:Resources/res_odbc
Versions:18.11.2 Frequency of
Occurrence		Constant
Related
Issues:
Environment:Linux Centos 7 and 8Attachments:( 0) asterisk-crash.txt
( 1) valgrind.log
Description:I use mariadb and the odbc connector. The command Isql works fine. when I try to load res_odbc, it crashes in both operating systems and with two versions of the driver. I have a trace, will upload it next
Comments:By: Asterisk Team (asteriskteam) 2023-04-24 12:56:07.263-0500

WARNING

JIRA will be going read-only at the end of April, 2023. We will be starting fresh on Github at https://github.com/asterisk/asterisk at that time. No issues or patches will be copied to Github. If you file an issue on JIRA at this time you will need to recreate it on Github after this date. The same applies if you have a patch.

WARNING

Thanks for creating a report! The issue has entered the triage process. That means the issue will wait in this status until a Bug Marshal has an opportunity to review the issue. Once the issue has been reviewed you will receive comments regarding the next steps towards resolution. Please note that log messages and other files should not be sent to the Sangoma Asterisk Team unless explicitly asked for. All files should be placed on this issue in a sanitized fashion as needed.

A good first step is for you to review the [Asterisk Issue Guidelines|https://wiki.asterisk.org/wiki/display/AST/Asterisk+Issue+Guidelines] if you haven't already. The guidelines detail what is expected from an Asterisk issue report.

Then, if you are submitting a patch, please review the [Patch Contribution Process|https://wiki.asterisk.org/wiki/display/AST/Patch+Contribution+Process].

Please note that once your issue enters an open state it has been accepted. As Asterisk is an open source project there is no guarantee or timeframe on when your issue will be looked into. If you need expedient resolution you will need to find and pay a suitable developer. Asking for an update on your issue will not yield any progress on it and will not result in a response. All updates are posted to the issue when they occur.

Please note that by submitting data, code, or documentation to Sangoma through JIRA, you accept the Terms of Use present at [https://www.asterisk.org/terms-of-use/|https://www.asterisk.org/terms-of-use/].
By: Joshua C. Colp (jcolp) 2023-04-24 13:03:13.533-0500

From looking at the backtrace the crash is occurring within UnixODBC itself on a strcmp. I don't think there's anything from an Asterisk perspective that can be done or is ultimately the cause. We are calling the SQLConnect API call with three strings. That's it. Same as we always have. We also haven't received any other reports of this issue, so it is likely environmental.

Your report is also conflicting. You specified both 18.11.2 and 18.17.1. Which is it?
What is the actual configuration?
By: Philip de Orleans (falves111) 2023-04-24 14:13:25.448-0500

It happens also in asterisk 16, latest, from today.
The version of this trace is 18.17.1
this is my res_odbc.conf

[cdr]
enabled=yes
dsn=cdr
sanitysql => select 1
isolation => read_uncommitted
username=root
pre-connect => yes
forcecommit => yes
connect_timeout => 10
negative_connection_cache => 0
max_connections =>500

[asterisk]
enabled=yes
dsn=local
sanitysql => select 1
isolation => read_uncommitted
username=root
pre-connect => yes
forcecommit => yes
connect_timeout => 10
negative_connection_cache => 0
max_connections =>500

If I do this
isql cdr
or isql local

it works fine.

By: Joshua C. Colp (jcolp) 2023-04-24 15:05:42.215-0500

And the UnixODBC underlying configuration?
By: Philip de Orleans (falves111) 2023-04-24 15:18:50.137-0500



[cdr]
Description = MySQL ODBC Driver Testing
Driver = maria
Socket = /var/lib/mysql/mysql.sock
User = root
Database = asterisk
Option = 3

[local]
Description = MySQL ODBC Driver Testing
Driver = maria
Socket = /var/lib/mysql/mysql.sock
User = root
Password =
Database = asterisk
Option = 3

cat /etc/odbcinst.ini
[ODBC]
Trace=no
Trace File=/tmp/sql.log
Pooling=yes

[maria]
Description=ODBC for MySQL
Driver=/usr/lib64/mariadb/libmaodbc.so
FileUsage=1
threading=0



as I said
isql cdr and isql local do work fine.

The only strange thing is the kernel, this is uname -r
6.1.0-0.deb11.5-amd64
cat /proc/cpuinfo | grep sse2
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts mmx fxsr sse sse2 ss syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts nopl xtopology tsc_reliable nonstop_tsc cpuid tsc_known_freq pni pclmulqdq ssse3 cx16 pcid

This a a vmware ESXI box, debian 11 with a Centos 8 container.
The weird thing is that Asterisk works fine, except for res_odbc.so
It just crashes if load it, always.

I am using gdb, but I can use valgrind if you send me precise instructions.


By: Philip de Orleans (falves111) 2023-04-24 15:24:34.295-0500

valgrind
[Apr 24 20:23:06] NOTICE[154068]: loader.c:2415 load_modules: 285 modules will be loaded.
Loading extconfig.
[ Initializing Custom Configuration Options ]
 == extconfig => (Configuration)
Loading logger.
 == logger => (Logger)
Loading res_curl.so.
 == res_curl.so => (cURL Resource Module)
Loading res_odbc_transaction.so.
 == Registered application 'ODBC_Commit'
 == Registered application 'ODBC_Rollback'
 == Registered custom function 'ODBC'
 == res_odbc_transaction.so => (ODBC transaction resource)
Loading res_odbc.so.
==154068== Invalid read of size 1
==154068==    at 0x4C3F114: strcmp (vg_replace_strmem.c:924)
==154068==    by 0x2E625ED5: ??? (in /usr/lib64/libodbc.so.2.0.0)
==154068==    by 0x2E6289D0: SQLConnect (in /usr/lib64/libodbc.so.2.0.0)
==154068==    by 0x2E4142F1: odbc_obj_connect (res_odbc.c:1052)
==154068==    by 0x2E413A52: _ast_odbc_request_obj2 (res_odbc.c:935)
==154068==    by 0x2E413E57: _ast_odbc_request_obj (res_odbc.c:983)
==154068==    by 0x2E4133B4: odbc_register_class (res_odbc.c:796)
==154068==    by 0x2E412CE7: load_odbc_config (res_odbc.c:696)
==154068==    by 0x2E41480F: load_module (res_odbc.c:1123)
==154068==    by 0x51EC4A: start_resource (loader.c:1728)
==154068==    by 0x51F6B6: start_resource_attempt (loader.c:1915)
==154068==    by 0x520060: start_resource_list (loader.c:2012)
==154068==  Address 0x0 is not stack'd, malloc'd or (recently) free'd
==154068==
==154068==
==154068== Process terminating with default action of signal 11 (SIGSEGV): dumping core
==154068==  Access not within mapped region at address 0x0
==154068==    at 0x4C3F114: strcmp (vg_replace_strmem.c:924)
==154068==    by 0x2E625ED5: ??? (in /usr/lib64/libodbc.so.2.0.0)
==154068==    by 0x2E6289D0: SQLConnect (in /usr/lib64/libodbc.so.2.0.0)
==154068==    by 0x2E4142F1: odbc_obj_connect (res_odbc.c:1052)
==154068==    by 0x2E413A52: _ast_odbc_request_obj2 (res_odbc.c:935)
==154068==    by 0x2E413E57: _ast_odbc_request_obj (res_odbc.c:983)
==154068==    by 0x2E4133B4: odbc_register_class (res_odbc.c:796)
==154068==    by 0x2E412CE7: load_odbc_config (res_odbc.c:696)
==154068==    by 0x2E41480F: load_module (res_odbc.c:1123)
==154068==    by 0x51EC4A: start_resource (loader.c:1728)
==154068==    by 0x51F6B6: start_resource_attempt (loader.c:1915)
==154068==    by 0x520060: start_resource_list (loader.c:2012)
==154068==  If you believe this happened as a result of a stack
==154068==  overflow in your program's main thread (unlikely but
==154068==  possible), you can try to increase the size of the
==154068==  main thread stack using the --main-stacksize= flag.
==154068==  The main thread stack size used in this run was 8388608.
==154068==
==154068== HEAP SUMMARY:
==154068==     in use at exit: 2,137,223 bytes in 9,295 blocks
==154068==   total heap usage: 17,957 allocs, 8,662 frees, 8,526,398 bytes allocated
==154068==
==154068== LEAK SUMMARY:
==154068==    definitely lost: 0 bytes in 0 blocks
==154068==    indirectly lost: 0 bytes in 0 blocks
==154068==      possibly lost: 433,615 bytes in 1,082 blocks
==154068==    still reachable: 1,703,608 bytes in 8,213 blocks
==154068==                       of which reachable via heuristic:
==154068==                         length64           : 164,272 bytes in 197 blocks
==154068==                         newarray           : 1,536 bytes in 16 blocks
==154068==         suppressed: 0 bytes in 0 blocks
==154068== Rerun with --leak-check=full to see details of leaked memory
==154068==
==154068== For lists of detected and suppressed errors, rerun with: -s
==154068== ERROR SUMMARY: 1 errors from 1 contexts (suppressed: 0 from 0)
Segmentation fault
By: Philip de Orleans (falves111) 2023-04-24 15:31:29.212-0500

The full valgrind log
By: Philip de Orleans (falves111) 2023-04-24 17:02:46.549-0500

I changed the names of the data sources to aaaa and bbbb, but it keeps blowing up

grep -A 10 -B 10 -i "size 1"  valgrind.log
==155431== Downloading debug info for /root/.cache/debuginfod_client/42bf80b053847aba0ac56c472f6c7ee661f476ef/debuginfo...
--155431--   Considering /root/.cache/debuginfod_client/1fafe29b3296abdc580acfaa7367684d8e91e7ab/debuginfo ..
--155431--   .. build-id is valid
==155431== Successfully downloaded debug file for /root/.cache/debuginfod_client/42bf80b053847aba0ac56c472f6c7ee661f476ef/debuginfo
--155431-- Reading syms from /usr/lib64/mariadb/libmaodbc.so
--155431-- Reading syms from /usr/lib64/libodbcinst.so.2.0.0
==155431== Downloading debug info for /usr/lib64/libodbcinst.so.2.0.0...
==155431== Server query failed: No such file or directory
--155431--    object doesn't have a symbol table
--155431-- REDIR: 0x80404e0 (libc.so.6:__strpbrk_sse42) redirected to 0x4c434c0 (strpbrk)
==155431== Invalid read of size 1
==155431==    at 0x4C3F114: strcmp (vg_replace_strmem.c:924)
==155431==    by 0x2E121ED5: ??? (in /usr/lib64/libodbc.so.2.0.0)
==155431==    by 0x2E1249D0: SQLConnect (in /usr/lib64/libodbc.so.2.0.0)
==155431==    by 0x2DF102F1: odbc_obj_connect (res_odbc.c:1052)
==155431==    by 0x2DF0FA52: _ast_odbc_request_obj2 (res_odbc.c:935)
==155431==    by 0x2DF0FE57: _ast_odbc_request_obj (res_odbc.c:983)
==155431==    by 0x2DF0F3B4: odbc_register_class (res_odbc.c:796)
==155431==    by 0x2DF0ECE7: load_odbc_config (res_odbc.c:696)
==155431==    by 0x2DF1080F: load_module (res_odbc.c:1123)
==155431==    by 0x51EC4A: start_resource (loader.c:1728)
--
==155431==      possibly lost: 435,079 bytes in 1,091 blocks
==155431==    still reachable: 1,734,464 bytes in 8,227 blocks
==155431==                       of which reachable via heuristic:
==155431==                         length64           : 164,272 bytes in 197 blocks
==155431==                         newarray           : 1,536 bytes in 16 blocks
==155431==         suppressed: 0 bytes in 0 blocks
==155431==
==155431== ERROR SUMMARY: 1014 errors from 1014 contexts (suppressed: 0 from 0)
==155431==
==155431== 1 errors in context 1 of 1014:
==155431== Invalid read of size 1
==155431==    at 0x4C3F114: strcmp (vg_replace_strmem.c:924)
==155431==    by 0x2E121ED5: ??? (in /usr/lib64/libodbc.so.2.0.0)
==155431==    by 0x2E1249D0: SQLConnect (in /usr/lib64/libodbc.so.2.0.0)
==155431==    by 0x2DF102F1: odbc_obj_connect (res_odbc.c:1052)
==155431==    by 0x2DF0FA52: _ast_odbc_request_obj2 (res_odbc.c:935)
==155431==    by 0x2DF0FE57: _ast_odbc_request_obj (res_odbc.c:983)
==155431==    by 0x2DF0F3B4: odbc_register_class (res_odbc.c:796)
==155431==    by 0x2DF0ECE7: load_odbc_config (res_odbc.c:696)
==155431==    by 0x2DF1080F: load_module (res_odbc.c:1123)
==155431==    by 0x51EC4A: start_resource (loader.c:1728)