Summary: | ASTERISK-22350: DUNDI - core dump on shutdown - segfault in sqlite3_reset from /usr/lib/libsqlite3.so.0 | ||
Reporter: | Birger "WIMPy" Harzenetter (wimpy) | Labels: | |
Date Opened: | 2013-08-20 16:58:58 | Date Closed: | 2013-12-19 10:40:01.000-0600 |
Priority: | Major | Regression? | |
Status: | Closed/Complete | Components: | Core/AstDB PBX/pbx_dundi |
Versions: | SVN | Frequency of Occurrence | |
Related Issues: | |||
Environment: | Attachments: | ( 0) pastebin.txt ( 1) pastebin.txt | |
Description: | segfault in sqlite3 after process_clearcache
Asterisk was running in foreground and was killed using ^C | ||
Comments: | By: Birger "WIMPy" Harzenetter (wimpy) 2013-08-20 17:01:24.146-0500 gdb output By: Birger "WIMPy" Harzenetter (wimpy) 2013-08-20 18:20:29.237-0500 output of thread apply all bt By: Rusty Newton (rnewton) 2013-08-21 16:47:23.142-0500 What branch was this reproduced in? Do you have any steps for reproduction? Can you also provide a VERBOSE and DEBUG log during the time of the shutdown? Thanks! By: Birger "WIMPy" Harzenetter (wimpy) 2013-08-21 21:06:14.172-0500 It happened when asterisk was running in foreground (asterisk -gc) and was killed by pressing ^C. I cannot reproduce the fault (on todays version). By: Corey Farrell (coreyfarrell) 2013-08-23 15:10:27.153-0500 Using Ctrl-C to shutdown causes SHUTDOWN_FAST. Modules are not unloaded for SHUTDOWN_FAST or SHUTDOWN_NORMAL, but ast_register_atexit callbacks are run. The 14k pastebin.txt shows that thread #1 is DUNDI trying to use ast_db_gettree. Thread #12 is close_logger() waiting for the logger thread to end. This means that astdb and most other things in the core have already been shutdown by ast_run_atexits. Since asterisk exit's very shortly after ast_run_atexits() this is difficult to reproduce. By: Corey Farrell (coreyfarrell) 2013-10-29 11:24:07.150-0500 This is an issue with asterisk 11+, probably 1.8 too. I think backporting ast_register_cleanup and using it for db.c shutdown is the correct fix. Otherwise pbx_dundi in 1.8/11 should use ast_register_atexit to kill it's background thread before db.c shuts down. By: Alexander Hömig (ahoemig) 2013-11-06 05:57:30.220-0600 I have possible found the reason: When the asterisk stops the "astdb_atexit"-Function will be called. This function calls "clean_statements()" which set the db-pointer in all prepared-Statements to null. When at the same time a database operation occurs and uses a prepared-statement the segfault happens in sqlite3_reset. {code:title=gdb-log|borderStyle=solid} (gdb) #0 0x4038d90e in sqlite3_reset (pStmt=0x82c7410) at sqlite3.c:63297 63297 sqlite3_mutex_enter(v->db->mutex); (gdb) list 63292 int rc; 63293 if( pStmt==0 ){ 63294 rc = SQLITE_OK; 63295 }else{ 63296 Vdbe *v = (Vdbe*)pStmt; 63297 sqlite3_mutex_enter(v->db->mutex); (gdb) (gdb) print *((Vdbe*)pStmt) $7 = {db = 0x0, aOp = 0x852dfa8, aMem = 0x852e1a0, apArg = 0x852e380, aColName = 0x0, pResultSet = 0x0, nMem = 9, nOp = 27, nOpAlloc = 51, nLabel = 2, aLabel = 0x0, nResColumn = 0, nCursor = 2, magic = 3053896648, zErrMsg = 0x0, pPrev = 0x0, pNext = 0x82c3ed0, apCsr = 0x852e388, aVar = 0x852e330, azVar = 0x852e380, nVar = 2, nzVar = 0, cacheCtr = 1, pc = -1, rc = 0, errorAction = 2 '\002', minWriteFileFormat = 255 '?', explain = 0, inVtabMethod = 0, changeCntOn = 1, expired = 0, runOnlyOnce = 0, usesStmtJournal = 0, readOnly = 0, isPrepareV2 = 0, doingRerun = 0, nChange = 0, btreeMask = 1, lockMask = 0, iStatement = 0, aCounter = {0, 0, 0}, startTime = 0, nFkConstraint = 0, nStmtDefCons = 0, zSql = 0x852a160 "`?R\bRT OR REPLACE INTO astdb (key, value) VALUES (?, ?)", pFree = 0x0, pFrame = 0x0, pDelFrame = 0x0, nFrame = 0, expmask = 0, pProgram = 0x0, nOnceFlag = 1, aOnceFlag = 0x852e390 ""} (gdb) {code} Is it possible to set the prepared-statement-pointer to null in the function "clean_stmt" after sqlite3_finalize was called? |