[Home]

Summary:ASTERISK-22763: Segfault in __ao2_find ()
Reporter:David Brillert (aragon)Labels:
Date Opened:2013-10-24 14:36:01Date Closed:2013-11-15 08:26:24.000-0600
Priority:CriticalRegression?
Status:Closed/CompleteComponents:Channels/chan_sip/General
Versions:11.6.0 Frequency of
Occurrence
Occasional
Related
Issues:
is a clone ofASTERISK-22875 CLONE - Segfault in __ao2_find ()
Environment:centos 5.9 64bitAttachments:( 0) gdb_thread_apply_all.txt
( 1) gdb_trace.txt
Description:Segfault.  Backtrace attached.
Asterisk was compiled with DONT_OPTIMIZE and BETTER_BACKTRACES
Comments:By: Matt Jordan (mjordan) 2013-10-27 14:08:59.359-0500

It looks like your backtrace is still from binaries that don't have debug symbols:

{noformat}
#0  0x00000000004455c2 in __ao2_find ()
No symbol table info available.
#1  0x00000000004bc2be in find_interface ()
No symbol table info available.
#2  0x00000000004bc419 in ast_format_cmp ()
No symbol table info available.
#3  0x00000000004c0262 in cmp_cb ()
No symbol table info available.
#4  0x0000000000444a5d in internal_ao2_callback ()
No symbol table info available.
#5  0x0000000000445091 in __ao2_callback ()
No symbol table info available.
{noformat}

Unfortunately, that means we don't really know much about what occurred leading up to the crash.

Can you reproduce the crash? What was the system doing leading up to it? Can you produce a backtrace with all of the symbols?

By: David Brillert (aragon) 2013-11-01 10:35:57.486-0500

We compiled Asterisk with MENUSELECT_CFLAGS=DONT_OPTIMIZE LOADABLE_MODULES BETTER_BACKTRACES

So we don't understand why the backtrace is missing symbols...?


By: Matt Jordan (mjordan) 2013-11-11 09:26:41.508-0600

It's possible you re-compiled Asterisk with the symbols, but didn't actually replace all of the modules with the ones that were re-compiled.

As it is, the backtrace only shows that at some point, something went terribly wrong when looking up a format on a channel. Because there isn't much information in the backtrace, it's unclear as to what went wrong however. If you can reproduce the problem with libraries that have all of the symbols and can re-generate the backtrace, that would help in trying to determine what the cause of the issue is.

By: David Brillert (aragon) 2013-11-15 08:25:47.824-0600

I haven't reproduced this since updating to a newer SVN.

By: David Brillert (aragon) 2013-11-15 08:26:24.167-0600

Since I cannot reproduce on latest SVN, I'm closing this out.

By: David Brillert (aragon) 2013-11-18 13:16:14.914-0600

I spoke to soon.
The box is crashing again.
Can a marshal please reopen?

By: David Brillert (aragon) 2013-11-18 13:18:50.490-0600

This is a GDB PID trace.  Since the box locked.

The core file gdb trace was pretty bad.  Not much there.

(gdb) bt
#0  0x0000000000445152 in __ao2_find ()
#1  0x00000000004bbf5e in find_interface ()
#2  0x00000000004bc0b9 in ast_format_cmp ()
#3  0x000000000046ff59 in ast_write ()
#4  0x00000000004b8969 in ast_readaudio_callback ()
#5  0x00000000004b8b19 in ast_fsread_audio ()
#6  0x00000000004757b7 in __ast_read ()
#7  0x00000000004b91c3 in waitstream_core ()
#8  0x00000000004b970f in ast_waitstream ()
#9  0x000000000050d353 in async_wait ()
#10 0x65706f5f666f5f73 in ?? ()
#11 0x5f736e6f69746172 in ?? ()
#12 0x0000000000375f39 in ?? ()
#13 0x00002b2542d446e0 in ?? ()
#14 0x1530000000d62b00 in ?? ()
#15 0x0000000000000001 in ?? ()
#16 0x00002b2542d44800 in ?? ()
#17 0x0000000000000000 in ?? ()
(gdb) bt full
#0  0x0000000000445152 in __ao2_find ()
No symbol table info available.
#1  0x00000000004bbf5e in find_interface ()
No symbol table info available.
#2  0x00000000004bc0b9 in ast_format_cmp ()
No symbol table info available.
#3  0x000000000046ff59 in ast_write ()
No symbol table info available.
#4  0x00000000004b8969 in ast_readaudio_callback ()
No symbol table info available.
#5  0x00000000004b8b19 in ast_fsread_audio ()
No symbol table info available.
#6  0x00000000004757b7 in __ast_read ()
No symbol table info available.
#7  0x00000000004b91c3 in waitstream_core ()
No symbol table info available.
#8  0x00000000004b970f in ast_waitstream ()
No symbol table info available.
#9  0x000000000050d353 in async_wait ()
No symbol table info available.
#10 0x65706f5f666f5f73 in ?? ()
No symbol table info available.
#11 0x5f736e6f69746172 in ?? ()
No symbol table info available.
#12 0x0000000000375f39 in ?? ()
No symbol table info available.
---Type <return> to continue, or q <return> to quit---
#13 0x00002b2542d446e0 in ?? ()
No symbol table info available.
#14 0x1530000000d62b00 in ?? ()
No symbol table info available.
#15 0x0000000000000001 in ?? ()
No symbol table info available.
#16 0x00002b2542d44800 in ?? ()
No symbol table info available.
#17 0x0000000000000000 in ?? ()
No symbol table info available.

By: David Brillert (aragon) 2013-11-21 08:12:21.302-0600

No need to re-open this.
I cloned it at ASTERISK-22875