[Home]

Summary:ASTERISK-19761: mp3_read crash
Reporter:Chris Maciejewski (chris-mac)Labels:
Date Opened:2012-04-20 02:20:49Date Closed:2012-05-11 18:57:19
Priority:CriticalRegression?
Status:Closed/CompleteComponents:Applications/app_mp3
Versions:1.8.5.0 Frequency of
Occurrence		Frequent
Related
Issues:
is duplicated byASTERISK-20482 Certain mp3 file will cause crash in format_mp3.c
is duplicated byASTERISK-19881 My Asterisk crashes multiple times daily
Environment:Ubuntu 8.04Attachments:( 0) backtrace-2012-04-22.txt
( 1) backtrace-2012-04-23.txt
( 2) backtrace-2012-05-04.txt
( 3) backtrace-2012-05-04-2.txt
Description:Our Asterisk 1.8.4 (processing 1000-2000 calls a day) crashes 2-4 times a day when playing MP3 files with BT as below:

-------
#0  0x00007f4094fc46e9 in mp3_read (s=0x7f409c57e228, whennext=0x4037cb04) at format_mp3.c:214
214 s->buf[p->buflen + AST_FRIENDLY_OFFSET] = p->dbuf[(p->buflen-save)+p->dbufoffset];
(gdb) bt
#0  0x00007f4094fc46e9 in mp3_read (s=0x7f409c57e228, whennext=0x4037cb04) at format_mp3.c:214
#1  0x00000000004b1975 in ast_readframe (s=0x0) at file.c:700
#2  0x00007f4099348767 in gen_generate (chan=0x214bf08, data=0x7f408c000b40, len=-1671961832,
   samples=<value optimized out>) at app_externalivr.c:246
#3  0x000000000046c380 in ast_read_generator_actions (chan=0x214bf08, f=0x7f409c53a2a0) at channel.c:3567
#4  0x0000000000472e99 in __ast_read (chan=0x214bf08, dropaudio=0) at channel.c:4132
#5  0x00007f4099348d56 in eivr_comm (chan=0x214bf08, u=0x4037e7c0, eivr_events_fd=0x4037dd74,
   eivr_commands_fd=0x4037dd60, eivr_errors_fd=0x4037dd50, args=<value optimized out>, flags=<value optimized out>)
   at app_externalivr.c:672
#6  0x00007f409934a4c7 in app_exec (chan=0x214bf08, data=<value optimized out>) at app_externalivr.c:560
#7  0x00000000004e646c in pbx_exec (c=0x214bf08, app=0x22b2590,
   data=0x40380df0 "/enc/etc/asterisk/ivr/ivr.php(902,-nodebug)") at pbx.c:1406
#8  0x00000000004f107a in pbx_extension_helper (c=0x214bf08, con=<value optimized out>, context=0x214c460 "Ivr",
   exten=0x214c4b0 "61871001199", priority=11, label=0x0, callerid=0x2165b20 "61401334005", action=E_SPAWN,
   found=0x40382f3c, combined_find_spawn=1) at pbx.c:4085
#9  0x00000000004f4cb0 in __ast_pbx_run (c=0x214bf08, args=0x0) at pbx.c:4608
#10 0x00000000004f65db in pbx_thread (data=0x0) at pbx.c:5017
#11 0x000000000052e04c in dummy_start (data=<value optimized out>) at utils.c:973
#12 0x00007f40a126f3f7 in start_thread () from /lib/libpthread.so.0
#13 0x00007f40a1aa7bbd in clone () from /lib/libc.so.6
-------

-------
#0  0x00007fc3f17e96e9 in mp3_read (s=0x7fc3f85734e8, whennext=0x410e12c4) at format_mp3.c:214
214 s->buf[p->buflen + AST_FRIENDLY_OFFSET] = p->dbuf[(p->buflen-save)+p->dbufoffset];
(gdb) bt
#0  0x00007fc3f17e96e9 in mp3_read (s=0x7fc3f85734e8, whennext=0x410e12c4) at format_mp3.c:214
#1  0x00000000004af2bf in ast_readaudio_callback (s=0x7fc3f85734e8) at file.c:700
#2  0x00000000004af519 in ast_fsread_audio (data=0x0) at file.c:786
#3  0x000000000047391f in __ast_read (chan=0x7fc3f8542558, dropaudio=0) at channel.c:3739
#4  0x00000000004b07ed in waitstream_core (c=0x7fc3f8542558, breakon=0x7fc3f4d50daf "", forward=0x557867 "", reverse=0x557867 "", skip_ms=<value optimized out>,
   audiofd=-1, cmdfd=-1, context=0x0) at file.c:1231
#5  0x00000000004b0ce6 in ast_waitstream (c=0x0, breakon=0x0) at file.c:1321
#6  0x00007fc3f4d5085e in playback_exec (chan=0x7fc3f8542558, data=0x410e1d7e "/tmp/VmAnnouncement.2491") at app_playback.c:468
#7  0x00000000004e646c in pbx_exec (c=0x7fc3f8542558, app=0x16336d0, data=0x410e1d7e "/tmp/VmAnnouncement.2491") at pbx.c:1406
#8  0x00007fc3fc30c808 in handle_exec (chan=0x7fc3f8542558, agi=0x410e2b00, argc=3, argv=0x410e18b0) at res_agi.c:2392
#9  0x00007fc3fc30d01e in agi_handle_command (chan=0x7fc3f8542558, agi=0x410e2b00, buf=0x410e1d70 "EXEC", dead=0) at res_agi.c:3274
#10 0x00007fc3fc30d6fc in run_agi (chan=0x7fc3f8542558, request=0x410e25f0 "vm-record", agi=0x410e2b00, pid=11450, status=0x410e2b68, dead=0, argc=4,
   argv=0x410e2648) at res_agi.c:3434
#11 0x00007fc3fc310d80 in agi_exec_full (chan=0x7fc3f8542558, data=<value optimized out>, enhanced=<value optimized out>, dead=0) at res_agi.c:3721
#12 0x00000000004e646c in pbx_exec (c=0x7fc3f8542558, app=0x7fc3f84f8900, data=0x410e5df0 "vm-record,110572,145942,-nodebug") at pbx.c:1406
#13 0x00000000004f107a in pbx_extension_helper (c=0x7fc3f8542558, con=<value optimized out>, context=0x7fc3f8542ab0 "AGI-Voicemail", exten=0x7fc3f8542b00 "0IVR",
   priority=3, label=0x0, callerid=0x7fc3f84d4e50 "16199487941", action=E_SPAWN, found=0x410e7f3c, combined_find_spawn=1) at pbx.c:4085
#14 0x00000000004f4cb0 in __ast_pbx_run (c=0x7fc3f8542558, args=0x0) at pbx.c:4608
#15 0x00000000004f65db in pbx_thread (data=0x0) at pbx.c:5017
#16 0x000000000052e04c in dummy_start (data=<value optimized out>) at utils.c:973
#17 0x00007fc3fda4e3f7 in start_thread () from /lib/libpthread.so.0
#18 0x00007fc3fe286bbd in clone () from /lib/libc.so.6
-------

-------
#0  0x00007f03bb2606e9 in mp3_read (s=0x7f03b4080648, whennext=0x41afcb04) at format_mp3.c:214
214 s->buf[p->buflen + AST_FRIENDLY_OFFSET] = p->dbuf[(p->buflen-save)+p->dbufoffset];
(gdb) bt
#0  0x00007f03bb2606e9 in mp3_read (s=0x7f03b4080648, whennext=0x41afcb04) at format_mp3.c:214
#1  0x00000000004b1975 in ast_readframe (s=0x0) at file.c:700
#2  0x00007f03bf5e4767 in gen_generate (chan=0x2424a38, data=0x246b610, len=-1274542280, samples=<value optimized out>) at app_externalivr.c:246
#3  0x000000000046c380 in ast_read_generator_actions (chan=0x2424a38, f=0x24341d0) at channel.c:3567
#4  0x0000000000472e99 in __ast_read (chan=0x2424a38, dropaudio=0) at channel.c:4132
#5  0x00007f03bf5e4d56 in eivr_comm (chan=0x2424a38, u=0x41afe7c0, eivr_events_fd=0x41afdd74, eivr_commands_fd=0x41afdd60, eivr_errors_fd=0x41afdd50,
   args=<value optimized out>, flags=<value optimized out>) at app_externalivr.c:672
#6  0x00007f03bf5e64c7 in app_exec (chan=0x2424a38, data=<value optimized out>) at app_externalivr.c:560
#7  0x00000000004e646c in pbx_exec (c=0x2424a38, app=0x7f03c04db870, data=0x41b00df0 "/enc/etc/asterisk/ivr/ivr.php(502,-nodebug)") at pbx.c:1406
#8  0x00000000004f107a in pbx_extension_helper (c=0x2424a38, con=<value optimized out>, context=0x2424f90 "Ivr", exten=0x2424fe0 "33182885560", priority=11,
   label=0x0, callerid=0x24edaa0 "33493297838", action=E_SPAWN, found=0x41b02f3c, combined_find_spawn=1) at pbx.c:4085
#9  0x00000000004f4cb0 in __ast_pbx_run (c=0x2424a38, args=0x0) at pbx.c:4608
#10 0x00000000004f65db in pbx_thread (data=0x0) at pbx.c:5017
#11 0x000000000052e04c in dummy_start (data=<value optimized out>) at utils.c:973
#12 0x00007f03c74083f7 in start_thread () from /lib/libpthread.so.0
#13 0x00007f03c7c40bbd in clone () from /lib/libc.so.6
-------

Has this been maybe fixed in the most recent 1.8.11?

Kind regards,
Chris
Comments:By: Matt Jordan (mjordan) 2012-04-20 13:08:48.014-0500

Thank you for your bug report. In order to move your issue forward, we require a backtrace[1] from the core file produced after the crash. Also, be sure you have DONT_OPTIMIZE enabled in menuselect within the Compiler Flags section, then:

make install

After enabling, reproduce the crash, and then execute the backtrace[1] instructions. When complete, attach that file to this issue report.

[1] https://wiki.asterisk.org/wiki/display/AST/Getting+a+Backtrace

Please attach a full and valid backtrace to this issue.

There have been no significant updates to app_mp3, so its doubtful that there is any change in behavior between 1.8.5 and 1.8.11.  Please keep in mind that app_mp3 is an extended support module, so response times will reflect that.
By: Chris Maciejewski (chris-mac) 2012-04-20 13:52:20.173-0500

I will re-compile our Asterisk with DONT_OPTIMIZE flag over the weekend and submit full BT as soon as it crashes again next week.
By: Chris Maciejewski (chris-mac) 2012-04-22 06:30:50.343-0500

Attaching Full Backtrace (DONT_OPTIMIZE flag set).

Regards,
Chris
By: Chris Maciejewski (chris-mac) 2012-04-23 01:30:21.361-0500

One more BT attached.
By: Chris Maciejewski (chris-mac) 2012-05-04 01:52:43.237-0500

Another BT attached.
By: Chris Maciejewski (chris-mac) 2012-05-04 04:15:36.126-0500

Another BT attached.
By: Matt Jordan (mjordan) 2012-05-04 10:07:50.524-0500

Chris:

I took a look at the backtraces, and this does indeed appear to be a problem in app_mp3.  As I noted earlier, app_mp3 is an extended support module [1].  As such, development efforts for it come from the Asterisk Open Source developer community.  You may want to contact the asterisk-biz mailing list to see if an open source developer would be willing to look at this issue.

Thanks,

Matt

[1] https://wiki.asterisk.org/wiki/display/AST/Asterisk+Module+Support+States
By: Russell Bryant (russell) 2012-05-06 17:29:09.945-0500

Some additional debug info from Chris:


#0  0x00007f0c94a76754 in mp3_read (s=0x1a325e8, whennext=0x409bb5dc) at format_mp3.c:214

214                    s->buf[p->buflen + AST_FRIENDLY_OFFSET] = p->dbuf[(p->buflen-save)+p->dbufoffset];
(gdb) p p
$1 = (struct mp3_private *) 0x1a32858
(gdb) p s
$2 = (struct ast_filestream *) 0x1a325e8
(gdb) p p->buflen
$3 = 202
(gdb) p save
$4 = 0
(gdb) p p->dbufoffset
$5 = 148382
(gdb)