[Home]

Summary:ASTERISK-07217: MixMonitor causes segfault with voicemail
Reporter:jmls (jmls)Labels:
Date Opened:2006-06-21 17:08:48Date Closed:2006-09-03 18:40:44
Priority:CriticalRegression?No
Status:Closed/CompleteComponents:Applications/app_mixmonitor
Versions:Frequency of
Occurrence
Related
Issues:
Environment:Attachments:( 0) 20060722__bug7411__1.2.diff.txt
( 1) 20060722__bug7411__trythis.diff.txt
( 2) btfull-20060903-1915.txt
( 3) console20060628-2217.txt
( 4) console20060820-0856.txt
( 5) gdb20060621-2307.txt
( 6) gdb20060731-1423.txt
( 7) gdb20060731-1427.txt
( 8) gdb20060731-1530.txt
( 9) gdb20060820-0856.txt
Description:If you are running MixMonitor on a channel, and the channel goes to voicemail, if the caller hangs up just after the "beep" then asterisk will segfault.

bt full attached (compiled with DONT_OPTIMIZE flag set in compiler flags)

****** ADDITIONAL INFORMATION ******

================================================================================
Extensions.conf portion that causes this problem:

exten => 704,1,Answer()
exten => 704,n,MixMonitor(${UNIQUEID}.gsm)
exten => 704,n,Voicemail(780|su)
exten => 704,n,Hangup()
================================================================================
Console output below

Asterisk SVN-trunk-r35366M, Copyright (C) 1999 - 2006 Digium, Inc. and others.
Created by Mark Spencer <markster@digium.com>
Asterisk comes with ABSOLUTELY NO WARRANTY; type 'show warranty' for details.
This is free software, with components licensed under the GNU General Public
License version 2 and other licenses; you are welcome to redistribute it under
certain conditions. Type 'show license' for details.
=========================================================================
Connected to Asterisk SVN-trunk-r35366M currently running on foxtrot (pid = 12100)
Verbosity is at least 3
 == Parsing '/etc/asterisk/manager.conf': Found
 == Manager 'from-debtnet' logged on from 127.0.0.1
   -- Executing [704@from-sip:1] Answer("SIP/706-20d7", "") in new stack
   -- Executing [704@from-sip:2] MixMonitor("SIP/706-20d7", "1150927371.0.gsm") in new stack
   -- Executing [704@from-sip:3] VoiceMail("SIP/706-20d7", "780|su") in new stack
   -- Playing 'vm-theperson' (language 'en')
 == Begin MixMonitor Recording SIP/706-20d7
   -- Playing 'digits/7' (language 'en')
   -- Playing 'digits/8' (language 'en')
   -- Playing 'digits/0' (language 'en')
   -- Playing 'vm-isunavail' (language 'en')
   -- Playing 'beep' (language 'en')
   -- Recording the message
   -- x=0, open writing:  /var/spool/asterisk/voicemail/default/780/tmp/kIIpk9 format: wav49, 0x92cade0
   -- x=1, open writing:  /var/spool/asterisk/voicemail/default/780/tmp/kIIpk9 format: gsm, 0x92c6ff0
   -- x=2, open writing:  /var/spool/asterisk/voicemail/default/780/tmp/kIIpk9 format: wav, 0x92e39d8
   -- User hung up
 == Spawn extension (from-sip, 704, 3) exited non-zero on 'SIP/706-20d7'
   -- Executing [h@from-sip:1] NoOp("SIP/706-20d7", "Reached Hangup Extension") in new stack
   -- Executing [h@from-sip:2] Hangup("SIP/706-20d7", "") in new stack
 == Spawn extension (from-sip, h, 2) exited non-zero on 'SIP/706-20d7'
 == End MixMonitor Recording SIP/706-20d7
foxtrot*CLI> *** glibc detected *** malloc(): memory corruption: 0x092df1a0 ***
Comments:By: jmls (jmls) 2006-06-21 17:10:04

I know that the example extensions are nonsensical, but this is the smallest example I can make that illustrates the problem. This was initially caused by a call coming into a call queue where there were no members so it jumped to voicemail
By: jmls (jmls) 2006-06-21 17:25:46

As a workaround, if you put StopMixMonitor before the voicemail, then it won't crash

exten => 704,1,Answer()
exten => 704,n,MixMonitor(${UNIQUEID}.gsm)
exten => 704,n,SayDigits(12345)
exten => 704,n,StopMixMonitor()
exten => 704,n,Voicemail(780|su)
exten => 704,n,Hangup()
By: Serge Vecher (serge-v) 2006-06-28 15:06:37

I can't reproduce this with r36192. Dialing from a SIP Phone straight into the the sample extenstion you've shown

   -- Executing [7000@ldaccess:1] Answer("SIP/108-094b58a0", "") in new stack
   -- Executing [7000@ldaccess:2] MixMonitor("SIP/108-094b58a0", "1151525047.23.gsm") in new stack
[Jun 28 16:04:07] DEBUG[12215]: channel.c:1164 ast_channel_spy_add: Spy MixMonitor added to channel SIP/108-094b58a0
   -- Executing [7000@ldaccess:3] VoiceMail("SIP/108-094b58a0", "105|su") in new stack
[Jun 28 16:04:07] DEBUG[12215]: channel.c:1318 queue_frame_to_spies: Building translator from gsm to SLINEAR for spies on channel SIP/108-094b58a0
   -- Playing '/var/spool/asterisk/voicemail/default/105/unavail' (language 'en')
 == Begin MixMonitor Recording SIP/108-094b58a0
[Jun 28 16:04:07] DEBUG[12215]: channel.c:1318 queue_frame_to_spies: Building translator from ulaw to SLINEAR for spies on channel SIP/108-094b58a0
[Jun 28 16:04:07] DEBUG[12215]: channel.c:1318 queue_frame_to_spies: Building translator from ulaw to SLINEAR for spies on channel SIP/108-094b58a0
[Jun 28 16:04:22] DEBUG[12215]: app.c:942 ast_lock_path: Locked path '/var/spool/asterisk/voicemail/default/105/INBOX'
[Jun 28 16:04:22] DEBUG[12215]: app.c:953 ast_unlock_path: Unlocked path '/var/spool/asterisk/voicemail/default/105/INBOX'
   -- Playing 'beep' (language 'en')
   -- Recording the message
[Jun 28 16:04:22] DEBUG[12215]: app.c:520 __ast_play_and_record: play_and_record: <None>, /var/spool/asterisk/voicemail/default/105/tmp/19Rssf, 'gsm|wav49|wav'
[Jun 28 16:04:22] DEBUG[12215]: app.c:541 __ast_play_and_record: Recording Formats: sfmts=gsm
   -- x=0, open writing:  /var/spool/asterisk/voicemail/default/105/tmp/19Rssf format: gsm, 0x94b7ce8
   -- x=1, open writing:  /var/spool/asterisk/voicemail/default/105/tmp/19Rssf format: wav49, 0x94bc090
   -- x=2, open writing:  /var/spool/asterisk/voicemail/default/105/tmp/19Rssf format: wav, 0x94ad188
   -- User hung up
[Jun 28 16:04:31] DEBUG[12215]: app.c:942 ast_lock_path: Locked path '/var/spool/asterisk/voicemail/default/105/INBOX'
[Jun 28 16:04:31] DEBUG[12215]: app.c:953 ast_unlock_path: Unlocked path '/var/spool/asterisk/voicemail/default/105/INBOX'
 == Spawn extension (ldaccess, 7000, 3) exited non-zero on 'SIP/108-094b58a0'
   -- Executing [h@ldaccess:1] Hangup("SIP/108-094b58a0", "") in new stack
 == Spawn extension (ldaccess, h, 1) exited non-zero on 'SIP/108-094b58a0'
[Jun 28 16:04:31] DEBUG[12215]: channel.c:1227 ast_channel_spy_remove: Spy MixMonitor removed from channel SIP/108-094b58a0
By: jmls (jmls) 2006-06-28 16:20:22

I can :)

Using SVN-trunk-r36192M

Executing [7704@from-sip:1] Answer("SIP/7706-0a01c138", "") in new stack
   -- Executing [7704@from-sip:2] MixMonitor("SIP/7706-0a01c138", "1151529324.0.gsm") in new stack
   -- Executing [7704@from-sip:3] SayDigits("SIP/7706-0a01c138", "123456") in new stack
   -- Playing 'digits/1' (language 'en')
 == Begin MixMonitor Recording SIP/7706-0a01c138
   -- Playing 'digits/2' (language 'en')
   -- Playing 'digits/3' (language 'en')
   -- Playing 'digits/4' (language 'en')
   -- Playing 'digits/5' (language 'en')
   -- Playing 'digits/6' (language 'en')
   -- Executing [7704@from-sip:4] VoiceMail("SIP/7706-0a01c138", "780|su") in new stack
   -- Playing 'vm-theperson' (language 'en')
   -- Playing 'digits/7' (language 'en')
   -- Playing 'digits/8' (language 'en')
   -- Playing 'digits/0' (language 'en')
   -- Playing 'vm-isunavail' (language 'en')
   -- Playing 'beep' (language 'en')
   -- Recording the message
   -- x=0, open writing:  /var/spool/asterisk/voicemail/default/780/tmp/nuXWSY format: gsm, 0xa020998
   -- User ended message by pressing #
   -- Playing 'auth-thankyou' (language 'en')
   -- Executing [7704@from-sip:5] Hangup("SIP/7706-0a01c138", "") in new stack
 == Spawn extension (from-sip, 7704, 5) exited non-zero on 'SIP/7706-0a01c138'
   -- Executing [h@from-sip:1] NoOp("SIP/7706-0a01c138", "Reached Hangup Extension") in new stack
   -- Executing [h@from-sip:2] Hangup("SIP/7706-0a01c138", "") in new stack
 == Spawn extension (from-sip, h, 2) exited non-zero on 'SIP/7706-0a01c138'
 == End MixMonitor Recording SIP/7706-0a01c138
Jun 28 22:15:41 ERROR[14810]: chan_sip.c:14180 sipsock_read: We could NOT get the channel lock for SIP/7706-0a01c138!
Jun 28 22:15:41 ERROR[14810]: chan_sip.c:14181 sipsock_read: SIP transaction failed: 00078599-3d3d0019-2952739f-1267494f@192.168.0.200
*** glibc detected *** malloc(): memory corruption: 0x0a02f0b8 ***
Aborted (core dumped)

ack! built without dont_optimize. I'll do that now and attach a back trace
By: jmls (jmls) 2006-06-28 16:24:34

sorry, I'll redo it with debug on as well
By: jmls (jmls) 2006-06-28 16:32:44

I've attached console20060628-2217.txt. This is the console output, but I've included the last few lines below:

Jun 28 22:27:45 DEBUG[19619]: pbx.c:1675 pbx_extension_helper: Launching 'Hangup'
   -- Executing [7704@from-sip:5] Hangup("SIP/7706-085cbd08", "") in new stack
Jun 28 22:27:45 DEBUG[19619]: pbx.c:2258 __ast_pbx_run: Spawn extension (from-sip,7704,5) exited non-zero on 'SIP/7706-085cbd08'
Jun 28 22:27:45 DEBUG[19619]: pbx.c:1675 pbx_extension_helper: Launching 'NoOp'
   -- Executing [h@from-sip:1] NoOp("SIP/7706-085cbd08", "Reached Hangup Extension") in new stack
Jun 28 22:27:45 DEBUG[19619]: pbx.c:1675 pbx_extension_helper: Launching 'Hangup'
   -- Executing [h@from-sip:2] Hangup("SIP/7706-085cbd08", "") in new stack
Jun 28 22:27:45 DEBUG[19619]: pbx.c:2378 __ast_pbx_run: Spawn extension (from-sip,h,2) exited non-zero on 'SIP/7706-085cbd08'
 == End MixMonitor Recording SIP/7706-085cbd08
*** glibc detected *** malloc(): memory corruption: 0x08596490 ***
Aborted (core dumped)
By: jmls (jmls) 2006-06-28 16:33:31

this is the gdb below:

(gdb) bt full
#0  0x0018b7a2 in _dl_sysinfo_int80 () from /lib/ld-linux.so.2
No symbol table info available.
#1  0x001cb7f5 in raise () from /lib/tls/libc.so.6
No symbol table info available.
#2  0x001cd199 in abort () from /lib/tls/libc.so.6
No symbol table info available.
#3  0x001ff4ea in __libc_message () from /lib/tls/libc.so.6
No symbol table info available.
#4  0x0020664c in _int_malloc () from /lib/tls/libc.so.6
No symbol table info available.
ASTERISK-1  0x002080b1 in malloc () from /lib/tls/libc.so.6
No symbol table info available.
ASTERISK-2  0x0022a11c in opendir () from /lib/tls/libc.so.6
No symbol table info available.
ASTERISK-3  0x028cb95c in __has_voicemail (context=Variable "context" is not available.
) at app_voicemail.c:2237
       dir = Variable "dir" is not available.
By: jmls (jmls) 2006-07-05 12:37:50

Had another ocurrence of this today. stack trace below:

(gdb) bt full
#0  0x0018b7a2 in _dl_sysinfo_int80 () from /lib/ld-linux.so.2
No symbol table info available.
#1  0x001cb7f5 in raise () from /lib/tls/libc.so.6
No symbol table info available.
#2  0x001cd199 in abort () from /lib/tls/libc.so.6
No symbol table info available.
#3  0x001ff4ea in __libc_message () from /lib/tls/libc.so.6
No symbol table info available.
#4  0x0020664c in _int_malloc () from /lib/tls/libc.so.6
No symbol table info available.
ASTERISK-1  0x002080b1 in malloc () from /lib/tls/libc.so.6
No symbol table info available.
ASTERISK-2  0x0022a11c in opendir () from /lib/tls/libc.so.6
No symbol table info available.
ASTERISK-3  0x01bfd264 in __has_voicemail (context=0x1c115b8 "default", mailbox=0x5071220 "706", folder=0x1c11f5e "INBOX", shortcircuit=0) at app_voicemail.c:2237
       dir = (DIR *) 0x0
       de = (struct dirent *) 0x0
       fn = "/var/spool/asterisk/voicemail/default/706/INBOX", '\0' <repeats 208 times>
       ret = 0
ASTERISK-4  0x01bfd6c8 in inboxcount (mailbox=0x9e0a024 "706", newmsgs=0x5071378, oldmsgs=0x5071374) at app_voicemail.c:2313
       tmp = "706", '\0' <repeats 225 times>, "\027=1\000(\023\a\005??\v\b<\023\a\005\000\000\000\000(\023\a\005Gy\005\b"
       context = 0x1c115b8 "default"
ASTERISK-5  0x0808a11f in ast_app_inboxcount (mailbox=0x9e0a024 "706", newmsgs=0x5071378, oldmsgs=0x5071374) at app.c:189
       warned = 0
ASTERISK-6 0x018c8cc5 in sip_send_mwi_to_peer (peer=0x9e09ae0) at chan_sip.c:14243
       p = (struct sip_pvt *) 0x189391a
       newmsgs = 0
       oldmsgs = 0
ASTERISK-7 0x018c949f in do_monitor (data=0x0) at chan_sip.c:14413
       res = 0
       sip = (struct sip_pvt *) 0x0
       peer = (struct sip_peer *) 0x9e09ae0
       t = 1152116940
       fastrestart = 1
       lastpeernum = 1
       curpeernum = 2
       reloading = 0
       __PRETTY_FUNCTION__ = "do_monitor"
ASTERISK-8 0x080bacd9 in dummy_start (data=0x9e0af60) at utils.c:536
       _buffer = {__routine = 0x80a66a1 <ast_unregister_thread>, __arg = 0x5071bb0, __canceltype = 0, __prev = 0x0}
       ret = (void *) 0x0
       a = {start_routine = 0x18c8f19 <do_monitor>, data = 0x0,
 name = 0x9e0af70 "do_monitor", ' ' <repeats 11 times>, "started at [14444] chan_sip.c restart_monitor()"}
ASTERISK-9 0x00312371 in start_thread () from /lib/tls/libpthread.so.0
No symbol table info available.
ASTERISK-10 0x0026b9be in clone () from /lib/tls/libc.so.6
No symbol table info available.
By: Serge Vecher (serge-v) 2006-07-05 12:57:14

jmls: don't forget to mention the revision when attaching the bt.
By: jmls (jmls) 2006-07-05 14:28:44

sorry, it was 36753
By: Serge Vecher (serge-v) 2006-07-05 15:06:53

Is this only reproducible if monitor a SIP call going into voicemail. Does this happen if a Zap channel is involved?
By: jmls (jmls) 2006-07-05 16:05:19

no, I just reproduced it with a zap call:

-- Accepting call from '07803034440' to '444604' on channel 0/1, span 1
   -- Executing [444604@isdn10:1] Goto("Zap/1-1", "common|7704|1") in new stack
   -- Goto (common,7704,1)
   -- Executing [7704@common:1] Answer("Zap/1-1", "") in new stack
   -- Executing [7704@common:2] MixMonitor("Zap/1-1", "1152133439.23.gsm") in new stack
   -- Executing [7704@common:3] SayDigits("Zap/1-1", "123456") in new stack
 == Begin MixMonitor Recording Zap/1-1
   -- Playing 'digits/1' (language 'en')
   -- Playing 'digits/2' (language 'en')
   -- Playing 'digits/3' (language 'en')
   -- Playing 'digits/4' (language 'en')
   -- Playing 'digits/5' (language 'en')
   -- Playing 'digits/6' (language 'en')
   -- Executing [7704@common:4] VoiceMail("Zap/1-1", "780|su") in new stack
Jul  5 22:04:04 NOTICE[3240]: chan_sip.c:13902 handle_request_register: Registration from 'sip:747@192.168.6.6' failed for '192.168.6.215' - No matching peer found
   -- Playing 'vm-theperson' (language 'en')
   -- Playing 'digits/7' (language 'en')
   -- Playing 'digits/8' (language 'en')
   -- Playing 'digits/0' (language 'en')
   -- Playing 'vm-isunavail' (language 'en')
   -- Playing 'beep' (language 'en')
   -- Recording the message
   -- x=0, open writing:  /var/spool/asterisk/voicemail/default/780/tmp/fj3Wmq format: gsm, 0x8939a98
   -- User ended message by pressing #
   -- Playing 'auth-thankyou' (language 'en')
   -- Executing [7704@common:5] Hangup("Zap/1-1", "") in new stack
 == Spawn extension (common, 7704, 5) exited non-zero on 'Zap/1-1'
 == End MixMonitor Recording Zap/1-1
foxtrot*CLI> *** glibc detected *** malloc(): memory corruption: 0x089394e8 ***

Disconnected from Asterisk server

I'll post a stack trace
By: jmls (jmls) 2006-07-05 16:06:04

(gdb) bt full
#0  0x0018b7a2 in _dl_sysinfo_int80 () from /lib/ld-linux.so.2
No symbol table info available.
#1  0x001cb7f5 in raise () from /lib/tls/libc.so.6
No symbol table info available.
#2  0x001cd199 in abort () from /lib/tls/libc.so.6
No symbol table info available.
#3  0x001ff4ea in __libc_message () from /lib/tls/libc.so.6
No symbol table info available.
#4  0x0020664c in _int_malloc () from /lib/tls/libc.so.6
No symbol table info available.
ASTERISK-1  0x002080b1 in malloc () from /lib/tls/libc.so.6
No symbol table info available.
ASTERISK-2  0x0022a11c in opendir () from /lib/tls/libc.so.6
No symbol table info available.
ASTERISK-3  0x0639b264 in __has_voicemail (context=0x63af5b8 "default", mailbox=0x2bd6220 "706", folder=0x63aff5e "INBOX", shortcircuit=0) at app_voicemail.c:2237
       dir = (DIR *) 0x0
       de = (struct dirent *) 0x0
       fn = "/var/spool/asterisk/voicemail/default/706/INBOX", '\0' <repeats 208 times>
       ret = 0
ASTERISK-4  0x0639b6c8 in inboxcount (mailbox=0x88f61e4 "706", newmsgs=0x2bd6378, oldmsgs=0x2bd6374) at app_voicemail.c:2313
       tmp = "706", '\0' <repeats 225 times>, "\027=1\000(c?\002??\v\b<c?\002\000\000\000\000(c?\002Gy\005\b"
       context = 0x63af5b8 "default"
ASTERISK-5  0x0808a11f in ast_app_inboxcount (mailbox=0x88f61e4 "706", newmsgs=0x2bd6378, oldmsgs=0x2bd6374) at app.c:189
       warned = 1
ASTERISK-6 0x01ad0cc5 in sip_send_mwi_to_peer (peer=0x88f5ca0) at chan_sip.c:14243
       p = (struct sip_pvt *) 0x1a9b91a
       newmsgs = 0
       oldmsgs = 0
ASTERISK-7 0x01ad149f in do_monitor (data=0x0) at chan_sip.c:14413
       res = 0
       sip = (struct sip_pvt *) 0x0
       peer = (struct sip_peer *) 0x88f5ca0
       t = 1152133456
       fastrestart = 1
       lastpeernum = 1
       curpeernum = 2
       reloading = 0
       __PRETTY_FUNCTION__ = "do_monitor"
ASTERISK-8 0x080bacd9 in dummy_start (data=0x88f70f0) at utils.c:536
       _buffer = {__routine = 0x80a66a1 <ast_unregister_thread>, __arg = 0x2bd6bb0, __canceltype = 0, __prev = 0x0}
       ret = (void *) 0x0
       a = {start_routine = 0x1ad0f19 <do_monitor>, data = 0x0,
 name = 0x88f7100 "do_monitor", ' ' <repeats 11 times>, "started at [14444] chan_sip.c restart_monitor()"}
ASTERISK-9 0x00312371 in start_thread () from /lib/tls/libpthread.so.0
No symbol table info available.
ASTERISK-10 0x0026b9be in clone () from /lib/tls/libc.so.6
No symbol table info available.
By: jmls (jmls) 2006-07-05 16:13:28

what I don't understand is why the following line  in the stack trace is there:

7 0x0639b264 in __has_voicemail (context=0x63af5b8 "default", mailbox=0x2bd6220 "706", folder=0x63aff5e "INBOX", shortcircuit=0) at app_voicemail.c:2237
       dir = (DIR *) 0x0
       de = (struct dirent *) 0x0
       fn = "/var/spool/asterisk/voicemail/default/706/INBOX", '\0' <repeats 208 times>
       ret = 0

the 706 is *my* sip phone's mailbox (for sip/7706), but it was a zap call that came into the 7704 extension, leaving a voicemail for mailbox 780.

my voicemail.conf is :

[general]
format=gsm
serveremail=asterisk
attach=yes
maxmsg=9999
skipms=3000
maxsilence=10
silencethreshold=128
maxlogins=3
emaildateformat=%A, %B %d, %Y at %r
[zonemessages]
eastern=America/New_York|'vm-received' Q 'digits/at' IMp
central=America/Chicago|'vm-received' Q 'digits/at' IMp
central24=America/Chicago|'vm-received' q 'digits/at' H N 'hours'
military=Zulu|'vm-received' q 'digits/at' H N 'hours' 'phonetic/z_p'
european=Europe/Copenhagen|'vm-received' a d b 'digits/at' HM

sendvoicemail=yes       ; Context to Send voicemail from [option 5 from the advanced menu]

[default]

706 => 0000,Julian Lyndon-Smith,jmls@tessera.co.uk,attach=no
780 => 0000,Test Voicemail,jmls@tessera.co.uk,attach=no

[other]
1234 => 5678,Company2 User,root@localhost
By: Serge Vecher (serge-v) 2006-07-05 16:29:48

hmm, let's see the respective sip.conf and extensions.conf snippets.
By: jmls (jmls) 2006-07-05 16:38:04

extensions.conf
=========================================================
[from-sip]

exten => 7704,1,Answer()
exten => 7704,n,MixMonitor(${UNIQUEID}.gsm)
exten => 7704,n,SayDigits(123456)
;exten => 7704,n,StopMixMonitor()
;exten => 7704,n,Wait(1)
exten => 7704,n,Voicemail(780|su)
exten => 7704,n,Hangup()

sip.conf
=========================================================
[7706]
type=friend
secret=7706
context=from-sip
callerid="Julian Lyndon-Smith" <7706>
host=dynamic                    ; This device needs to register
nat=no
canreinvite=no
disallow=all
allow=ulaw
allow=alaw
allow=gsm                       ; GSM consumes far less bandwidth than ulaw
mailbox=706
By: jmls (jmls) 2006-07-07 07:05:37

Is there anything else I can do in order to try to resolve this issue ?
By: Serge Vecher (serge-v) 2006-07-07 10:08:19

Julian: hopefully somebody from the development team will be able to pick up enough clues from the backtrace as to how to solve this.

If possible, see if you can narrow this issue down a bit better -- I don't know why, but I can't reproduce this on my machine
By: jmls (jmls) 2006-07-07 10:11:44

I'll try to narrow it down, but the 4 extension lines cause the problem for me :) I'm not sure what else I can do
By: jonr800 (jonr800) 2006-07-11 22:56:40

It appears that I'm having the same issue with 1.2-r37378
By: Serge Vecher (serge-v) 2006-07-12 08:29:17

in that case, please also upload a backtrace from Asterisk built with 'make dont-optimize'
By: Serge Vecher (serge-v) 2006-07-21 14:14:12

jmls: bweschke has committed a fix to very related bug ASTERISK-7380 under r38051. Please test the latest trunk and report back, please. Thanks
By: Tilghman Lesher (tilghman) 2006-07-22 12:23:47

Please try this patch.  This isn't a committable patch, but if it makes the heap corruption go away, then I know exactly where the problem lies, and we can work to address it.
By: jonr800 (jonr800) 2006-07-22 12:38:51

Patch works on Asterisk SVN-branch-1.2-r37856M  (made the changes by hand)
By: Tilghman Lesher (tilghman) 2006-07-22 14:19:41

jonr800:  here is a potential fix for 1.2.  Can you test this and ensure that it works?
By: jonr800 (jonr800) 2006-07-22 14:38:10

No luck with this patch..

   -- Playing 'beep' (language 'en')
   -- Recording the message
   -- x=0, open writing:  /var/spool/asterisk/voicemail/default/1000/tmp/ffHPVw format: wav, 0x817f450
   -- User hung up
 == Spawn extension (macro-std-exten, sw-5-NOANSWER, 1) exited non-zero on 'SIP/7344205555-081d36c0' in macro 'std-exten'
 == Spawn extension (macro-std-exten, sw-5-NOANSWER, 1) exited non-zero on 'SIP/7344205555-081d36c0'
 == End MixMonitor Recording SIP/7344205555-081d36c0
 == Executing []
milkshakes*CLI> *** glibc detected *** malloc(): memory corruption: 0x08184b68 ***

Disconnected from Asterisk server
/usr/sbin/safe_asterisk: line 50: 15965 Aborted                 (core dumped) ${ASTSBINDIR}/asterisk ${CLIARGS} ${ASTARGS} >&/dev/${TTY} </dev/${TTY}
Asterisk ended with exit status 134
Asterisk exited on signal 6.
Automatically restarting Asterisk.
Executing last minute cleanups
By: Tilghman Lesher (tilghman) 2006-07-22 14:50:59

jmis:  the cause of this error is heap corruption.  Unfortunately, it's very difficult to track down what exactly is causing the heap corruption, because in many cases, the thread that finds the heap corruption is not the thread that caused the heap corruption.

One of the developers needs remote root access to a machine which is exhibiting this issue.  Come find one of us on IRC on irc.freenode.net, channel #asterisk-dev.  We'll need a corefile from a non-optimized build to be left on that machine, so we can examine the heap memory and try to find what bit of code is causing the memory heap corruption.
By: Serge Vecher (serge-v) 2006-07-27 08:51:46

jmls: any luck with testing > r38051
By: jmls (jmls) 2006-07-30 17:21:10

sorry guys, have been on holiday (!) for the past two weeks. I will post my findings tomorrow.
By: jmls (jmls) 2006-07-31 09:25:16

vechers: Still able to make it crash with r38548. I have two more core dumps available if required.

Corydon76: Root access may be difficult as we are behind a firewall, and all remote access is done via VPN.
By: jmls (jmls) 2006-07-31 09:32:55

Corydon76: just tried the patch on svn trunk. I had to try several times to make it crash (normally takes 2 or 3) but it still crashed.
By: Serge Vecher (serge-v) 2006-07-31 10:13:21

jmls: let's see the bt's from the new revision ...
By: jmls (jmls) 2006-07-31 10:25:09

bt files uploaded as requested
By: jmls (jmls) 2006-08-08 01:51:55

Is there anything else I can do in order to try to resolve this issue ?
By: Serge Vecher (serge-v) 2006-08-08 08:44:34

jmls: let's revisit Corydon's suggestion and try to get one of the developers from #asterisk-dev to log onto your system and diagnose the problem directly.
By: jmls (jmls) 2006-08-08 08:48:34

I could possibly ask the network guys to allow ssh access to that machine, but would need source ip and a time frame as they won't leave it open
By: Serge Vecher (serge-v) 2006-08-08 09:34:38

jmls: that's good to know. Please join the #asterisk-dev channel, find a developer and coordinate the time, IP, and other good stuff directly.
By: jmls (jmls) 2006-08-09 02:33:51

we did that, had a few "technical" problems. We will try again either later today or tomorrow.
By: jmls (jmls) 2006-08-19 04:43:22

I'm going to try build a new machine with the latest svn trunk to retest this.
By: jmls (jmls) 2006-08-20 03:02:01

I have installed a new machine from scratch (Dell 2850 with 2 GB ram, using Centos4.3 with all patches and updates applied) and installed the latest trunk (40602) and compiled. I can still reproduce this error.

I have uploaded two more files:  gdb20060820-0856.txt and  console20060820-0856.txt.

I press hangup just as the beep ends. It is not always possible to crash it, but within 4 or 5 attempts I will be able to cause the crash. We often have 300+ voicemails a day, a lot of which people hangup without leaving a message, so we simply cannot use mixmonitor at the moment.
By: jmls (jmls) 2006-08-20 13:21:02

fwiw, I tried the patch for ASTERISK-7043, but it had no effect
By: jmls (jmls) 2006-08-24 17:12:35

does someone need to look at this on my machine, or is the fact that I was able to reproduce this on a totally different machine good enough to move this forward ?
By: Serge Vecher (serge-v) 2006-08-30 09:31:09

julian: there was a patch patch posted in 7829 that may fix this. Do you mind trying it out? The line that the patch affects is slightly different in trunk, or later version of 1.2.x, but you should be able to figure it out.

thanks.
By: jmls (jmls) 2006-08-30 09:46:45

I was in the process of trying to do exactly that - but with the latest trunk MixMonitor segfaults straight away, so I can't :) There have been a couple of messages (http://lists.digium.com/pipermail/asterisk-dev/2006-August/022697.html) about this. Once this is fixed, I will try the patch.
By: Serge Vecher (serge-v) 2006-08-30 09:48:16

you can always downgrade and patch the last-known-working-revision ;)
By: Serge Vecher (serge-v) 2006-08-31 13:02:54

alright, that segfault introduced by frame-caching stuff should be fixed as of r41389...
By: jmls (jmls) 2006-08-31 14:34:40

Nope - mixmonitor segfaults as soon as it starts with r41611. I've raised a new bug for that one (ASTERISK-7641). As soon as I can, I'll try the patch :)
By: jmls (jmls) 2006-09-03 13:13:07

just tried the patch from 7829, but that did not work. I will attach a new bt full to demonstrate
By: jmls (jmls) 2006-09-03 13:16:53

btfull-20060903-1915.txt file attached
By: Joshua C. Colp (jcolp) 2006-09-03 18:40:42

Fixed in trunk as of revision 41959. That is all.