Summary:	ASTERISK-23759: Crash when IMAP voicemail count reaches a high number of messages +250
Reporter:	Alejandro Rios P. (alerios)	Labels:
Date Opened:	2014-05-19 19:30:17	Date Closed:	2014-05-27 09:00:48
Priority:	Critical	Regression?
Status:	Closed/Complete	Components:	Applications/app_voicemail/IMAP
Versions:	SVN 12.4.0	Frequency of Occurrence
Related Issues:
Environment:	SVN URL: http://svn.asterisk.org/svn/asterisk/branches/12 Revision: 414209 CentOS release 6.4 (Final) # uname -a Linux 2.6.32-358.23.2.el6.x86_64 #1 SMP Wed Oct 16 18:37:12 UTC 2013 x86_64 x86_64 x86_64 GNU/Linux # dovecot --version 2.0.13	Attachments:	( 0) full-backtrace-ASTERISK-23759 ( 1) full-backtrace-ASTERISK-23759.txt ( 2) full-backtrace-dont-optimize-ASTERISK-23759.txt
Description:	An Asterisk setup with IMAP works ok with the default value of "maxmsg=100" on voicemail.conf However, if I increase that value to a higher limit (maxmsg=400, for example), Asterisk crashes when reaching a voicemail count of about 250 to 280 messages (see the core dump and backtrace below) I have reproduced this issue with different asterisk versions (1.6.2.20, 12.0.0 and the latest asterisk 12 branch from SVN) **Message count for the test mailbox: # grep X-Asterisk-VM-Message-Num /home/imap_user/mail/19779362020 | tail -n 1 X-Asterisk-VM-Message-Num: 257 **Backtrace: {noformat} Core was generated by `/usr/local/asterisk_12branch_vdc400/sbin/asterisk -f -C /usr/l'. Program terminated with signal 11, Segmentation fault. #0  mail_open_work (d=0x7f39aa054f60, stream=0x101,    name=0x7f39bc00f900 "{vms400:143/imap/authuser=asterisk/notls/user=19779362020}INBOX", options=0) at mail.c:1283 1283    if ((stream->dtb == d) && (d->flags & DR_RECYCLE) && Missing separate debuginfos, use: debuginfo-install audit-libs-2.2-2.el6.x86_64 glibc-2.12-1.132.el6.x86_64 keyutils-libs-1.4-4.el6.x86_64 krb5-libs-1.10.3-10.el6_4.6.x86_64 libcom_err-1.41.12-18.el6.x86_64 libgcc-4.4.7-4.el6.x86_64 libselinux-2.0.94-5.3.el6_4.1.x86_64 libstdc++-4.4.7-4.el6.x86_64 libuuid-2.17.2-12.14.el6.x86_64 libxml2-2.7.6-14.el6.x86_64 mysql-libs-5.1.71-1.el6.x86_64 ncurses-libs-5.7-3.20090208.el6.x86_64 nss-softokn-freebl-3.14.3-3.el6_4.x86_64 openssl-1.0.1e-16.el6_5.4.x86_64 pam-1.1.1-17.el6.x86_64 sqlite-3.6.20-1.el6.x86_64 zlib-1.2.3-29.el6.x86_64 (gdb) (gdb) (gdb) (gdb) (gdb) bt #0  mail_open_work (d=0x7f39aa054f60, stream=0x101,    name=0x7f39bc00f900 "{vms400:143/imap/authuser=asterisk/notls/user=19779312345}INBOX", options=0) at mail.c:1283 #1  0x00007f39a9d789ee in mail_open (stream=0x101,    name=0x7f39a8089a60 "{vms400:143/imap/authuser=asterisk/notls/user=19779312345}INBOX", options=0) at mail.c:1260 #2  0x00007f39a9d3df68 in init_mailstream (vms=0x7f39bc01a390, box=1) at app_voicemail.c:2953 #3  0x00007f39a9d3e7d3 in __messagecount (context=0x7f39a808a2ac "voicemail", mailbox=0x7f39a808a2a0 "19779312345",    folder=<value optimized out>) at app_voicemail.c:2451 #4  0x00007f39a9d3f007 in inboxcount2 (mailbox_context=<value optimized out>, urgentmsgs=0x7f39a808b2fc, newmsgs=0x7f39a8094e9c,    oldmsgs=0x7f39a8094e98) at app_voicemail.c:2771 #5  0x00007f39a9d3f590 in inboxcount (mailbox=<value optimized out>, newmsgs=0x7f39a8094e9c, oldmsgs=<value optimized out>)    at app_voicemail.c:5967 #6  0x00007f39a9d5687e in leave_voicemail (chan=0x7f39980148a8, ext=<value optimized out>, options=0x7f39a8095050)    at app_voicemail.c:6616 #7  0x00007f39a9d599b0 in vm_exec (chan=0x7f39980148a8, data=<value optimized out>) at app_voicemail.c:11648 #8  0x0000000000546685 in ?? () #9  0x0000000000000000 in ?? () {noformat}
Comments:	By: Matt Jordan (mjordan) 2014-05-20 08:41:49.084-0500 The backtrace appears to be incomplete. Please attach a fully generated backtrace using the instructions on the Asterisk wiki: https://wiki.asterisk.org/wiki/display/AST/Getting+a+Backtrace By: Alejandro Rios P. (alerios) 2014-05-20 14:08:49.237-0500 Sorry, I sent the wrong file last time. Here is the backtraces as per the guidelines. Thanks, Alejandro. By: Rusty Newton (rnewton) 2014-05-22 09:04:25.458-0500 re-attaching reporters trace as .txt so it can be viewed easily. By: Rusty Newton (rnewton) 2014-05-22 09:06:00.998-0500 [~alerios] did you recompile with DONT_OPTIMIZE  as well as BETTER_BACKTRACES? I still see a lot of values optimized out. By: Alejandro Rios P. (alerios) 2014-05-22 17:49:08.418-0500 Hi, I recompiled using DONT_OPTIMIZE and BETTER_BACKTRACES, sorry, it's been a long time since I reported my last issue. Cheers, Alejandro. By: Matt Jordan (mjordan) 2014-05-27 09:00:41.229-0500 This is a bug in the IMAP libraries. Where the crash occurs in Asterisk, Asterisk is attempting to open a mail stream in IMAP via {{mail_open}}: {code} vms->mailstream = mail_open (stream, tmp, debug ? OP_DEBUG : NIL); {code} At this juncture in the backtrace, we have already previously opened a mailstream via IMAP and gotten back a {{MAILSTREAM*}}. Unfortunately, we can see in the backtrace that this stream is junk: {code} #2  0x00007f6996f2a971 in init_mailstream (vms=0x7f699801a390, box=1) at app_voicemail.c:2953        stream = 0x101        debug = 0        tmp = "{vms400:143/imap/authuser=asterisk/notls/user=19779312345}INBOX", '\000' <repeats 16 times>, "М\r\224i\177\000\000\000\016\000\230i\177\000\000(\243\r\224i\177\000\000\342\312\362\226i\177\000\000\200\233\r\224i\177\000\000\360\211$\227i\177\000\000\304z\374\226i\177\000\000ۚR\000\000\000\000\000\240\233\r\224\266\006\000\000\060\211$\227i\177\000\000\a\212\374\226i\177\000\000\060\357\374\226i\177\000\000,\243\r\224<\r\000\000\324i\374\226i\177\000\000\260\233\r\224i\177\000\000 \243\r\224\000\000\000\000\340\233\r\224i\177\000\000\206\311\362\226i\177\000\000\000\000\000\000\000\000\000\000$\241\r\224i\177\000"        __PRETTY_FUNCTION__ = "init_mailstream" {code} That is, we previously called {{mail_open}} and the IMAP library gave us back an invalid handle to a {{MAILSTREAM}}. At this point, it's a ticking time bomb until something explodes. In this case, when we call {{mail_open}} a subsequent time (which is valid), the IMAP library inevitably dereferences the invalid handle it gave back to us previously, and a seg fault occurs. Something in the IMAP library can't handle the size of the mailbox you're attempting to use it with. There isn't anything Asterisk can do to work around that.