| Summary: | ASTERISK-24052: app_voicemail reloads result in leaked IMAP sockets. | ||||
| Reporter: | Louis Jocelyn Paquet (ljpaquet) | Labels: | |||
| Date Opened: | 2014-07-17 09:09:16 | Date Closed: | 2017-06-15 13:53:09 | ||
| Priority: | Major | Regression? | Yes | ||
| Status: | Closed/Complete | Components: | Applications/app_voicemail/IMAP | ||
| Versions: | SVN 1.8.22.0 | Frequency of Occurrence | Frequent | ||
| Related Issues: |
| ||||
| Environment: | Free pbx distro | Attachments: | ( 0) Issue24052 | ||
| Description: | When we do a voicemail reload, the application is recreating new IMAP connection whithout closing the active one. This is resulting with extra open connection. After about 30 minutes, the old connections moving in states close_wait and stay like this until we restart asterisk.
For my production server with 65 users using IMAP storage vor voicemail, this is resulting with 65 extra IMAP connections every time we do an apply. So after 1 week, we got all IMAP TCP socket use and we need to restart Asterisk because the server cannot establish new IMAP connections. I did some more investigation, here more details about the issue. I need to know who is the right group to adress this issue (free pbx or asterisk) So, I reproduce the issue on a small server were I have only 1 custommer with IMAP connection. I have the same issue on my server were I have 66 custommer with imap voicemail. So to reproduce the issue, you just have to connect one ATA (sip client) on a extension with an IMAP account on an other server (Imap storage) So in the server, if I do tne netstat, I can see a conncetions. {noformat} [root@voip5b ~]# netstat -ap |grep imap tcp 0 0 192.168.101.21:48237 publicIPhidden:imap ESTABLISHED 18675/asterisk tcp 0 0 192.168.101.21:48236 publicIPhidden:imap ESTABLISHED 18675/asterisk {noformat} If I do an voicemail reload, you will see that the application is recreating new connections whithout closing the prevoius one. after 25-30 minutes, the old connections come in close_wait status and never get completely close. After a week, all port become use and the server cannot establish imap connections. {noformat} [root@voip5b ~]# asterisk -r Asterisk 1.8.22.0, Copyright (C) 1999 - 2012 Digium, Inc. and others. Created by Mark Spencer <markster@digium.com> Asterisk comes with ABSOLUTELY NO WARRANTY; type 'core show warranty' for details. This is free software, with components licensed under the GNU General Public License version 2 and other licenses; you are welcome to redistribute it under certain conditions. Type 'core show license' for details. ========================================================================= Connected to Asterisk 1.8.22.0 currently running on voip5b (pid = 18675) Verbosity is at least 3 voip5b*CLI> voicemail reload Reloading voicemail configuration... == Parsing '/etc/asterisk/voicemail.conf': == Found == Parsing '/etc/asterisk/vm_general.inc': == Found == Parsing '/etc/asterisk/vm_email.inc': == Found voip5b*CLI> exit {noformat} {noformat} [root@voip5b ~]# netstat -ap |grep imap tcp 0 0 192.168.101.21:48239 publicIPhidden:imap ESTABLISHED 18675/asterisk tcp 0 0 192.168.101.21:48237 publicIPhidden:imap ESTABLISHED 18675/asterisk tcp 0 0 192.168.101.21:48240 publicIPhidden:imap ESTABLISHED 18675/asterisk tcp 0 0 192.168.101.21:48236 publicIPhidden:imap ESTABLISHED 18675/asterisk {noformat} This is what happening on my production server (66 custommers). 66 IMAP new connections every time we do an apply in free pbx. | ||||
| Comments: | By: Matt Jordan (mjordan) 2014-07-17 09:23:55.539-0500 We require a complete debug log to help triage the issue. This document will provide instructions on how to collect debugging logs from an Asterisk machine for the purpose of helping bug marshals troubleshoot an issue: https://wiki.asterisk.org/wiki/display/AST/Collecting+Debug+Information By: Louis Jocelyn Paquet (ljpaquet) 2014-07-17 10:23:41.419-0500 Hi files and explanation are attach. Thanks By: Louis Jocelyn Paquet (ljpaquet) 2014-07-17 10:25:13.464-0500 The file is attach By: Rusty Newton (rnewton) 2014-07-19 12:35:19.872-0500 Moving reporters descriptive notes from the reference notes field to the description field, as it was difficult to read with no formatting in the reference notes field. By: Rusty Newton (rnewton) 2014-07-19 12:42:49.544-0500 Possible regression of or relation to ASTERISK-13368. Linking to it. By: Louis Jocelyn Paquet (ljpaquet) 2014-07-21 08:22:20.941-0500 Thank you to looking at this issue. Do not hesitate to ask me if you need anything. By: Rusty Newton (rnewton) 2014-07-22 12:26:28.758-0500 I am able to reproduce the issue in the latest SVN of branch 1.8. SVN-branch-1.8-r418641 The issue is as you describe. Each run of "voicemail reload" or "module reload app_voicemail.so" results in an established connection on a new port, without closing the old. The issue is compounded by the number of IMAP voicemail users. {quote} I did some more investigation, here more details about the issue. I need to know who is the right group to adress this issue (free pbx or asterisk) {quote} The issue appears to be with the IMAP feature in app_voicemail. I think you can close out your FreePBX issue. By: Rusty Newton (rnewton) 2014-07-22 12:30:44.227-0500 I'm going to mark this as regression since the symptom is similar to ASTERISK-13368. By: Rusty Newton (rnewton) 2014-11-05 14:54:56.980-0600 Sorry, I missed your comment. {quote} Hi, sorry for the question, what that mean, is it a fix issue in an other version of asterisk or with a patch or you just identify it as an issue that already get identified by somebody else? What is the next step? {quote} Regression means the behavior of the feature has regressed from a working/desirable state to a non-working/undesirable state. Whereas previously it appeared to be working, now it is not. The next step is to wait for a developer to take on the issue. I don't have an ETA for you, but the issue is open and available for any Asterisk developer to take on. If the issue is very urgent you might consider hiring a developer to fix it for you, or posting a bug bounty on the asterisk-dev list. [Here is a link to info on bug bounties.|https://wiki.asterisk.org/wiki/display/AST/Asterisk+Bug+Bounties] By: Louis Jocelyn Paquet (ljpaquet) 2014-12-15 07:29:50.553-0600 Hi, Rusty, If Asterisk want to take in charge directly this issue, i will pay you for the develpoment. In this case, I like to know if this is possible and I will appreciate if you can let me know the cost of getting this module updated. If this is not possible, I will look to find out a developper (I already start to look with some local guys. I will also try the bug bounties as you propose. I I found a local developper to fix this, I will look forward to send you the fix version to ensure everybody can get the benifit of this fix. Thank you Louis | ||||