Summary: | ASTERISK-24478: IAX hits maxcallnumber limit of 2048 with only one concurrent channel | ||||
Reporter: | POUILLY GUILLAUME (softattitude) | Labels: | |||
Date Opened: | 2014-11-01 11:11:07 | Date Closed: | 2018-01-02 08:30:31.000-0600 | ||
Priority: | Critical | Regression? | |||
Status: | Closed/Complete | Components: | Channels/chan_iax2 | ||
Versions: | 12.6.1 | Frequency of Occurrence | Constant | ||
Related Issues: |
| ||||
Environment: | Linux autodial 3.2.0-4-amd64 #1 SMP Debian 3.2.57-3+deb7u2 x86_64 GNU/Linux | Attachments: | ( 0) dead_astobj2.txt ( 1) dead_callnumber.txt ( 2) dead_debug.log ( 3) dead_show_locks.txt ( 4) maxcallnumber_count.log.bz2 ( 5) maxcallnumber_full.log.bz2 | ||
Description: | [Edit by Rusty - Even if we can't track down the issue here, we should improve iax.conf to mention the 2048 default limit and describe the max values for maxcallnumbers and maxcallnumbers_nonvalidated]
Hi, With ARI, I create channels to make outbounds call via a IAX Trunk. After some time, i have on the asterisk cli : [Nov 1 16:14:46] ERROR[5236][C-00005077]: chan_iax2.c:2588 peercnt_add: maxcallnumber limit of 2048 for 10.100.5.1 has been reached! [Nov 1 16:14:46] WARNING[5236][C-00005077]: chan_iax2.c:12400 iax2_request: Unable to create call [Nov 1 16:14:46] WARNING[5236][C-00005077]: app_dial.c:2421 dial_exec_full: Unable to create channel of type 'IAX2' (cause 34 - Circuit/channel congestion) There is no maxcallnumber, on each side of the iax Trunk. core show channels: Channel Location State Application(Data) IAX2/autodial_b-2918 (None) Ringing AppDial((Outgoing Line)) Local/0964158794@REA (None) Ring Dial(IAX2/autodial_b/096415879 1 active channel 1 active call 20672 calls processed The active call does not exist in reality module show like iax; Module Description Use Count Status chan_iax2.so Inter Asterisk eXchange (Ver 2) 1 Running 1 modules loaded the asterisk cli stop working and i have to lauch it again. | ||||
Comments: | By: Rusty Newton (rnewton) 2014-11-05 15:55:14.751-0600 Please [attach a debug log|https://wiki.asterisk.org/wiki/display/AST/Collecting+Debug+Information] demonstrating the point at which the call limit is reached. Be sure DEBUG messages show in the log. Please also attach the iax.conf for each system and asterisk.conf. Be sure to scrub your private information. By: Rusty Newton (rnewton) 2014-11-20 12:16:38.644-0600 [~softattitude] without your debug we may not be able to investigate this issue. We don't have enough information to reproduce the issue and no one else is running into it. That being said, even if you can't provide the requested information - we should improve iax.conf to mention the 2048 default limit and describe the max values for maxcallnumbers and maxcallnumbers_nonvalidated. By: Y Ateya (yateya) 2015-01-01 14:50:48.494-0600 I saw the same problem and I was able to reproduce it. My setup is as follows: SIP1 <> IAX Trunk <> SIP2 <> GSM gateway Hint: Calls always flow from SIP1 (always the caller) to SIP2 (always the callee). The problem occurs when there is a failure in GSM gateway; which leads to rejecting many of SIP calls. Here is what should happen for increment, schedule decrement and actual decrement of the call number counter: - call comes - increment counter - call rejected - schedule counter decrement after 60 seconds and this pattern will be repeated until GSM return to normal operation. So after 60 seconds of GSM-back-to-life; all calls will continue as usual. But this DON'T happen. What happens is: - increment, schedule decrement, after 60 seconds decrement event occurs. - increment, schedule decrement, after 60 seconds decrement event occurs. . . - increment, schedule decrement, after 60 seconds decrement event occurs. =======And after a while (~15 minutes in my case), decrement event stops! - increment, schedule decrement - increment, schedule decrement - increment, schedule decrement Decrement don't happen even after I stopped SIP1 calls. And the result is that IAX peers have no active calls and max call number is 2048. Raising max call number doesn't solve the problem; Neither decreasing MIN_REUSE_TIME. The only solution then is to restart asterisk; reload didn't affect call number count. The attached log is from Jan 1 18:12:13 to Jan 1 18:32:46. You will that the latest decrement "ip callno count decremented" was at Jan 1 18:23:32. For TEN minutes, scheduled decrements didn't happen. Attachments: maxcallnumber_count.log.bz2: Extracted counter-related messages from debug log. maxcallnumber_full.log.bz2: The full log Environment: - Running Asterisk 13.1.0 compiled from sources and using PJPROJECT. - Ubuntu 12.04 x86_64 By: Y Ateya (yateya) 2015-01-01 14:53:10.804-0600 Attachments: maxcallnumber_count.log.bz2: Extracted counter-related messages from debug log. maxcallnumber_full.log.bz2: The full log By: Y Ateya (yateya) 2015-01-20 13:00:59.588-0600 While trying to reproduce the bug I found a LOT of deadlocks messages. Attached dead_astobj2.txt: Result of astobj2 commands dead_show_locks.txt: Result of core show locks dead_debug.txt: The interesting part from the debug log. dead_callnumber.txt: Result of iax2 show callnumber usage It seems that it revolves around some dead locks in iax2_hangup and transmit_frame. By: Y Ateya (yateya) 2015-04-21 14:59:58.597-0500 This bug is result of ASTERISK-24983. When the scheduler is locked (due to the deadlock), calls are not destroyed (schedule_destroy queued in scheduler but not executed). This leads to maximum call number. By: Matthias Urlichs (smurfix) 2017-05-18 04:53:13.236-0500 There may be another cause for this bug. I just saw it on one of my slave systems. Asterisk 13.2. By: Joshua C. Colp (jcolp) 2017-12-18 09:23:43.618-0600 I noticed your comment that this was caused by another issue. Does this mean it was also fixed by it? By: Asterisk Team (asteriskteam) 2018-01-02 08:30:31.513-0600 Suspended due to lack of activity. This issue will be automatically re-opened if the reporter posts a comment. If you are not the reporter and would like this re-opened please create a new issue instead. If the new issue is related to this one a link will be created during the triage process. Further information on issue tracker usage can be found in the Asterisk Issue Guidlines [1]. [1] https://wiki.asterisk.org/wiki/display/AST/Asterisk+Issue+Guidelines |