| Summary: | ASTERISK-27334: Pjsip Performance issues with qualify enabled | ||||||
| Reporter: | VeeamIT (devox) | Labels: | |||||
| Date Opened: | 2017-10-12 10:19:40 | Date Closed: | 2017-10-12 10:38:23 | ||||
| Priority: | Major | Regression? | No | ||||
| Status: | Closed/Complete | Components: | pjproject/pjsip | ||||
| Versions: | 13.17.2 | Frequency of Occurrence | Constant | ||||
| Related Issues: |
| ||||||
| Environment: | Linux, Centos 7.2.1511 (Core) , 3.10.0-514.26.2.el7.x86_64 1 CPU, Intel(R) Xeon(R) CPU E5-2630 v3 @ 2/40GHz 1GB RAM VMware Box | Attachments: | |||||
| Description: | I encountered a problem with Asterisk version 13.17.2. I have several Asterisk systems with lots of extensions which work perfectly on chan_sip. Recently we decided to migrate to chan_pjsip as it has many features we would like to use and which are not available in chan_sip, one of them is maxcontacts directive and ability have static contacts along with dynamic ones. I set everything up, first with 1-2 extensions, tested - everything is good. However as soon as I add all of the extensions i need, and it is around 3200, on pjsip reload console freezes, CPU goes to 100% and after pjsip is unresponsive, all extensions went to Unreachable state and phones can’t register, asterisk doesn't respond to any SIP messages and pjsip module seem to be not working at all.
I can see messages like: {code}The ‘subm:endpoint_topic_all-cached- task’ - processor queue reached 500 scheduled tasks.{code} {code}core show taskprocessors{code} shows queue and number in queue increasing under pjsip/default like this: {code}Processor Processed In Queue Max Depth Low water High water app_voicemail 0 0 0 450 500 ast_msg_queue 0 0 0 450 500 CCSS_core 0 0 0 450 500 iax2_transmit 0 0 0 450 500 pjsip/default-0000000b 1708 22675 22674 450 500 pjsip/default-0000000c 1834 22673 22653 450 500 pjsip/default-0000000d 1825 22672 22668 450 500 pjsip/default-0000000e 1777 22668 22665 450 500 pjsip/default-0000000f 1821 22674 22655 450 500 pjsip/default-00000010 1697 22678 22654 450 500 pjsip/default-00000011 1429 22864 22854 450 500 pjsip/default-00000012 1825 22664 22639 450 500 {code} I added 1 more core to this test server to see if it helps but asterisk now eat 200% of CPU and situation has not changed. It is worth mentioning that all of my extensions have static {code} contact=sip:${EXTEN}@IP:5060{code} directive because i need asterisk to dial not only registered IP but sending a call to another server as well. As soon as i lower down number of loaded extensions to, say, 400 it loads, a bit slow but extensions are working fine. So it seems like the problem is in amount of extensions. I tested this configuration and this problem happens on all versions beginning from version 13.8(didn’t test earlier ones) and to 15. I gathered backtrace and “core show locks” information during this problem. Should this be added to asterisk tracker? I have 2 pjsip configuration files. pjsip.conf {code}[global] type=global user_agent=MY PBX regcontext=registerstate keep_alive_interval=0 disable_multi_domain=yes default_from_user=mypbx01 mwi_tps_queue_high=5000 mwi_tps_queue_low=-1 [transport-udp] type=transport protocol=udp bind=MY_IP local_net=MY_NETWORK [transport-tcp] type=transport protocol=tcp bind=My_IP [system] type=system threadpool_max_size=0 #include /etc/asterisk/pjsip_users.conf {code} pjsip_users.conf {code}[sets](!) type=endpoint context=internal dtmf_mode=rfc4733 allow=!all,alaw,g722 inband_progress=no transport=transport-tcp identify_by=username allow_subscribe=yes direct_media=yes direct_media_method=reinvite connected_line_method=update moh_suggest=support rtp_ipv6=no send_pai=yes send_rpid=yes allow_transfer=yes [auth](!) type=auth auth_type=userpass [id](!) type=identify [aor](!) type=aor qualify_frequency=60 max_contacts=3 [5801](sets) from_user=5801 auth=5801 aors=5801 allow_subscribe=yes callerid=Testing account <5801> mailboxes=5801@default [5801](auth) password=secret username=5801 [5801](aor) contact=sip:5801@CUSTOM_IP:5060 [5801](id) endpoint=5801 {code} There are ~3200 extensions added in the same way as 5801. with {code}qualify_frequency=60{code} on {code} reload res_pjsip.so {code} Actual Result: pjsip module freezes and stops responding to commands and SIP messages. Expected Result: smooth reload with working pjsip with {code}qualify_frequency=0{code} on {code} reload res_pjsip.so {code} Actual Result: pjsip module freezes for about a minute or so and then loads up all extensions Expected Result: smooth reload with working pjsip | ||||||
| Comments: | By: Asterisk Team (asteriskteam) 2017-10-12 10:19:41.296-0500 Thanks for creating a report! The issue has entered the triage process. That means the issue will wait in this status until a Bug Marshal has an opportunity to review the issue. Once the issue has been reviewed you will receive comments regarding the next steps towards resolution. A good first step is for you to review the [Asterisk Issue Guidelines|https://wiki.asterisk.org/wiki/display/AST/Asterisk+Issue+Guidelines] if you haven't already. The guidelines detail what is expected from an Asterisk issue report. Then, if you are submitting a patch, please review the [Patch Contribution Process|https://wiki.asterisk.org/wiki/display/AST/Patch+Contribution+Process]. | ||||||