[Home]

Summary:ASTERISK-25871: Asterisk deadlock when using confbridge
Reporter:Val (terramda)Labels:
Date Opened:2016-03-29 11:02:36Date Closed:2016-03-29 13:10:46
Priority:MajorRegression?
Status:Closed/CompleteComponents:Channels/chan_sip/General Resources/res_rtp_asterisk
Versions:13.7.2 Frequency of
Occurrence
Constant
Related
Issues:
duplicatesASTERISK-25275 A11 SIGSEGV from pjnpath check_cached_response (ast_rtcp_read -> pj_stun_session_on_rx_pkt)
Environment:CPU Family:Intel Core i3 Sandy Bridge Operating System: Linux Ubuntu 14.04 LTS Architecture: 64-bit RAM: 8 GB Disk Mode: Standalone (No RAID, 1-4 Disks) RAID Mode: None (2-4 Disks) 1st Disk: 500 GB HDDAttachments:( 0) backtrace.txt
( 1) core-show-locks.txt
Description:We use asterisk to mix WEBRTC streams and encounter random deadlocks.
When there are 2 or more participants in a conference (the number is not relevant, the deadlock may occur when there are only 2 participants and work fine when there are 140, and sometimes vice versa) after a random time 1 CPU goes 100% and all calls are dropped.

The console doesn't show something special:
{noformat}
[Mar 29 18:15:33]DEBUG[15106]:acl.c:958 ast_find_ourip: Not an IPv4 nor IPv6 address, cannot get port.
[Mar 29 18:15:33]DEBUG[15106]:acl.c:958 ast_find_ourip: Not an IPv4 nor IPv6 address, cannot get port.
[Mar 29 18:15:33]DEBUG[15106]:acl.c:958 ast_find_ourip: Not an IPv4 nor IPv6 address, cannot get port.
[Mar 29 18:15:38]DEBUG[15106]:acl.c:958 ast_find_ourip: Not an IPv4 nor IPv6 address, cannot get port.
[Mar 29 18:15:38]DEBUG[15106]:acl.c:958 ast_find_ourip: Not an IPv4 nor IPv6 address, cannot get port.
[Mar 29 18:15:38]DEBUG[15106]:acl.c:958 ast_find_ourip: Not an IPv4 nor IPv6 address, cannot get port.
[Mar 29 18:15:38]DEBUG[15106]:acl.c:958 ast_find_ourip: Not an IPv4 nor IPv6 address, cannot get port.
[Mar 29 18:15:39]DEBUG[15106]:chan_sip.c:8783 __sip_alloc: Allocating new SIP dialog for 4282459-26fd40e6-56fb7d75@178.175.129.194 - REGISTER (No RTP)
[Mar 29 18:15:39]DEBUG[15106]:chan_sip.c:3715 __sip_xmit: Trying to put 'SIP/2.0 401' onto UDP socket destined for 69.30.232.245:5082
[Mar 29 18:15:39]NOTICE[15106]:chan_sip.c:27928 handle_request_register:Registration from '"3009" <sip:3009@178.175.129.194:5060>' failed for '69.30.232.245:5082' - Not a local domain
[Mar 29 18:15:39]DEBUG[15106]:res_config_mysql.c:1589 mysql_reconnect: MySQL RealTime: Connection okay.
[Mar 29 18:15:39]DEBUG[15106]:res_config_mysql.c:377 realtime_mysql: MySQL RealTime: Retrieve SQL: SELECT * FROM sipfriends WHERE name = '' AND host = 'dynamic'
[Mar 29 18:15:39]DEBUG[15106]:res_config_mysql.c:1589 mysql_reconnect: MySQL RealTime: Connection okay.
[Mar 29 18:15:39]DEBUG[15106]:res_config_mysql.c:377 realtime_mysql: MySQL RealTime: Retrieve SQL: SELECT * FROM sipfriends WHERE name = ''
[Mar 29 18:15:40]DEBUG[15106]:chan_sip.c:3715 __sip_xmit: Trying to put 'SIP/2.0 403' onto UDP socket destined for 69.30.232.245:5082
[Mar 29 18:15:40]NOTICE[15106]:chan_sip.c:27928 handle_request_register: Registration from '"3009" <sip:3009@178.175.129.194:5060>' failed for '69.30.232.245:5082' - Not a local domain
[Mar 29 18:15:40]DEBUG[15106]:res_config_mysql.c:1589 mysql_reconnect: MySQL RealTime: Connection okay.
[Mar 29 18:15:40]DEBUG[15106]:res_config_mysql.c:377 realtime_mysql: MySQL RealTime: Retrieve SQL: SELECT * FROM sipfriends WHERE name = '' AND host = 'dynamic'
[Mar 29 18:15:40]DEBUG[15106]:res_config_mysql.c:1589 mysql_reconnect: MySQL RealTime: Connection okay.
[Mar 29 18:15:40]DEBUG[15106]:res_config_mysql.c:377 realtime_mysql: MySQL RealTime: Retrieve SQL: SELECT * FROM sipfriends WHERE name = ''
[Mar 29 18:15:43]DEBUG[15106]:acl.c:958 ast_find_ourip: Not an IPv4 nor IPv6 address, cannot get port.
[Mar 29 18:15:43]DEBUG[15106]:acl.c:958 ast_find_ourip: Not an IPv4 nor IPv6 address, cannot get port.
[Mar 29 18:15:43]DEBUG[15106]:acl.c:958 ast_find_ourip: Not an IPv4 nor IPv6 address, cannot get port.
[Mar 29 18:15:43]DEBUG[15106]:acl.c:958 ast_find_ourip: Not an IPv4 nor IPv6 address, cannot get port.
[Mar 29 18:15:46]DEBUG[15061]:manager.c:6138 process_message: Running action 'Ping'
[Mar 29 18:15:48]DEBUG[15106]:acl.c:958 ast_find_ourip: Not an IPv4 nor IPv6 address, cannot get port.
[Mar 29 18:15:48]DEBUG[15106]:acl.c:958 ast_find_ourip: Not an IPv4 nor IPv6 address, cannot get port.
[Mar 29 18:15:48]DEBUG[15106]:acl.c:958 ast_find_ourip: Not an IPv4 nor IPv6 address, cannot get port.
[Mar 29 18:15:48]DEBUG[15106]:acl.c:958 ast_find_ourip: Not an IPv4 nor IPv6 address, cannot get port.
[Mar 29 18:16:16]DEBUG[15061]:manager.c:6138 process_message: Running action 'Ping'
[Mar 29 18:16:46]DEBUG[15061]:manager.c:6138 process_message: Running action 'Ping'
{noformat}

by the way we get a lot of these:
{noformat}
[Mar 29 18:15:33] DEBUG[15106]: acl.c:958 ast_find_ourip: Not an IPv4 nor IPv6 address, cannot get port.
{noformat}

We have compiled our asterisk with DEBUG_THREADS and DONT_OPTIMIZE.

Please see core-show-locks.txt and backtrace.txt
Comments:By: Asterisk Team (asteriskteam) 2016-03-29 11:02:37.517-0500

Thanks for creating a report! The issue has entered the triage process. That means the issue will wait in this status until a Bug Marshal has an opportunity to review the issue. Once the issue has been reviewed you will receive comments regarding the next steps towards resolution.

A good first step is for you to review the [Asterisk Issue Guidelines|https://wiki.asterisk.org/wiki/display/AST/Asterisk+Issue+Guidelines] if you haven't already. The guidelines detail what is expected from an Asterisk issue report.

Then, if you are submitting a patch, please review the [Patch Contribution Process|https://wiki.asterisk.org/wiki/display/AST/Patch+Contribution+Process].

By: Joshua C. Colp (jcolp) 2016-03-29 11:18:18.664-0500

This is the same issue as ASTERISK-25275, just in this case the memory is such that it is blocked instead.

I'd suggest upgrading to the latest pjproject from its SVN, changes have gone in which may fix this issue.

By: Val (terramda) 2016-03-29 11:21:11.756-0500

Will try and report sir! Many thanks.

By: Asterisk Team (asteriskteam) 2016-03-29 11:21:11.915-0500

This issue has been reopened as a result of your commenting on it as the reporter. It will be triaged once again as applicable.

By: Val (terramda) 2016-03-29 12:52:28.106-0500

So I got the latest pjproject from SVN.
Steps I did:
cd pjproject
./configure --enable-shared --with-external-speex --with-external-gsm --with-external-srtp --disable-sound --disable-resample && make && make install
/sbin/ldconfig

cd /usr/src/asterisk-13.7.2
./bootstrap.sh
./configure --with-crypto --with-ssl --with-srtp --prefix=/usr
make menuselect # made sure all pj_* are selected
make && make install

restarted asterisk

after 10-15 minutes I got a DEADLOCK (2 users in conference)

Am I doing something wrong?

By: Asterisk Team (asteriskteam) 2016-03-29 12:52:28.317-0500

This issue has been reopened as a result of your commenting on it as the reporter. It will be triaged once again as applicable.

By: Joshua C. Colp (jcolp) 2016-03-29 12:55:27.735-0500

That should be correct, depending on where your pjproject was previously installed. As well I have not tested the latest pjproject for solving this issue, I merely saw commits in the area.

By: Val (terramda) 2016-03-29 13:05:59.710-0500

So... now we wait for ASTERISK-25275 resolution?

By: Asterisk Team (asteriskteam) 2016-03-29 13:05:59.808-0500

This issue has been reopened as a result of your commenting on it as the reporter. It will be triaged once again as applicable.

By: Joshua C. Colp (jcolp) 2016-03-29 13:10:46.882-0500

Yes, that is the ticket tracking it.