[Home]

Summary:ASTERISK-26583: res_pjsip: When using transports in MariaDB table via ODBC; traffic from Asterisk is not seen on the network.
Reporter:Jeff Clay (Jeff_C)Labels:
Date Opened:2016-11-11 11:04:07.000-0600Date Closed:
Priority:MinorRegression?
Status:Open/NewComponents:Core/Sorcery Resources/res_pjsip
Versions:13.9.1 13.12.0 13.11.1 13.12.2 14.1.1 Frequency of
Occurrence		Constant
Related
Issues:
Environment:Centos 7.2.1511 kernel 3.10.0-327.36.3.el7Attachments:( 0) ast_2016.sh
( 1) extconfig_conf.txt
( 2) pjsip_cli_debug.txt
( 3) ps_endpoints.txt
( 4) tcpdump.txt
Description:When using pjsip transports in mariadb table via odbc traffic from asterisk is not seen on the network. For example, when a phone attempts to register, the registration request is seen in pjsip debug as well as a tcpdump; then when asterisk replies with the 401 auth challenge that is only seen in the pjsip debug and is not seen in a tcpdump and the endpoint never receives the challenge.

Using transports within pjsip.conf works properly with the exact same transport settings that were failing when used in database.

Full configs, db tables, tcpdump and debug logs below.

*config files*
{code:title=sorcery.conf}
[res_pjsip]
endpoint=realtime,ps_endpoints
auth=realtime,ps_auths
transport=realtime,ps_transports
aor=realtime,ps_aors
domain_alias=realtime,ps_domain_aliases

[res_pjsip_endpoint_identifier_ip]
identify=realtime,ps_endpoint_id_ips
system=realtime,ps_systems
{code}

[Edit by Rusty - Moved extconfig to attachments]

*database tables*
{code:title=ps_aors}
INSERT INTO `ps_aors` (`id`, `contact`, `default_expiration`, `mailboxes`, `max_contacts`, `minimum_expiration`, `remove_existing`, `qualify_frequency`, `authenticate_qualify`, `maximum_expiration`, `outbound_proxy`, `support_path`, `qualify_timeout`, `voicemail_extension`)
VALUES
('1506', NULL, NULL, NULL, 3, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL);
{code}

{code:title=ps_auths}
INSERT INTO `ps_auths` (`id`, `auth_type`, `nonce_lifetime`, `md5_cred`, `password`, `realm`, `username`)
VALUES
('phone', 'userpass', NULL, NULL, 'pass', NULL, '1506');
{code}

[Edit by Rusty - Moved ps_endpoints data to attachments]

{code:title=ps_transports}
INSERT INTO `ps_transports` (`id`, `async_operations`, `bind`, `ca_list_file`, `cert_file`, `cipher`, `domain`, `external_media_address`, `external_signaling_address`, `external_signaling_port`, `method`, `local_net`, `password`, `priv_key_file`, `protocol`, `require_client_cert`, `verify_client`, `verify_server`, `tos`, `cos`, `allow_reload`)
VALUES
('nat_udp', NULL, '0.0.0.0', NULL, NULL, NULL, NULL, '104.197.69.7', '104.197.69.7', NULL, NULL, '10.128.0.0/20', NULL, NULL, 'udp', NULL, NULL, NULL, NULL, NULL, 'yes');
{code}


*DEBUG*

You can see here that asterisk receives the register request and sends the auth challenge.

[Edit by Rusty - moved to attachments]

And you can see here that the auth challenge sent by Asterisk is never actually seen on the network.

[Edit by Rusty - moved to attachments]
Comments:By: Asterisk Team (asteriskteam) 2016-11-11 11:04:07.978-0600

Thanks for creating a report! The issue has entered the triage process. That means the issue will wait in this status until a Bug Marshal has an opportunity to review the issue. Once the issue has been reviewed you will receive comments regarding the next steps towards resolution.

A good first step is for you to review the [Asterisk Issue Guidelines|https://wiki.asterisk.org/wiki/display/AST/Asterisk+Issue+Guidelines] if you haven't already. The guidelines detail what is expected from an Asterisk issue report.

Then, if you are submitting a patch, please review the [Patch Contribution Process|https://wiki.asterisk.org/wiki/display/AST/Patch+Contribution+Process].
By: Rusty Newton (rnewton) 2016-11-13 09:10:35.914-0600

[~Jeff_C] thanks for this report.

Two things that would help us track this down further:

1. Within one of the branches you tried, can you go back a few minor versions to see if the behavior changed at some point within the past six months?

If this is affecting many people we should have seen it before now, unless this is a regression.

2. Can you try configuring the database with Alembic to see if you get different results? As that is likely how we would set it up for testing.

https://wiki.asterisk.org/wiki/display/AST/Managing+Realtime+Databases+with+Alembic



By: Jeff Clay (Jeff_C) 2016-11-14 14:25:42.594-0600

I first added a new db and created the tables using alembic then tested with 13.12.2 and the issue persisted.

Using the same db created with alembic and the same configuration files, I tried versions 13.12.0, 13.11.1 and 13.9.1.

I switched between versions by doing the following:
{code}
rm -rf /usr/lib64/asterisk/modules
./configure --libdir=/usr/lib64 --with-pjproject-bundled
make menuselect.makeopts
menuselect/menuselect --enable streamplayer --enable stereorize --enable smsq --enable astman --enable aelparse --enable app_alarmreceiver --enable app_amd --enable app_chanisavail --enable res_fax --enable res_fax_spandsp --enable app_dictate --enable app_festival --enable app_image --enable app_morsecode --enable app_sms --enable app_url --enable app_waitforring --enable app_waitforsilence --enable app_ices --enable res_mwi_external --enable cdr_pgsql --enable cdr_csv --enable cdr_odbc --enable chan_alsa --enable chan_console --enable chan_mgcp --enable chan_oss --enable chan_phone --enable chan_sip --enable chan_skinny --enable chan_unistim --enable format_vox --enable pbx_realtime --enable pbx_lua --enable pbx_dundi --enable pbx_ael --enable res_mwi_external --enable odbc_storage  --enable res_mwi_external_ami --enable chan_mobile --enable chan_ooh323 --enable format_mp3 --enable app_meetme --enable res_config_mysql --enable res_corosync --enable res_endpoint_stats --enable ODBC_STORAGE --enable aelparse --enable astman --enable check_expr --enable check_expr2 --enable conf2ael --enable muted --enable smsq --enable stereorize --enable streamplayer --enable CORE-SOUNDS-EN-WAV --enable CORE-SOUNDS-EN-ULAW --enable CORE-SOUNDS-EN-ALAW --enable CORE-SOUNDS-EN-G729 --enable CORE-SOUNDS-EN-G722 --enable CORE-SOUNDS-EN-SLN16 --enable CORE-SOUNDS-EN-SIREN7 --enable CORE-SOUNDS-EN-SIREN14 --enable MOH-OPSOUND-ULAW --enable MOH-OPSOUND-ALAW --enable MOH-OPSOUND-GSM --enable MOH-OPSOUND-G729 --enable MOH-OPSOUND-G722 --enable MOH-OPSOUND-SLN16 --enable MOH-OPSOUND-SIREN7 --enable MOH-OPSOUND-SIREN14 --enable EXTRA-SOUNDS-EN-WAV --enable EXTRA-SOUNDS-EN-ULAW --enable EXTRA-SOUNDS-EN-ALAW --enable EXTRA-SOUNDS-EN-GSM --enable EXTRA-SOUNDS-EN-G729 --enable EXTRA-SOUNDS-EN-G722 --enable EXTRA-SOUNDS-EN-SLN16 --enable EXTRA-SOUNDS-EN-SIREN7 --enable EXTRA-SOUNDS-EN-SIREN14
./contrib/scripts/get_mp3_source.sh
make
make install
make config
{code}

*Testing Results*
{code:title=13.12.0}
pjsip show endpoints and show transports crashes *
attempting to register endpoint replicates registration issue
removed transports from extconfig and sorcery then restarted * service with a transport in the flat file.
show transports now shows the flat file transport
show endpoints does not crash *
endpoint registers properly
{code}

{code:title=13.11.1}
pjsip show commands do not crash * as they did with 13.12.0
pjsip show transports properly shows the transport that is in the db.
same registration issue when using realtime transports
commenting out transport lines from extconfig and sorcery, allowing pjsip to load from flat file registers the endpoint fine.
{code}

{code:title=13.9.1}
pjsip show commands do not crash * as they did with 13.12.0
pjsip show transports properly shows the transport that is in the db.
same registration issue when using realtime transports
commenting out transport lines from extconfig and sorcery, allowing pjsip transport to load from flat file registers the endpoint fine.
{code}


By: Rusty Newton (rnewton) 2016-11-15 14:20:48.959-0600

Interesting, thanks. So it looks like the disappearing registration issue has been around for a while. I'm not sure why we haven't seen more reports for it. Perhaps there is something very specific to your environment. Hopefully we'll find out.

I see also that you found crashes that occur during reproduction of the issue described on this ticket.

Can you file a new issue for the crashes, and attach the backtraces of the crashes to that new issue? We can link the issues as related, but it will be efficient to track the crash issue separately from the disappearing traffic problem.

Do the crashes occur in 13.12.2 ? Please note that on the new crash ticket.

To help clean up this ticket I'm going to move some of your debug from the description field to attached files. As updates to this issue are generating very lengthy E-mails.
By: Rusty Newton (rnewton) 2016-11-15 14:30:12.402-0600

Attaching various debug removed from description field - per the guidelines to keep E-mail notifications from being bonkers large.
By: Jeff Clay (Jeff_C) 2016-11-16 11:09:12.079-0600

Here is the build script I made for my installation. I have reproduced my issue many times by just installing Centos 7 64bit in an ESXi or Google Cloud VM and then running this script. I'm providing it in hopes of helping anyone testing to completely reproduce the exact system with configs that is causing the issue or maybe someone can point out something wrong in the way I'm doing things. The only thing that you will need to change once the script completes is the local_net value in ps_transports.
By: Ilgiz Badamshin (i.badamshin) 2018-01-22 07:24:49.534-0600

Hi.
I have same problems with asterisk 14.7.2, 14.7.5 behind nat.
Bug with external_media_address, external_signaling_address, external_signaling_port.
If transport settings set in pjsip.conf all ok.
When settings moved in to database (pgsql), then responses from asterisk are losts.
Example with failed registration.
1. Asterisk received "Register" from softphone (twinkle)
2. Asterisk transmitting "Unauthorized" - visible in console with pjs sip debug, but not in tcpdump, wireshark and no handled by softphone.
3. Softphone send "Register" again, because no responses to previos request.
4. Asterisk transmitting "Unauthorized" and it lost again.

Tryed to digging.
File: res_pjsip_nat.c
Method: {code} static int find_transport_state_in_use(void *obj, void *arg, int flags)  {code}
Return 0, so method nat_on_tx_message return with PJ_SUCCESS, but without sending response to net:
{code}
if (!(transport_state = ao2_callback(transport_states, 0, find_transport_state_in_use, &details))) {
return PJ_SUCCESS;
}
{code}

Is it will be fixed or i should not use transport in realtime?
Thanks.

By: Joshua C. Colp (jcolp) 2018-01-22 07:36:11.084-0600

There is noone actively working on this issue, as a result configuring transports in the configuration file is the best option.