[Home]

Summary:ASTERISK-27148: res_pjsip: Transports recreated wrongly, tls connections drop when config changes
Reporter:Tony Guadagno (tonyg)Labels:pjsip
Date Opened:2017-07-21 09:43:11Date Closed:
Priority:MajorRegression?
Status:Open/NewComponents:Resources/res_pjsip
Versions:13.17.0 Frequency of
Occurrence		Constant
Related
Issues:
Environment:freepbx 14.0.1.1 pjsip 2.6 asterisk 13.17Attachments:( 0) core_reload_after_touch.txt
( 1) core_reload_before_touch.txt
Description:hi, i am having an issue i hope you can help me with. I am running freepbx 14.0.1.1 with asterisk 13.17 and pjsip 2.6. most of my endpoints are connected pjsip sip-tls and srtp and they drop connection when config changes are applied.
. when i first installed the system, things were great, but about 2 weeks ago, the system started dropping all tls connections when i applied config changes. initially, i thought this was a freepbx issue, but after much investigation, i am sure now, it is not. i can simply and reliable reproduce the issue this way.
initially, i run asterisk -rx 'core reload'....asterisk reloads and there is no issue with endpoints BUT if there are any config changes in /etc/asterisk and i do a core reload, all sip-tls connected endpoints drop. none of the sip/udp connected endpoints are affected.
to take the test one step further, i tried running touch /etc/asterisk/*...i made NO changes to the file contents, just touched them and ran a core reload and connections dropped.

https://community.asterisk.org/t/touching-etc-asterisk-config-files-causes-pjsip-sip-tls-endpoints-to-drop/71433
Comments:By: Asterisk Team (asteriskteam) 2017-07-21 09:43:12.565-0500

Thanks for creating a report! The issue has entered the triage process. That means the issue will wait in this status until a Bug Marshal has an opportunity to review the issue. Once the issue has been reviewed you will receive comments regarding the next steps towards resolution.

A good first step is for you to review the [Asterisk Issue Guidelines|https://wiki.asterisk.org/wiki/display/AST/Asterisk+Issue+Guidelines] if you haven't already. The guidelines detail what is expected from an Asterisk issue report.

Then, if you are submitting a patch, please review the [Patch Contribution Process|https://wiki.asterisk.org/wiki/display/AST/Patch+Contribution+Process].
By: Tony Guadagno (tonyg) 2017-07-21 09:48:37.792-0500

logs
By: George Joseph (gjoseph) 2017-07-21 11:11:22.011-0500

I've been trying to reproduce this and can't seem to.

In stock asterisk, "core reload" doesn't touch any pjsip modules but even if I do "module reload res_pjsip", it won't actually do anything unless one of the objects defined in pjsip.conf is actually changed.  Just touching the file does nothing.  I even tested changing my tls transport (defined with allow_reload=yes) and reloading and in progress tls/srtp calls did NOT drop.  They shouldn't anyway because when we reload a tcp/tls transport, we're only reloading the listener socket, not the sockets in use.

I'm going to try and follow up with the FreePBX team to see if they do something special on a "core reload".

By: George Joseph (gjoseph) 2017-07-21 11:12:05.104-0500


[~tm1000] :  Andrew, Does FreePBX do special things when a "core reload" CLI command is issued?


By: Tony Guadagno (tonyg) 2017-07-21 11:14:50.464-0500

George, i should have been more clear, in working with jcolp (https://community.asterisk.org/t/touching-etc-asterisk-config-files-causes-pjsip-sip-tls-endpoints-to-drop/71433/5) he suggested that the issue was caused when many extensions were defined.  indeed,  i did not see this issue until i loaded 80+ extensions in.  please take a look at that thread.

thanks

By: Andrew Nagy (tm1000) 2017-07-21 12:38:38.960-0500

[~gjoseph], on rare occasions we generate hints. That is all we do.
By: George Joseph (gjoseph) 2017-07-24 14:10:45.641-0500

Thanks [~tm1000]!

[~tonyg]:  Yep, It was Josh who alerted me to this issue and I read the thread.  I just created 1000 extensions with hints and still nothing.  "core reload" simply doesn't touch anything in pjsip-land but clearly your logs do show a res_pjsip reload.   Can you reproduce this issue on a stock asterisk system with a set of config files you can attach to this issue?  

By: Tony Guadagno (tonyg) 2017-07-24 14:20:30.467-0500

George, i will try, it might take a while, is there distro you would like me to use?  since i would be building from scratch, it does not matter to me.

thanks

By: George Joseph (gjoseph) 2017-07-24 15:13:30.549-0500

CentOS 7 is kinda our debugging goto distro.  Another alternative would be to package up your entire config so we can try it but we're kind of flying blind when it comes to FreePBX.  Have you touched base with them officially at all?  Might be a good idea to open an issue with them as well then maybe we can figure it out together.  At least they may be able to help extract a config we can try.


By: Tony Guadagno (tonyg) 2017-07-24 15:17:20.619-0500

george...open a ticket with freepbx....lol, your funny! :)  

https://community.freepbx.org/t/clicking-on-apply-causes-pjsip-sip-tls-endpoints-to-drop/42756/59

I would be happy to bundle up my config, can i send them to you privately? how?

thanks

By: George Joseph (gjoseph) 2017-07-25 11:43:23.481-0500

Tar up your entire /etc/asterisk directory and send it to me at
gjoseph AT digium DOT com

By: Tony Guadagno (tonyg) 2017-07-27 08:38:43.201-0500

george, i send that out a couple of days ago, please let me know if you have not received it.
thanks

By: Tony Guadagno (tonyg) 2017-08-11 13:57:51.933-0500

george, please let me know if you need anything else on this, its had using our system right now because i have to batch all my changes on the weekends.

thanks

By: George Joseph (gjoseph) 2017-08-14 08:46:09.869-0500

Sorry.  I'll try and look this afternoon.

By: Tony Guadagno (tonyg) 2017-08-15 18:03:38.219-0500

george, sorry to be a pest, just wondering what you found on this.

thanks

By: George Joseph (gjoseph) 2017-08-17 07:33:49.680-0500

I've got time set aside today.

By: Tony Guadagno (tonyg) 2017-08-18 07:22:23.001-0500

george, how did it go yesterday?  did you figure out whats going on?

thanks
By: Joshua C. Colp (jcolp) 2017-08-18 07:35:43.378-0500

[~tonyg] George is quite busy (as we all are) and is trying to fit in reproducing this issue so he can triage it as soon as he can. If he has any comments or suggestions he'll comment on here. You don't need to continue asking for an update.
By: George Joseph (gjoseph) 2017-10-31 16:38:35.533-0500

Tony, I'm closing this issue now.  We just don't have the time to try and reproduce ourselves.  If you can reproduce the issue with a stock asterisk setup, feel free to re-open.

By: Matthias Binder (mitterhuemer) 2017-12-27 04:23:50.880-0600

Hey there.
I did a TestPBX Setup with FreePBX SNG 1712 today. Asterisk 13.18.4

All endpoints with PJSIP -> TLS -> SRTP (Letsencrypt) are unavailable after a reload.
The tested endpoints are Sangoma Phones (S500, S700) and Yealink Phones (T48S, T56A)

fwconsole reload... and all PJSIP phones are unavailable. But they can still make a echo test (*43).

After a reboot of the PBX alle phones are available again.
By: Tony Guadagno (tonyg) 2017-12-28 07:40:31.333-0600

Matthias, thanks for your interest, I am desperate to get this fixed.  are you a developer or a user like me?  whats your interest in this issue?

thanks
By: Asterisk Team (asteriskteam) 2017-12-28 07:40:31.932-0600

This issue has been reopened as a result of your commenting on it as the reporter. It will be triaged once again as applicable.
By: Matthias Binder (mitterhuemer) 2017-12-28 07:49:07.995-0600

@tonyg
Our company is Sangoma Partner. We sell cloud based FreePBX and PBXAct UC Systems and integrate them directly by our customers.

I know FreePBX well and also do some developing on existing modules like Superfecta for our Region.

We love open Source and invest some of our time into testing new Phones and technologies like PJSIP.

My knowledge of the asterisk itself is not soo good. So i dont know how how i can help with that issue except testing and hoping that someone has an idea why this is happening.


By: Tony Guadagno (tonyg) 2017-12-28 09:23:36.097-0600

matthias, too bad, we are desperate to have this fixed, we are willing to pay.

thanks

By: Matthias Binder (mitterhuemer) 2017-12-28 09:32:15.853-0600

Why not using Chan_SIP until this will be fixed?

I think they have to fix it when chan_sip will not be supported anymore later.
By: Joshua C. Colp (jcolp) 2018-01-02 07:25:20.804-0600

Using the provided configuration files I wasn't able to reproduce the dropping of the TLS connections but it did uncover two things which may contribute:

1. PJSIP doesn't support binding multiple TCP or TLS transports currently, so one is being ignored.
2. The working transports are not being ignored due to no changes, the code is incorrectly thinking that the configuration has changed and destroying/recreating them - this may be the cause of the connections dropping but I haven't been able to reproduce that specifically.

Since there are issues I'm opening this up.
By: Tony Guadagno (tonyg) 2018-01-02 08:53:39.698-0600

josh thanks for taking the time on this, if you need anything from me, please let me know.  this reproduces for me every time.

thanks
By: Leo Mirell (tmirell) 2021-11-05 02:14:28.989-0500

Hi,

I'm having the same problem and have been for a long time.
I've installed the latest version of asterisk 16.21.1 with pjsip 2.10.
When a do a core reload or pjsip reload all tls encrypted phones get unregistered and lose ability to receive incoming calls, but udp phones are not affected at all..
I can reproduce the problem anytime.
In order to have my phones reconnect, I'll either need to wait a while for the phones to reregister, or restart asterisk.
I've worked around this issue for a while by using asterisk realtime, but when configuring or editing trunks which need to register to sip providers, as far as I'm aware, I need to keep those configurations in pjsip.conf and cannot use realtime.
In this case, I have no choice but to reload pjsip , which then knocks out my phones.

Thanks,
Any help would be greatly appreciated.
Leo