Summary:	ASTERISK-28452: pjsip: <sess-version> of SDP is not incremented though SDP may be changed on reinvite without SDP offer
Reporter:	Michael Maier (micha)	Labels:	patch pjsip
Date Opened:	2019-06-17 14:08:20	Date Closed:	2021-02-25 08:44:32.000-0600
Priority:	Minor	Regression?
Status:	Closed/Complete	Components:	pjproject/pjsip Resources/res_pjsip_sdp_rtp
Versions:	16.3.0	Frequency of Occurrence	Constant
Related Issues:	is duplicated by ASTERISK-29110 res_pjsip_sdp_rtp: Asterisk does not increment session version information in late SDP reinvite scenario
Environment:	CentOS 7	Attachments:	( 0) ASTERISK-28452.diff ( 1) sdp-version.patch ( 2) sdp-version-v2.patch ( 3) uac-reinv-no-sdp.xml.txt ( 4) wrong-SDP-sess-version.txt
Description:	During late / delayed offer as part of a reInivte during an outbound call, the SDP sent to the provider in the 200 OK is changed compared to the initial INVITE - but the <sess-version> isn't incremented. This is against RFC6337, 5.2.5. Subsequent Offers and Answers. See the attached Trace!
Comments:	By: Asterisk Team (asteriskteam) 2019-06-17 14:08:20.941-0500 Thanks for creating a report! The issue has entered the triage process. That means the issue will wait in this status until a Bug Marshal has an opportunity to review the issue. Once the issue has been reviewed you will receive comments regarding the next steps towards resolution. A good first step is for you to review the [Asterisk Issue Guidelines|https://wiki.asterisk.org/wiki/display/AST/Asterisk+Issue+Guidelines] if you haven't already. The guidelines detail what is expected from an Asterisk issue report. Then, if you are submitting a patch, please review the [Patch Contribution Process|https://wiki.asterisk.org/wiki/display/AST/Patch+Contribution+Process]. Please note that once your issue enters an open state it has been accepted. As Asterisk is an open source project there is no guarantee or timeframe on when your issue will be looked into. If you need expedient resolution you will need to find and pay a suitable developer. Asking for an update on your issue will not yield any progress on it and will not result in a response. All updates are posted to the issue when they occur. By: Michael Maier (micha) 2019-06-17 14:11:14.783-0500 SIP packages as shown by asterisk. Please take care of the SDP <sess-version> in the initial INVITE and the 200 OK as answer to the reInvite. By: Joshua C. Colp (jcolp) 2019-06-18 05:27:25.630-0500 I believe the underlying problem is in the PJSIP INVITE session code or the PJMEDIA SDP negotiator. After negotiation the codec list is pruned based on it, but the session version is not updated. As a result when the last local SDP is taken as the offer it keeps the same session version when it shouldn't. By: Michael Maier (micha) 2019-06-18 08:26:10.114-0500 Quick fix: This patch always increments the SDP version before it's sent out. This fixes the problem and calls to 116116 are working now as expected (without this patch, calls are dying - 116116 is an emergency number in Germany). It would be better to check if the version has been already incremented or if it's really necessary (because SDP changed) - but this isn't a failure as such if it's done regardless. By: Michael Maier (micha) 2019-08-10 05:01:25.068-0500 The previously attached quick fix doesn't fix the problem completely, because there is not just the increment broken, but the SDP session handling completely (the SDP session used for the ReInvite is the SDP of the initial INVITE instead of the INVITE + AUTH). See https://www.mail-archive.com/asterisk-dev@lists.digium.com/msg44766.html The sdp-version-v2.patch takes this into consideration and better tries to ensure, that SDP session is always incremented and never decremented by adding timestamp and cseq number. By: Richard Mudgett (rmudgett) 2019-08-12 07:23:52.352-0500 Thanks for the contribution! If you'd like your contribution to be included faster, you should submit your patch for code review by the Asterisk Developer Community. To do so, please follow the Code Review [1] instructions on the wiki. Be sure to: * Verify that your patch conforms to the Coding Guidelines [2] * Review the Code Review Checklist [3] for common items reviewers will look for * If necessary, provide tests for the Asterisk Test Suite that verify the correctness of your patch [4] When ready, submit your patch and any tests to Gerrit [5] for code review. Thanks! [1] https://wiki.asterisk.org/wiki/display/AST/Code+Review [2] https://wiki.asterisk.org/wiki/display/AST/Coding+Guidelines [3] https://wiki.asterisk.org/wiki/display/AST/Code+Review+Checklist [4] https://wiki.asterisk.org/wiki/display/AST/Asterisk+Test+Suite+Documentation [5] https://wiki.asterisk.org/wiki/display/AST/Gerrit+Usage Additionally these patches are not marked as contributions so they do not have the associated release by the license agreement.  Have you signed the license agreement? By: Holger Hans Peter Freyther (zecke) 2020-11-30 05:55:00.481-0600 I dived a bit into pjmedia and some of the glue to asterisk. The first find is this https://github.com/pjsip/pjproject/blob/fe963266b13d708a56c654ac5690de5f9253daeb/pjmedia/src/pjmedia/sdp_neg.c#L411 place to increase the version when the local media changes. pjmedia_sdp_neg_modify_local_offer does not appear to be called in asterisk at all and probably does not need to be. But it means we need to increase the version when we modify the pjmedia (e.g. add or remove content). I have seen a couple of places that this can happen. E.g. when we overwrite IPs for multi-homing, topology changes in the bridge. By: Holger Hans Peter Freyther (zecke) 2021-01-07 08:10:48.070-0600 Going through the code and in the specific testcase it seems an issue of pjmedia. After we receive the answer to our offer pjmedia modifies the media but doesn't increment the version. By: Michael Maier (micha) 2021-01-07 13:05:17.559-0600 The initial outgoing Invite had the following SDP offer: o=- 211936312 211936312 IN IP4 46.17.31.55 m=audio 10008 RTP/AVP 9 8 0 101 The incoming reInvite from the ISP had no SDP, but Asterisk answered on base of the 200 OK SDP from the ISP before with o=- 211936312 211936312 IN IP4 46.17.31.55 m=audio 10008 RTP/AVP 8 101 => This is wrong, because Asterisk changed SDP offer (from RTP/AVP 9 8 0 101 -> RTP/AVP 8 101) w/o changing the session version. That's why the call fails, or better, must fail. My previously attached workarounds unfortunately don't work for each situation (it's pretty bad to try to fix problems based on the problem itself instead of fixing it at the right place). Therefore [Holger Hans Peter Freyther|https://issues.asterisk.org/jira/secure/ViewProfile.jspa?name=zecke], could you please refer the actual code you are suspecting in pjmedia to be fixed? Unfortunately, the 116116 can't be used for testcases any more, because the signaling is meanwhile pretty normal as each other call (at least from Deutsche Telekom MagentaZuhause): -> INVITE / SDP <- TRYING <- OK / SDP => That's it. But it still takes 2 seconds until the call is accepted - obviously those forwards are now hidden and can't be seen any more by the caller. I know of another situation: I'm the caller, too. -> INVITE / SDP m=audio 10058 RTP/SAVP 8 0 101 <- TRYING <- OK / SDP m=audio 10008 RTP/AVP 8 101 Later on (15 minutes later), the callee sends a reInivte as session handling packet. Asterisk answers with <- INVITE / SDP m=audio 56220 RTP/SAVP 8 101 -> OK / SDP m=audio 10058 RTP/AVP 8 101 => This is working fine w/o patch (at least since Asterisk 18). But the reInvite does have a SDP - that's the difference compared to 116116. What a pity that 116116 now behaves as a pretty normal call ... . By: Holger Hans Peter Freyther (zecke) 2021-01-07 19:39:17.123-0600 https://gerrit.asterisk.org/c/asterisk/+/15305 contains the patch against pjmedia. By: Michael Maier (micha) 2021-01-07 20:28:41.451-0600 Thanks! Applied it for testing. As I'm not aware of any real live testcase at the moment, I have to wait until there is an outbound call to some specific numbers using reInvites for session handling each 15 minutes. Besides that it is working meanwhile even w/o any patch as mentioned above, I am curious how it behaves with this patch applied. By: Michael Maier (micha) 2021-01-08 09:22:27.915-0600 As I feared: now, the SDP version is incremented twice in case of an outbound call if callee sends a reInvite for session management as previously described. Some time ago, this used not to work w/o my own patches. Suddenly it worked w/o my patches as it should. Now, if this patch here is added (w/o my patch added!), the SDP version is now incremented twice. Conclusion: There must have been some change(s) in the meantime, which (unintentionally?) fixed the existing SDP problem. By: Ivan Poddubny (ipoddubny) 2021-01-25 10:18:42.622-0600 On receiving a re-INVITE without SDP, Asterisk gets the current local SDP offer which is indented to be constant (1), and *modifies* it, without updating the version number. This behavior was introduced in (2), a fix for (3). Adding a couple of lines of code to increment ''origin.version'' solves the issue in my environment. Though, it's still a dirty hack in my opinion, because the local offer is not supposed to be modified without pjmedia knowing about it. I guess, the right solution should use modify_local_offer, but it also changes the negotiator's state from DONE to LOCAL_OFFER. I think, if it's not used, maybe there were some reasons not to use it, but I coulnd't find any clue neither in the issue (3) nor in the reviews on Gerrit (4, 5). (1) https://github.com/pjsip/pjproject/blob/3e8211db8b817e2dc2a9ef03701f40d9309b0589/pjmedia/include/pjmedia/sdp_neg.h#L51-L66 (2) https://github.com/asterisk/asterisk/commit/a098251e7e4f9e019787b996e5897a067c93f41a (3) ASTERISK-25854 (4) https://gerrit.asterisk.org/c/asterisk/+/2533 (5) https://gerrit.asterisk.org/c/asterisk/+/14888 By: Stanislav Abramenkov (silentindark) 2021-02-17 01:50:32.716-0600 This answer was written by sauwming https://github.com/pjsip/pjproject/issues/2620 In the issue page, I believe Ivan has explained it well. Since app/user modifies the local SDP, I think it should be okay to also increase the version. Or is there any reason not to? Note that I tested the scenario here with pjsua app and without the SDP modification, everything seems to work correctly. Attached is the sipp scenario I used for testing: uac-reinv-no-sdp.xml.txt By: Stanislav Abramenkov (silentindark) 2021-02-17 01:51:58.530-0600 the sipp scenario for testing By: Joshua C. Colp (jcolp) 2021-02-17 04:18:57.291-0600 There are multiple cases where this can occur, which is causing confusion. The two known cases that I'm aware of: 1. When an SDP offer is sent in an initial INVITE with a set of codecs (ulaw, alaw) and the remote side answers with a single codec (ulaw). In this case the SDP negotiator updates the active local SDP to be the pruned list (ulaw) without updating the session version number. 2. The case that [~ipoddubny] mentions with re-INVITE with no SDP being used for unhold. The case in 1 occurs because we have not implemented the INVITE session on_create_offer callback, and thus it uses automatically the active local SDP without incrementing the version number. I'd say that it is a bug that PJSIP is doing that, but implementing on_create_offer works fine. This is also why pjsua is working as expected - it implements that callback and provides the SDP each time. I've attached a patch which implements the callback and done some testing which shows the version number incrementing as expected. I plan on doing some more and then putting it up for review. I'll also be commenting on the Teluu side. By: Michael Maier (micha) 2021-02-24 04:34:50.989-0600 @Joshua C. Colp: I'm testing your proposed patches above since 18/Feb/21. I couldn't find any problems so far. SDP version is incremented as expected. Unfortunately I have to say, that the original error use case doesn't exist any more and even other use cases, which showed the same problem, have been resolved by the peer by removing the second codec in the initiating SDP. Therefore, there isn't any difference any more between the SDP in the 200 OK of the initial invite of asterisk and the subsequent 200 OK of the reInvite of asterik. The SDP version is always incremented by one as expected - even though the SDP didn't change. By: Friendly Automation (friendly-automation) 2021-02-25 08:44:33.589-0600 Change 15432 merged by George Joseph: res_pjsip_session: Always produce offer on re-INVITE without SDP. [https://gerrit.asterisk.org/c/asterisk/+/15432|https://gerrit.asterisk.org/c/asterisk/+/15432] By: Friendly Automation (friendly-automation) 2021-02-25 08:51:54.628-0600 Change 15447 merged by Friendly Automation: res_pjsip_session: Always produce offer on re-INVITE without SDP. [https://gerrit.asterisk.org/c/asterisk/+/15447|https://gerrit.asterisk.org/c/asterisk/+/15447] By: Friendly Automation (friendly-automation) 2021-02-25 08:53:03.876-0600 Change 15433 merged by George Joseph: res_pjsip_session: Always produce offer on re-INVITE without SDP. [https://gerrit.asterisk.org/c/asterisk/+/15433|https://gerrit.asterisk.org/c/asterisk/+/15433]