[Home]

Summary:ASTERISK-29664: PJSIP processing token with % incorrectly
Reporter:Dan Cropp (daninmadison)Labels:
Date Opened:2021-09-21 14:06:33Date Closed:2022-01-05 13:12:53.000-0600
Priority:MajorRegression?
Status:Closed/CompleteComponents:pjproject/pjsip
Versions:16.17.0 Frequency of
Occurrence
Constant
Related
Issues:
Environment:Ubuntu 18Attachments:
Description:We have a SIP provider who sends an INVITE with a Branch header.
Everything is fine in most cases.  However, there are times they send a Branch header with a % character.
From what I have been told, the header is a token, so it is acceptable to include the % character.

There seem to be two different issues with PJSIP processing of headers with the % character in tokenized fields.

Example.
branch=z9hG4bKf5s1psess0KGj0gRsgyJIBWEYL%QWIjkWmDXr8xnPVgaOBzOLSV28


In one case, it can receive the portion and process it, but when sending out the same token it changes it to lower case.  Example, received %8A portion, but sent %8a.  This means it’s not a match for the token so the other side will not match.
Received
branch=z9hG4bKf5s1psess0gbkeviMkQYREmK31pLWRKQ+FRmVzHQNG1Xv%8AnzLVA!;extension=QKIT3jRNBcmZT7yu5m7absBxXxlfn1kLwfYKYT0MOyY!
Sent
branch=z9hG4bKf5s1psess0gbkeviMkQYREmK31pLWRKQ+FRmVzHQNG1Xv%8anzLVA!;extension=QKIT3jRNBcmZT7yu5m7absBxXxlfn1kLwfYKYT0MOyY!


It another case, seems PJSIP has another scenario where the % character is processed as the escaped character if it would be printable character.  Example, received %4C.  The send replaces that portion with upper case L

Received
branch=z9hG4bKf5s1psess0gbkeviMkQYREmK31pLWRKQ+FRmVzHQNG1Xv%4CnzLVA!;extension=QKIT3jRNBcmZT7yu5m7absBxXxlfn1kLwfYKYT0MOyY!
Sent
branch=z9hG4bKf5s1psess0gbkeviMkQYREmK31pLWRKQ+FRmVzHQNG1XvLnzLVA!;extension=QKIT3jRNBcmZT7yu5m7absBxXxlfn1kLwfYKYT0MOyY!
Comments:By: Asterisk Team (asteriskteam) 2021-09-21 14:06:38.707-0500

Thanks for creating a report! The issue has entered the triage process. That means the issue will wait in this status until a Bug Marshal has an opportunity to review the issue. Once the issue has been reviewed you will receive comments regarding the next steps towards resolution. Please note that log messages and other files should not be sent to the Sangoma Asterisk Team unless explicitly asked for. All files should be placed on this issue in a sanitized fashion as needed.

A good first step is for you to review the [Asterisk Issue Guidelines|https://wiki.asterisk.org/wiki/display/AST/Asterisk+Issue+Guidelines] if you haven't already. The guidelines detail what is expected from an Asterisk issue report.

Then, if you are submitting a patch, please review the [Patch Contribution Process|https://wiki.asterisk.org/wiki/display/AST/Patch+Contribution+Process].

Please note that once your issue enters an open state it has been accepted. As Asterisk is an open source project there is no guarantee or timeframe on when your issue will be looked into. If you need expedient resolution you will need to find and pay a suitable developer. Asking for an update on your issue will not yield any progress on it and will not result in a response. All updates are posted to the issue when they occur.

Please note that by submitting data, code, or documentation to Sangoma through JIRA, you accept the Terms of Use present at [https://www.asterisk.org/terms-of-use/|https://www.asterisk.org/terms-of-use/].

By: Dan Cropp (daninmadison) 2021-09-21 14:10:03.890-0500

Here is a log of the INVITE with the branch header being incorrect in the SIP 100 Trying response.


By: Dan Cropp (daninmadison) 2021-09-22 09:02:12.288-0500

Additional item we noticed, it seems as though it's only hex characters immediately following the '%' character that are impacted.  Seems like PJSIP may be processing it as an escape and attempting to interpret the %__ as the hex character equivalent.

For example, branch=z9hG4bKf5s1psess0gbkeviMkQYREmK31pLWRKQ+FRmVzHQNG1Xv%4QnzLVA!;extension=QKIT3jRNBcmZT7yu5m7absBxXxlfn1kLwfYKYT0MOyY!
worked during a test.  (Or at least %4Q worked when we had customer on 16.15.0 and encountered it.)  Those in our development area suspect it's because PJSIP saw Q was not a valid hex character so it left it alone.

The SIP provider claims this work on Asterisk customers running chan_sip, this is PJSIP only.



By: Sean Bright (seanbright) 2021-09-22 09:08:38.218-0500

Thanks. Can you follow [the instructions here|https://wiki.asterisk.org/wiki/display/AST/Collecting+Debug+Information] for collecting debug information?

By: Sean Bright (seanbright) 2021-09-22 09:12:20.852-0500

Also, can you try with the latest version of Asterisk 16? 16.2 is pretty old.

By: Dan Cropp (daninmadison) 2021-09-22 09:13:21.481-0500

Do you need the debug information of a %4Q case that works?
Or did I do something incorrectly for the previous debug information I supplied with it failing?

Just trying to understand what is needed.


By: Sean Bright (seanbright) 2021-09-22 09:15:28.922-0500

There is no debug information attached. There are two identical files that contain a single request/response, but no Asterisk debug logging. Are you following [the instructions from the page that I linked|https://wiki.asterisk.org/wiki/display/AST/Collecting+Debug+Information]?

By: Dan Cropp (daninmadison) 2021-09-22 09:21:05.742-0500

My apologies.  I trimmed the previous files trying to keep the sample smaller.
Attaching the entire files this time.

By: Sean Bright (seanbright) 2021-09-22 09:22:48.427-0500

Thanks. While we're reviewing can you please try and reproduce this on a newer version of Asterisk 16? 16.2 is two years old and 16.20 is the latest release.

By: Sean Bright (seanbright) 2021-09-22 09:30:19.570-0500

I'm able to reproduce with Asterisk 16 from Git with bundled project.

By: Dan Cropp (daninmadison) 2021-09-22 09:31:35.876-0500

The debug_log attached should have been 16.17.0, not 16.2.x.
Not sure where 16.2 came from.  When I initially supplied issue, I indicated 16.17.0 (and can be seen in the e-mails generated).  Seems someone changed that to 16.2.1

In the debug_log provided, you can see various AGI interactions "agi_version%3A%2016.17.0"

By: Sean Bright (seanbright) 2021-09-22 10:07:14.810-0500

I updated it on the issue - it came from the messages.txt you attached:

{noformat}
...
Server: Asterisk PBX 16.2.1
...
{noformat}

By: Sean Bright (seanbright) 2021-09-22 10:07:47.970-0500

Tentative patch attached

By: Dan Cropp (daninmadison) 2021-09-22 10:11:41.465-0500

Thank you Sean.

I see why you got the 16.2.1 from. Just talked with support and learned this SIP provider is so old they literally require the User Agent to be Asterisk PBX 16.2.1 (or earlier).  Tech support has to program the PJSIP User Agent setting to that asterisk version (or earlier) when they work with this SIP provider.

As you can see from the debug_log it's really 16.17.0.

Just the PJSIP global section had that setting to appease this exact SIP provider.  (Same SIP provider keeps telling our customers to use chan_sip and we have to keep correcting them).

By: Sean Bright (seanbright) 2021-09-22 10:24:07.662-0500

[~daninmadison], do you mind if I share your example SIP request/response with the pjsip team on their public issue tracker?

By: Dan Cropp (daninmadison) 2021-09-22 10:34:22.170-0500

Would it be possible to modify information like any names, phone numbers, IP addresses?


By: Sean Bright (seanbright) 2021-09-22 10:42:33.070-0500

It would be easier for me to just create new ones. I've deleted the files you've attached to this issue (this is also a public issue tracker).

By: Dan Cropp (daninmadison) 2021-09-22 10:44:45.297-0500

Thank you Sean

By: Friendly Automation (friendly-automation) 2022-01-05 13:12:53.973-0600

Change 17804 merged by Friendly Automation:
pjproject: Fix incorrect unescaping of tokens during parsing

[https://gerrit.asterisk.org/c/asterisk/+/17804|https://gerrit.asterisk.org/c/asterisk/+/17804]

By: Friendly Automation (friendly-automation) 2022-01-05 13:13:50.138-0600

Change 17803 merged by Joshua Colp:
pjproject: Fix incorrect unescaping of tokens during parsing

[https://gerrit.asterisk.org/c/asterisk/+/17803|https://gerrit.asterisk.org/c/asterisk/+/17803]

By: Friendly Automation (friendly-automation) 2022-01-05 13:14:53.449-0600

Change 17783 merged by Friendly Automation:
pjproject: Fix incorrect unescaping of tokens during parsing

[https://gerrit.asterisk.org/c/asterisk/+/17783|https://gerrit.asterisk.org/c/asterisk/+/17783]

By: Friendly Automation (friendly-automation) 2022-01-05 13:17:36.461-0600

Change 17805 merged by Joshua Colp:
pjproject: Fix incorrect unescaping of tokens during parsing

[https://gerrit.asterisk.org/c/asterisk/+/17805|https://gerrit.asterisk.org/c/asterisk/+/17805]