Summary:	ASTERISK-29022: Crash when manipulating PJSIP invite dlg ref counts
Reporter:	Sean Bright (seanbright)	Labels:
Date Opened:	2020-08-07 10:52:49	Date Closed:	2020-12-09 13:08:10.000-0600
Priority:	Major	Regression?	Yes
Status:	Closed/Complete	Components:	Channels/chan_pjsip
Versions:	GIT	Frequency of Occurrence	Occasional
Related Issues:	is related to ASTERISK-29186 chan_pjsip: Endpoint not registered: log level 6: assert!
Environment:	Asterisk 16 from Git (15a3318f1f3fb72669b5659d12948fe3a01dd21b) Ubuntu 20.04 amd64	Attachments:	( 0) core-asterisk-9283-HOSTNAME-1604323228-thread1.txt ( 1) crash.txt
Description:	Asterisk crashes in pjsip_inv_add_ref() and pjsip_inv_dec_ref(). Core files sent to the Asterisk Team e-mail address. It's happened twice in as many days. Planning to roll back to an older known-good version (cfaf8dfc4d95a2123b682f540cd1801de7177171) so I won't be able to test any patches.
Comments:	By: Asterisk Team (asteriskteam) 2020-08-07 10:52:50.219-0500 Thanks for creating a report! The issue has entered the triage process. That means the issue will wait in this status until a Bug Marshal has an opportunity to review the issue. Once the issue has been reviewed you will receive comments regarding the next steps towards resolution. Please note that log messages and other files should not be sent to the Sangoma Asterisk Team unless explicitly asked for. All files should be placed on this issue in a sanitized fashion as needed. A good first step is for you to review the [Asterisk Issue Guidelines|https://wiki.asterisk.org/wiki/display/AST/Asterisk+Issue+Guidelines] if you haven't already. The guidelines detail what is expected from an Asterisk issue report. Then, if you are submitting a patch, please review the [Patch Contribution Process|https://wiki.asterisk.org/wiki/display/AST/Patch+Contribution+Process]. Please note that once your issue enters an open state it has been accepted. As Asterisk is an open source project there is no guarantee or timeframe on when your issue will be looked into. If you need expedient resolution you will need to find and pay a suitable developer. Asking for an update on your issue will not yield any progress on it and will not result in a response. All updates are posted to the issue when they occur. By: Ross Beer (rossbeer) 2020-11-02 07:47:24.942-0600 I have been testing the patch for this and have just had the following segfault: {noformat} Thread 1 (Thread 0x7f779eee8700 (LWP 44980)): #0  0x00007f7a86691eec in pj_pool_release (pool=0x0) at ../include/pj/pool_i.h:102 #1  0x00007f7a865bdc1e in inv_session_destroy (inv=0x7f792082c9d8) at ../src/pjsip-ua/sip_inv.c:229 #2  0x00007f7a865bdcbe in pjsip_inv_dec_ref (inv=0x7f792082c9d8) at ../src/pjsip-ua/sip_inv.c:251        ref_cnt = 0 #3  0x00007f789888f7c0 in session_destructor (obj=0x7f79201f98a0) at res_pjsip_session.c:3003        session = 0x7f79201f98a0        delay = 0x0        __PRETTY_FUNCTION__ = "session_destructor" {noformat} Thread 1 Backtrace attached. It may not be related, however, I had not seen this issue until I started testing the patch. By: nappsoft (nappsoft) 2020-12-04 09:32:32.350-0600 Just got a similar backtrace with the patch applied. (However I do not think that the patch made the situation worse, but it didn't help) => applied it because I had strange crashes (did not make a backtrace before applying the patch) with TCP/TLS sip sessions on canceling invites By: Joshua C. Colp (jcolp) 2020-12-04 09:36:02.208-0600 The patch currently up has an issue where it can still crash, which was found by the testsuite. By: nappsoft (nappsoft) 2020-12-07 01:22:22.928-0600 Thanks for your feedback. Btw: I am not sure whether patchset 3 is fixing the issue either. While investigating ASTERISK-29197 and ASTERISK-29024 I applied patchset 3 from gerrit as well. With patchset 3 applied a got a crash after 40 minutes of testing (unfortunately without a backtrace as I didn't run asterisk in gdb and did not have enough space for a memory dump) but with the patch reverted I was able to let an automated test run for about 28 hours without any issue. => will retest in the debugger tomorrow or on Wednesday if you like. By: Joshua C. Colp (jcolp) 2020-12-07 04:05:19.812-0600 A backtrace would be needed to see what is going on. By: nappsoft (nappsoft) 2020-12-09 04:11:52.502-0600 False alarm it seems. Now, with the new patch for ASTERISK-29024, a was not able to reproduce any crash, even with your patch applied. Even running your patch on a productive system atm without any issues so far. By: Friendly Automation (friendly-automation) 2020-12-09 13:08:12.015-0600 Change 15122 merged by Joshua Colp: pjsip: Match lifetime of INVITE session to our session. [https://gerrit.asterisk.org/c/asterisk/+/15122|https://gerrit.asterisk.org/c/asterisk/+/15122] By: Friendly Automation (friendly-automation) 2020-12-09 13:08:27.073-0600 Change 15123 merged by Joshua Colp: pjsip: Match lifetime of INVITE session to our session. [https://gerrit.asterisk.org/c/asterisk/+/15123|https://gerrit.asterisk.org/c/asterisk/+/15123] By: Friendly Automation (friendly-automation) 2020-12-09 13:08:37.489-0600 Change 15113 merged by Joshua Colp: pjsip: Match lifetime of INVITE session to our session. [https://gerrit.asterisk.org/c/asterisk/+/15113|https://gerrit.asterisk.org/c/asterisk/+/15113]