[Home]

Summary:ASTERISK-28157: Asterisk crashes when the res_pjsip_* modules unload
Reporter:sungtae kim (pchero)Labels:pjsip
Date Opened:2018-11-09 03:49:37.000-0600Date Closed:2018-12-03 09:10:24.000-0600
Priority:MinorRegression?
Status:Closed/CompleteComponents:Resources/res_pjsip_session
Versions:15.5.0 16.0.0 Frequency of
Occurrence
Related
Issues:
Environment:DebianAttachments:
Description:Asterisk crashes when the res_pjsip_* modules unloading.

The pjsip supplement register/unregister has a race-condition problem.

When the Asterisk traverse the registered callback for the supplements, it's possible to hits the unregistered callback already. And it did.

Tested with Asterisk-15.5.0 and * master a3fc97aa13 res_pjsip: Send a 503 response when overload state if reliable transport.

{noformat}
AST_LIST_TRAVERSE(&session->supplements, supplement, next) {
if (supplement->incoming_request && does_method_match(&req.method.name, supplement->method)) {
if (supplement->incoming_request(session, rdata)) {
break;
}
}
}
{noformat}

Here's detail core dump.
{noformat}
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
Core was generated by `asterisk -vvvvvvgc'.
Program terminated with signal SIGSEGV, Segmentation fault.
#0  0x00007f0423ad035e in ast_strlen_zero (s=0x7f0426946977 <error: Cannot access memory at address 0x7f0426946977>)
   at /home/sungtaekim/worksapce/asterisk/include/asterisk/strings.h:67
67 return (!s || (*s == '\0'));
[Current thread is 1 (Thread 0x7f0420021700 (LWP 19499))]
(gdb) where
#0  0x00007f0423ad035e in ast_strlen_zero (s=0x7f0426946977 <error: Cannot access memory at address 0x7f0426946977>) at /home/sungtaekim/worksapce/asterisk/include/asterisk/strings.h:67
#1  does_method_match (supplement_method=0x7f0426946977 <error: Cannot access memory at address 0x7f0426946977>, message_method=0x7f0420020968) at res_pjsip_session.c:3129
#2  handle_incoming_request (session=session@entry=0x7f0460152770, rdata=rdata@entry=0x7f0480009938) at res_pjsip_session.c:3310
#3  0x00007f0423ad05b3 in handle_incoming (session=session@entry=0x7f0460152770, rdata=rdata@entry=0x7f0480009938, response_priority=response_priority@entry=AST_SIP_SESSION_BEFORE_MEDIA) at res_pjsip_session.c:3344
#4  0x00007f0423ad5eb4 in handle_incoming_before_media (rdata=0x7f0480009938, session=0x7f0460152770, inv=0x7f0460185d08) at res_pjsip_session.c:3459
#5  session_inv_on_state_changed (inv=0x7f0460185d08, e=<optimized out>) at res_pjsip_session.c:3519
#6  0x00007f048bd19248 in inv_set_state (state=PJSIP_INV_STATE_DISCONNECTED, e=0x7f0420020b30, inv=0x7f0460185d08) at ../src/pjsip-ua/sip_inv.c:318
#7  inv_respond_incoming_bye (inv=0x7f0460185d08, bye_tsx=<optimized out>, rdata=<optimized out>, e=0x7f0420020b30) at ../src/pjsip-ua/sip_inv.c:3385
#8  0x00007f048bd1ffb3 in inv_on_state_confirmed (inv=0x7f0460185d08, e=0x7f0420020b30) at ../src/pjsip-ua/sip_inv.c:4790
#9  0x00007f048bd19637 in mod_inv_on_tsx_state (tsx=0x7f04601418f8, e=0x7f0420020b30) at ../src/pjsip-ua/sip_inv.c:718
#10 0x00007f048bd58979 in pjsip_dlg_on_tsx_state (dlg=0x7f04601704a8, tsx=0x7f04601418f8, e=0x7f0420020b30) at ../src/pjsip/sip_dialog.c:2069
#11 0x00007f048bd52fe9 in tsx_set_state (flag=<optimized out>, event_src=<optimized out>, event_src_type=<optimized out>, state=<optimized out>, tsx=<optimized out>) at ../src/pjsip/sip_transaction.c:1272
#12 tsx_on_state_null (tsx=0x7f04601418f8, event=<optimized out>) at ../src/pjsip/sip_transaction.c:2429
#13 0x00007f048bd55f6f in pjsip_tsx_recv_msg (tsx=0x7f04601418f8, rdata=rdata@entry=0x7f0480009938) at ../src/pjsip/sip_transaction.c:1832
#14 0x00007f048bd58702 in pjsip_dlg_on_rx_request (dlg=dlg@entry=0x7f04601704a8, rdata=rdata@entry=0x7f0480009938) at ../src/pjsip/sip_dialog.c:1716
#15 0x00007f048bd59c97 in mod_ua_on_rx_request (rdata=0x7f0480009938) at ../src/pjsip/sip_ua_layer.c:704
#16 0x00007f048bd3b316 in pjsip_endpt_process_rx_data (endpt=<optimized out>, rdata=rdata@entry=0x7f0480009938, p=p@entry=0x7f0427de10e0 <param>, p_handled=p_handled@entry=0x7f0420020d4c) at ../src/pjsip/sip_endpoint.c:893
#17 0x00007f0427bb429f in distribute (data=0x7f0480009938) at res_pjsip/pjsip_distributor.c:951
#18 0x000055ce9036db28 in ast_taskprocessor_execute (tps=tps@entry=0x55ce92856db0) at taskprocessor.c:974
#19 0x000055ce903748f0 in execute_tasks (data=0x55ce92856db0) at threadpool.c:1348
#20 0x000055ce9036db28 in ast_taskprocessor_execute (tps=0x55ce925a95f0) at taskprocessor.c:974
#21 0x000055ce90375154 in threadpool_execute (pool=0x55ce925a8d00) at threadpool.c:367
#22 worker_active (worker=0x7f0468000980) at threadpool.c:1131
#23 worker_start (arg=arg@entry=0x7f0468000980) at threadpool.c:1050
#24 0x000055ce9037d56c in dummy_start (data=<optimized out>) at utils.c:1249
#25 0x00007f048a496494 in start_thread (arg=0x7f0420021700) at pthread_create.c:333
#26 0x00007f04890a0acf in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:97
{noformat}
Comments:By: Asterisk Team (asteriskteam) 2018-11-09 03:49:39.481-0600

Thanks for creating a report! The issue has entered the triage process. That means the issue will wait in this status until a Bug Marshal has an opportunity to review the issue. Once the issue has been reviewed you will receive comments regarding the next steps towards resolution.

A good first step is for you to review the [Asterisk Issue Guidelines|https://wiki.asterisk.org/wiki/display/AST/Asterisk+Issue+Guidelines] if you haven't already. The guidelines detail what is expected from an Asterisk issue report.

Then, if you are submitting a patch, please review the [Patch Contribution Process|https://wiki.asterisk.org/wiki/display/AST/Patch+Contribution+Process].
By: sungtae kim (pchero) 2018-11-09 04:02:39.655-0600

And I would like fix this. May I assign it to myself?
By: Kevin Harwell (kharwell) 2018-11-09 09:32:37.510-0600

@sungtae kim, yes, and if you can provide/push the patch to gerrit it would be much appreciated. I went ahead and assigned you the issue.

Thanks!
By: sungtae kim (pchero) 2018-11-12 03:59:06.555-0600

Created merge request.

Please someone review this. :)

https://gerrit.asterisk.org/#/c/asterisk/+/10617/
By: sungtae kim (pchero) 2018-11-12 16:35:22.539-0600

Received feedback.
By: sungtae kim (pchero) 2018-11-13 17:47:57.617-0600

Submitted another commit. Could someone review this? Thanks. :)
By: Joshua C. Colp (jcolp) 2018-11-13 18:43:19.269-0600

There is no need to ask for reviews on JIRA issues, everything that is up for review gets reviewed.
By: sungtae kim (pchero) 2018-11-14 06:22:54.599-0600

@Joshua, I've got it! Thanks for all your comments. :)
By: Friendly Automation (friendly-automation) 2018-12-03 09:10:26.299-0600

Change 10716 merged by Jenkins2:
res_pjsip: Patch for res_pjsip_* module load/reload crash

[https://gerrit.asterisk.org/10716|https://gerrit.asterisk.org/10716]
By: Friendly Automation (friendly-automation) 2018-12-18 10:44:02.161-0600

Change 10617 merged by George Joseph:
res_pjsip: Patch for res_pjsip_* module load/reload crash

[https://gerrit.asterisk.org/10617|https://gerrit.asterisk.org/10617]
By: Friendly Automation (friendly-automation) 2018-12-18 10:44:24.463-0600

Change 10704 merged by George Joseph:
res_pjsip: Patch for res_pjsip_* module load/reload crash

[https://gerrit.asterisk.org/10704|https://gerrit.asterisk.org/10704]