[Home]

Summary:ASTERISK-25736: pbx core: Deadlock during a reload
Reporter:Kevin Harwell (kharwell)Labels:
Date Opened:2016-02-01 12:58:34.000-0600Date Closed:
Priority:MinorRegression?No
Status:Open/NewComponents:PBX/General
Versions:13.7.0 Frequency of
Occurrence		Occasional
Related
Issues:
is duplicated byASTERISK-25784 lock-order-inversion (potential deadlock) on dialplan reload
is related toASTERISK-29774 Deadlock where hints lock blocks contexts lock
Environment:Attachments:( 0) backtrace-threads-last.txt
( 1) core-show-locks-last.txt
Description:The following thread (Thread ID: 0x7f48d3b03700) locks the hints container and then waits for the contexts lock to be released:
{noformat}
Lock #3 (pbx.c): MUTEX 3841 ast_add_hint hints 0x1336a60 (1)
main/backtrace.c:59 __ast_bt_get_addresses() (0x46d368+1D)
main/lock.c:258 __ast_pthread_mutex_lock() (0x551105+C9)
main/astobj2.c:200 __ao2_lock() (0x460fa7+F4)
main/pbx.c:3844 ast_add_hint()
main/pbx.c:7230 ast_add_extension2_lockopt()
main/pbx.c:7006 ast_add_extension2() (0x59df10+67)
pbx/pbx_config.c:1841 pbx_load_config()
pbx/pbx_config.c:2045 pbx_load_module()
pbx/pbx_config.c:2097 reload()
main/loader.c:962 ast_module_reload() (0x54e546+43F)
main/cli.c:344 handle_core_reload()
main/cli.c:2717 ast_cli_command_full() (0x4e3404+277)
main/cli.c:2741 ast_cli_command_multiple_full() (0x4e3797+C1)
main/asterisk.c:1435 netconsole()
main/utils.c:1232 dummy_start()
nptl/pthread_create.c:312 start_thread()
libc.so.6 clone() (0x7f48d47f8410+6D)
=== ---> Waiting for Lock #4 (pbx.c): MUTEX 8183 ast_rdlock_contexts &conlock 0x917100 (1)
main/backtrace.c:59 __ast_bt_get_addresses() (0x46d368+1D)
main/lock.c:258 __ast_pthread_mutex_lock() (0x551105+C9)
main/pbx.c:8184 ast_rdlock_contexts() (0x5a19e2+23)
main/pbx.c:2782 pbx_extension_helper()
main/pbx.c:3998 ast_exists_extension() (0x59426f+61)
res/res_pjsip_exten_state.c:355 new_subscribe()
res/res_pjsip_pubsub.c:978 build_resource_tree()
res/res_pjsip_pubsub.c:2689 pubsub_on_rx_subscribe_request()
res/res_pjsip_pubsub.c:3188 pubsub_on_rx_request()
:0 pjsip_endpt_process_rx_data() (0x7f48d2196ae9+1E9)
res_pjsip/pjsip_distributor.c:461 distribute()
main/taskprocessor.c:852 ast_taskprocessor_execute() (0x60f193+10D)
main/threadpool.c:1320 execute_tasks()
main/taskprocessor.c:852 ast_taskprocessor_execute() (0x60f193+10D)
main/threadpool.c:351 threadpool_execute()
main/threadpool.c:1103 worker_active()
main/threadpool.c:1024 worker_start()
main/utils.c:1232 dummy_start()
nptl/pthread_create.c:312 start_thread()
libc.so.6 clone() (0x7f48d47f8410+6D)
{noformat}

However the following thread (Thread ID: 0x7f48199bb700) has locked the contexts lock and it waiting for the hints container:
{noformat}
Lock #0 (pbx.c): MUTEX 8183 ast_rdlock_contexts &conlock 0x917100 (1)
main/backtrace.c:59 __ast_bt_get_addresses() (0x46d368+1D)
main/lock.c:258 __ast_pthread_mutex_lock() (0x551105+C9)
main/pbx.c:8184 ast_rdlock_contexts() (0x5a19e2+23)
main/pbx.c:4636 find_context_locked()
main/pbx.c:6699 ast_add_extension() (0x59d264+31)
main/pbx.c:3586 extension_state_add_destroy()
main/pbx.c:3643 ast_extension_state_add_destroy_extended() (0x593001+41)
res/res_pjsip_exten_state.c:378 subscription_established()
res/res_pjsip_pubsub.c:2582 generate_initial_notify()
res/res_pjsip_pubsub.c:2606 initial_notify_task()
main/taskprocessor.c:852 ast_taskprocessor_execute() (0x60f193+10D)
main/threadpool.c:1320 execute_tasks()
main/taskprocessor.c:852 ast_taskprocessor_execute() (0x60f193+10D)
main/threadpool.c:351 threadpool_execute()
main/threadpool.c:1103 worker_active()
main/threadpool.c:1024 worker_start()
main/utils.c:1232 dummy_start()
nptl/pthread_create.c:312 start_thread()
libc.so.6 clone() (0x7f48d47f8410+6D)
=== ---> Waiting for Lock #1 (pbx.c): MUTEX 3841 ast_add_hint hints 0x1336a60 (1)
main/backtrace.c:59 __ast_bt_get_addresses() (0x46d368+1D)
main/lock.c:258 __ast_pthread_mutex_lock() (0x551105+C9)
main/astobj2.c:200 __ao2_lock() (0x460fa7+F4)
main/pbx.c:3686 ast_extension_state_del() (0x593119+CD)
res/res_pjsip_exten_state.c:347 subscription_shutdown()
res/res_pjsip_pubsub.c:1179 shutdown_subscriptions()
res/res_pjsip_pubsub.c:3305 pubsub_on_evsub_state()
evsub.c:0 set_state()
:0 pjsip_evsub_send_request() (0x7f48d23dd7d3+13F)
res/res_pjsip_pubsub.c:1696 sip_subscription_send_request()
res/res_pjsip_pubsub.c:2147 send_notify()
res/res_pjsip_pubsub.c:3219 serialized_pubsub_on_server_timeout()
main/taskprocessor.c:852 ast_taskprocessor_execute() (0x60f193+10D)
main/threadpool.c:1320 execute_tasks()
main/taskprocessor.c:852 ast_taskprocessor_execute() (0x60f193+10D)
main/threadpool.c:351 threadpool_execute()
main/threadpool.c:1103 worker_active()
main/threadpool.c:1024 worker_start()
main/utils.c:1232 dummy_start()
nptl/pthread_create.c:312 start_thread()
libc.so.6 clone() (0x7f48d47f8410+6D)
{noformat}
Comments: