| Summary: | ASTERISK-25777: data race in threadpool | ||
| Reporter: | Badalian Vyacheslav (slavon) | Labels: | |
| Date Opened: | 2016-02-11 23:00:15.000-0600 | Date Closed: | 2016-05-24 17:56:59 |
| Priority: | Major | Regression? | |
| Status: | Closed/Complete | Components: | |
| Versions: | 13.7.2 | Frequency of Occurrence | |
| Related Issues: | |||
| Environment: | [root@vm-asterisk04t asterisk]# git branch -v * master a394865 Merge "Resources/res_phoneprov: fix memory leak and heap-use-after-free" | Attachments: | ( 0) ASTERISK-25777.patch |
| Description: | {code}
Asterisk Ready. ================== WARNING: ThreadSanitizer: data race (pid=6553) Write of size 4 at 0x7d440001dd20 by thread T2 (mutexes: write M969): #0 worker_set_state /root/asterisk/main/threadpool.c:1173 (asterisk+0x00000069680d) #1 worker_shutdown /root/asterisk/main/threadpool.c:979 (asterisk+0x000000695976) #2 worker_thread_destroy /root/asterisk/main/threadpool.c:997 (asterisk+0x000000695b69) #3 __ao2_ref /root/asterisk/main/astobj2.c:528 (asterisk+0x0000002d87e0) #4 thread_worker_pair_destructor /root/asterisk/main/threadpool.c:206 (asterisk+0x00000069170b) #5 __ao2_ref /root/asterisk/main/astobj2.c:528 (asterisk+0x0000002d87e0) #6 queued_idle_thread_dead /root/asterisk/main/threadpool.c:317 (asterisk+0x00000069211b) #7 ast_taskprocessor_execute /root/asterisk/main/taskprocessor.c:850 (asterisk+0x00000067f1bb) #8 default_tps_processing_function /root/asterisk/main/taskprocessor.c:183 (asterisk+0x00000067b1c4) #9 dummy_start /root/asterisk/main/utils.c:1232 (asterisk+0x0000006b6674) #10 <null> <null> (libtsan.so.0+0x000000023659) Previous read of size 4 at 0x7d440001dd20 by thread T7: #0 worker_start /root/asterisk/main/threadpool.c:1040 (asterisk+0x000000695e0a) #1 dummy_start /root/asterisk/main/utils.c:1232 (asterisk+0x0000006b6674) #2 <null> <null> (libtsan.so.0+0x000000023659) Location is heap block of size 272 at 0x7d440001dc40 allocated by thread T2: #0 calloc <null> (libtsan.so.0+0x000000025bed) #1 _ast_calloc /root/asterisk/include/asterisk/utils.h:573 (asterisk+0x0000002d752c) #2 __ao2_alloc /root/asterisk/main/astobj2.c:597 (asterisk+0x0000002d8af6) #3 worker_thread_alloc /root/asterisk/main/threadpool.c:1061 (asterisk+0x000000695f53) #4 grow /root/asterisk/main/threadpool.c:520 (asterisk+0x000000693150) #5 queued_set_size /root/asterisk/main/threadpool.c:837 (asterisk+0x000000694d3b) #6 ast_taskprocessor_execute /root/asterisk/main/taskprocessor.c:850 (asterisk+0x00000067f1bb) #7 default_tps_processing_function /root/asterisk/main/taskprocessor.c:183 (asterisk+0x00000067b1c4) #8 dummy_start /root/asterisk/main/utils.c:1232 (asterisk+0x0000006b6674) #9 <null> <null> (libtsan.so.0+0x000000023659) Mutex M969 (0x7d440001dcd8) created at: #0 pthread_mutex_init <null> (libtsan.so.0+0x000000028645) #1 __ast_pthread_mutex_init /root/asterisk/main/lock.c:149 (asterisk+0x000000503c43) #2 worker_thread_alloc /root/asterisk/main/threadpool.c:1066 (asterisk+0x000000695fe5) #3 grow /root/asterisk/main/threadpool.c:520 (asterisk+0x000000693150) #4 queued_set_size /root/asterisk/main/threadpool.c:837 (asterisk+0x000000694d3b) #5 ast_taskprocessor_execute /root/asterisk/main/taskprocessor.c:850 (asterisk+0x00000067f1bb) #6 default_tps_processing_function /root/asterisk/main/taskprocessor.c:183 (asterisk+0x00000067b1c4) #7 dummy_start /root/asterisk/main/utils.c:1232 (asterisk+0x0000006b6674) #8 <null> <null> (libtsan.so.0+0x000000023659) Thread T2 (tid=7036, running) created by main thread at: #0 pthread_create <null> (libtsan.so.0+0x000000027b07) #1 ast_pthread_create_stack /root/asterisk/main/utils.c:1285 (asterisk+0x0000006b6abd) #2 default_listener_start /root/asterisk/main/taskprocessor.c:200 (asterisk+0x00000067b33a) #3 __allocate_taskprocessor /root/asterisk/main/taskprocessor.c:682 (asterisk+0x00000067e298) #4 ast_taskprocessor_get /root/asterisk/main/taskprocessor.c:728 (asterisk+0x00000067e4c1) #5 threadpool_alloc /root/asterisk/main/threadpool.c:402 (asterisk+0x00000069266e) #6 ast_threadpool_create /root/asterisk/main/threadpool.c:894 (asterisk+0x000000695202) #7 stasis_init /root/asterisk/main/stasis.c:1620 (asterisk+0x00000063db9c) #8 asterisk_daemon /root/asterisk/main/asterisk.c:4420 (asterisk+0x0000002d65dd) #9 main /root/asterisk/main/asterisk.c:4230 (asterisk+0x0000002d5cfd) Thread T7 (tid=7041, finished) created by thread T2 at: #0 pthread_create <null> (libtsan.so.0+0x000000027b07) #1 ast_pthread_create_stack /root/asterisk/main/utils.c:1285 (asterisk+0x0000006b6abd) #2 worker_thread_start /root/asterisk/main/threadpool.c:1077 (asterisk+0x000000696200) #3 grow /root/asterisk/main/threadpool.c:525 (asterisk+0x0000006931eb) #4 queued_set_size /root/asterisk/main/threadpool.c:837 (asterisk+0x000000694d3b) #5 ast_taskprocessor_execute /root/asterisk/main/taskprocessor.c:850 (asterisk+0x00000067f1bb) #6 default_tps_processing_function /root/asterisk/main/taskprocessor.c:183 (asterisk+0x00000067b1c4) #7 dummy_start /root/asterisk/main/utils.c:1232 (asterisk+0x0000006b6674) #8 <null> <null> (libtsan.so.0+0x000000023659) SUMMARY: ThreadSanitizer: data race /root/asterisk/main/threadpool.c:1173 worker_set_state {code} | ||
| Comments: | By: Asterisk Team (asteriskteam) 2016-02-11 23:00:17.078-0600 Thanks for creating a report! The issue has entered the triage process. That means the issue will wait in this status until a Bug Marshal has an opportunity to review the issue. Once the issue has been reviewed you will receive comments regarding the next steps towards resolution. A good first step is for you to review the [Asterisk Issue Guidelines|https://wiki.asterisk.org/wiki/display/AST/Asterisk+Issue+Guidelines] if you haven't already. The guidelines detail what is expected from an Asterisk issue report. Then, if you are submitting a patch, please review the [Patch Contribution Process|https://wiki.asterisk.org/wiki/display/AST/Patch+Contribution+Process]. By: Corey Farrell (coreyfarrell) 2016-02-12 10:06:00.626-0600 It looks like the problem is that {{worker_start}} checks {{worker->status == ZOMBIE}} after releasing {{worker->lock}}. The unlock needs to be done after checking status. This change will require testing so I will post to gerrit once I (or someone else) has had the chance to try the patch. By: Badalian Vyacheslav (slavon) 2016-02-12 10:58:46.951-0600 i will test patch By: Badalian Vyacheslav (slavon) 2016-02-12 12:07:16.037-0600 Now looks like patch was help... i will test more time... By: Badalian Vyacheslav (slavon) 2016-02-12 12:59:17.243-0600 i think its was fixed by patch... can't catch it anymore... By: Badalian Vyacheslav (slavon) 2016-02-12 15:14:47.975-0600 Fixed by patch. Tested. | ||