| Summary: | ASTERISK-29247: Asterisk create too many threads | ||
| Reporter: | cagdas kopuz (cagdas) | Labels: | |
| Date Opened: | 2021-01-15 02:20:01.000-0600 | Date Closed: | 2021-01-29 12:00:27.000-0600 |
| Priority: | Minor | Regression? | |
| Status: | Closed/Complete | Components: | Resources/General |
| Versions: | GIT 16.15.0 | Frequency of Occurrence | Constant |
| Related Issues: | |||
| Environment: | kubernetes, docker container | Attachments: | ( 0) threads.txt |
| Description: | My asterisk server has about 5000 peer and max 200 calls in rush hours. Sometimes, asterisk write this error to logs,
{quote}Jan 15 10:51:46 sip asterisk \[3181454\]: ERROR \[3181456\]: threadpool.c: 545 in grow: Unable to start worker thread 734436. Destroying.{quote} When this error occurs asterisk can not initiate a new call. Right at this time when I ran the "core show threads" command. Output gives me "18039 threads listed." I use full realtime, chan_sip, AMI in my environment extconfig.conf queue_log => odbc,proxysql,queue_log,1 queue_log => odbc,proxysql,queue_log,2 sipusers => odbc,pbxconf,sip_users,1 sippeers => odbc,pbxconf,sip_users,1 voicemail => odbc,pbxconf,voicemail_users,1 queues => odbc,pbxconf,queues,1 queue_members => odbc,pbxconf,queue_members,1 queue_dmembers => odbc,pbxconf,queue_dmembers,1 extensions => odbc,pbxconf,extensions,1 extensions_realtime.include => odbc,pbxconf,n23dialplan,1 extensions.include => odbc,pbxconf,n23dialplan,1 menus.include => odbc,pbxconf,n23dialplan,1 schedule.include => odbc,pbxconf,n23dialplan,1 musiconhold => odbc,pbxconf,musiconhold,1 musiconhold_entry => odbc,pbxconf,musiconhold_entry,1 confbridge.conf => odbc,pbxconf,conference,1 sip.conf [general] bindport=5060 nat=force_rport,comedia 100rel=required rtcachefriends=yes rtupdate=yes rtautoclear=10 allowguest=no alwaysauthreject=yes trustrpid=yes sendrpid=yes context=from-outside allowsubscribe=yes callcounter=yes videosupport=yes avpf=no rtptimeout=120 progressinband=yes prematuremedia=no defaultexpiry=600 registertimeout=3600 minexpiry=108 maxexpiry=108 tcpenable=yes tcpbindaddr=0.0.0.0 | ||
| Comments: | By: Asterisk Team (asteriskteam) 2021-01-15 02:20:02.395-0600 Thanks for creating a report! The issue has entered the triage process. That means the issue will wait in this status until a Bug Marshal has an opportunity to review the issue. Once the issue has been reviewed you will receive comments regarding the next steps towards resolution. Please note that log messages and other files should not be sent to the Sangoma Asterisk Team unless explicitly asked for. All files should be placed on this issue in a sanitized fashion as needed. A good first step is for you to review the [Asterisk Issue Guidelines|https://wiki.asterisk.org/wiki/display/AST/Asterisk+Issue+Guidelines] if you haven't already. The guidelines detail what is expected from an Asterisk issue report. Then, if you are submitting a patch, please review the [Patch Contribution Process|https://wiki.asterisk.org/wiki/display/AST/Patch+Contribution+Process]. Please note that once your issue enters an open state it has been accepted. As Asterisk is an open source project there is no guarantee or timeframe on when your issue will be looked into. If you need expedient resolution you will need to find and pay a suitable developer. Asking for an update on your issue will not yield any progress on it and will not result in a response. All updates are posted to the issue when they occur. Please note that by submitting data, code, or documentation to Sangoma through JIRA, you accept the Terms of Use present at [https://www.asterisk.org/terms-of-use/|https://www.asterisk.org/terms-of-use/]. By: Joshua C. Colp (jcolp) 2021-01-15 04:24:10.860-0600 If you set a maximum size of the threadpool in the stasis.conf configuration file does that limit things as you'd like? By: Joshua C. Colp (jcolp) 2021-01-15 04:25:47.849-0600 Ah, actually it defaults to 50. I'm not sure what other things would have no limit on a threadpool, so I'll leave this in triage for others to look into. By: cagdas kopuz (cagdas) 2021-01-15 06:29:33.241-0600 [threadpool] max_size = 0 ; Maximum number of threads in the Stasis threadpool. ; ; 0 means no limit to the number of threads in the ; ; threadpool. My threadpool size is unlimited. By: Joshua C. Colp (jcolp) 2021-01-15 06:34:33.934-0600 You should set a limit, then. By: cagdas kopuz (cagdas) 2021-01-15 06:49:53.382-0600 There was a limit before, I increased it when I saw that there was an error. My main concern is, there are 5 servers with exactly the same features and settings(simultaneous call, and peer count), but other's thread count is about 2000. Only users are different. What causes the thread count to increase too much? How can i find it? By: Joshua C. Colp (jcolp) 2021-01-15 07:00:47.061-0600 A core debug log may show when threads are being created which may then give detail as to why. From a general sense something is obviously different - so usage patterns, AMI usage, database response time. By: Joshua C. Colp (jcolp) 2021-01-15 07:01:11.947-0600 You really do want to set a limit though on the stasis threadpool. By: Asterisk Team (asteriskteam) 2021-01-29 12:00:26.249-0600 Suspended due to lack of activity. This issue will be automatically re-opened if the reporter posts a comment. If you are not the reporter and would like this re-opened please create a new issue instead. If the new issue is related to this one a link will be created during the triage process. Further information on issue tracker usage can be found in the Asterisk Issue Guidlines [1]. [1] https://wiki.asterisk.org/wiki/display/AST/Asterisk+Issue+Guidelines | ||