| Summary: | ASTERISK-29374: res_prometheus: Crash when scraping channels | ||||
| Reporter: | BJ Weschke (bweschke) | Labels: | |||
| Date Opened: | 2021-03-29 13:02:11 | Date Closed: | 2021-03-31 10:12:57 | ||
| Priority: | Minor | Regression? | |||
| Status: | Closed/Complete | Components: | Resources/General | ||
| Versions: | 18.2.0 | Frequency of Occurrence | Occasional | ||
| Related Issues: |
| ||||
| Environment: | Ubuntu 18.04 | Attachments: | ( 0) core.tucana-2021-03-22T12-00-52-0400.tar.bz2 | ||
| Description: | Asterisk crash with the same ARI application and dial plan that was running for over 12-18 months with Asterisk 16. | ||||
| Comments: | By: Asterisk Team (asteriskteam) 2021-03-29 13:02:12.351-0500 Thanks for creating a report! The issue has entered the triage process. That means the issue will wait in this status until a Bug Marshal has an opportunity to review the issue. Once the issue has been reviewed you will receive comments regarding the next steps towards resolution. Please note that log messages and other files should not be sent to the Sangoma Asterisk Team unless explicitly asked for. All files should be placed on this issue in a sanitized fashion as needed. A good first step is for you to review the [Asterisk Issue Guidelines|https://wiki.asterisk.org/wiki/display/AST/Asterisk+Issue+Guidelines] if you haven't already. The guidelines detail what is expected from an Asterisk issue report. Then, if you are submitting a patch, please review the [Patch Contribution Process|https://wiki.asterisk.org/wiki/display/AST/Patch+Contribution+Process]. Please note that once your issue enters an open state it has been accepted. As Asterisk is an open source project there is no guarantee or timeframe on when your issue will be looked into. If you need expedient resolution you will need to find and pay a suitable developer. Asking for an update on your issue will not yield any progress on it and will not result in a response. All updates are posted to the issue when they occur. Please note that by submitting data, code, or documentation to Sangoma through JIRA, you accept the Terms of Use present at [https://www.asterisk.org/terms-of-use/|https://www.asterisk.org/terms-of-use/]. By: BJ Weschke (bweschke) 2021-03-29 13:03:38.995-0500 ast_coredumper output By: George Joseph (gjoseph) 2021-03-29 13:09:17.898-0500 How often does this happen? Were you using res_prometheus with Asterisk 16? By: BJ Weschke (bweschke) 2021-03-29 13:15:00.465-0500 We've had several crashes recorded over the past 10 days or so. We were not using res_prometheus with Asterisk 16. 18 is the first time we are using it. -rw------- 1 root root 629059584 Mar 19 18:55 core.tucana-2021-03-20T17:03:11-0400 -rw------- 1 root root 439373824 Mar 20 17:03 core.tucana-2021-03-22T12:00:52-0400 -rw------- 1 root root 275496960 Mar 22 12:01 core.tucana-2021-03-22T19:02:31-0400 -rw------- 1 root root 320806912 Mar 23 12:23 core.tucana-2021-03-23T18:33:08-0400 -rw------- 1 root root 185458688 Mar 24 09:52 core.tucana-2021-03-24T09:52:32-0400 -rw------- 1 root root 326733824 Mar 26 12:51 core.tucana-2021-03-26T18:38:38-0400 -rw------- 1 root root 530841600 Mar 27 21:36 core.tucana-2021-03-28T17:58:32-0400 By: George Joseph (gjoseph) 2021-03-29 13:31:48.763-0500 Can you re-run ast_coredumper with the --tarball-coredumps option? This will include the coredump itself as well as the asterisk binaries. The tarball will be fairly large so if you could upload it to DropBox, Google Drive, etc and send the link to asteriskteam@digium.com with ASTERISK-29374, that'd help a lot. By: George Joseph (gjoseph) 2021-03-29 13:32:04.577-0500 {panel:title=Private Submission of Information Disclaimer}You have indicated that you wish to submit unredacted information privately. It is not recommended to do this as it will substantially restrict the number of individuals who can help with your issue, as submitted information is only available to Sangoma. Note that submission of such information does not change the priorization of this issue. If you still wish to proceed you may do so by sending it to asteriskteam@digium.com with the issue number in the subject. For large files please send a link where they can then be downloaded. By sending this information you agree to the Website Terms of Use available on the Sangoma website at https://www.sangoma.com/legal/. Any exchange of private information between you and an Asterisk community member outside of the Asterisk JIRA is not subject to the Website Terms of Use and should be privately discussed between yourself and the Asterisk community member.{panel} By: BJ Weschke (bweschke) 2021-03-29 15:26:51.942-0500 Ok. I'll get them submitted up this evening and then email over the links to Google Drive as requested. By: George Joseph (gjoseph) 2021-03-30 07:05:37.286-0500 [~bweschke]I've got the files so you can delete them from Google Drive whenever you like. By: George Joseph (gjoseph) 2021-03-31 10:10:23.584-0500 I think what happened is the channel cache changed between the time we got the count of channels and the time we started iterating over it so we were trying to get info for a snapshot that didn't exist. By: George Joseph (gjoseph) 2021-03-31 10:12:57.505-0500 oops. I meant to close this as a dup of ASTERISK-29130. | ||||