Summary:	ASTERISK-29093: Asterisk crash on 16.13.0 in media file playback
Reporter:	BJ Weschke (bweschke)	Labels:
Date Opened:	2020-09-23 16:37:29	Date Closed:	2020-10-21 12:00:01
Priority:	Major	Regression?
Status:	Closed/Complete	Components:	Core/Stasis
Versions:	16.13.0	Frequency of Occurrence	One Time
Related Issues:
Environment:	Ubuntu 18.04.3 LTS	Attachments:	( 0) 202009231535-asterisk-16-13-0-crash.tar.bz2 ( 1) 202010021755-asterisk-16-13-0-crash.tar.bz2 ( 2) full-log-from-20201002-for-jira.log.bz2
Description:	Asterisk crashed after two days of runtime after updating to 16.13.0 on Monday of this week.
Comments:	By: Asterisk Team (asteriskteam) 2020-09-23 16:37:30.338-0500 Thanks for creating a report! The issue has entered the triage process. That means the issue will wait in this status until a Bug Marshal has an opportunity to review the issue. Once the issue has been reviewed you will receive comments regarding the next steps towards resolution. Please note that log messages and other files should not be sent to the Sangoma Asterisk Team unless explicitly asked for. All files should be placed on this issue in a sanitized fashion as needed. A good first step is for you to review the [Asterisk Issue Guidelines|https://wiki.asterisk.org/wiki/display/AST/Asterisk+Issue+Guidelines] if you haven't already. The guidelines detail what is expected from an Asterisk issue report. Then, if you are submitting a patch, please review the [Patch Contribution Process|https://wiki.asterisk.org/wiki/display/AST/Patch+Contribution+Process]. Please note that once your issue enters an open state it has been accepted. As Asterisk is an open source project there is no guarantee or timeframe on when your issue will be looked into. If you need expedient resolution you will need to find and pay a suitable developer. Asking for an update on your issue will not yield any progress on it and will not result in a response. All updates are posted to the issue when they occur. Please note that by submitting data, code, or documentation to Sangoma through JIRA, you accept the Terms of Use present at [https://www.asterisk.org/terms-of-use/|https://www.asterisk.org/terms-of-use/]. By: BJ Weschke (bweschke) 2020-09-23 16:39:39.310-0500 Files produced from /var/lib/asterisk/scripts/ast_coredumper core with our ARI credential masked. By: Joshua C. Colp (jcolp) 2020-09-23 17:02:00.073-0500 The crash appears to be memory corruption, and not strictly in ARI or playback code. What version was previously in use that worked fine and for how long? By: BJ Weschke (bweschke) 2020-09-23 17:10:53.368-0500 Asterisk GIT-16-c255dbd836 was the version previously in use which had been running without any issues since about March. We had just setup an ansible playbook to do periodic updates to later major/minor versions of Asterisk and this was the first tier we were rolling this out with. By: George Joseph (gjoseph) 2020-09-24 08:22:53.551-0500 Was this the only occurrence?  Were any other changes made to the system when the upgrade was done?  As [~jcolp] stated, this looks like a general memory corruption since the abort happened deep down in the system libraries.  If it happens again with  this release, post the ast_coredumper output again. By: BJ Weschke (bweschke) 2020-09-24 11:28:58.911-0500 Yes. This was the only occurrence so far. We will continue to keep an eye out and will repost if something else comes up. By: BJ Weschke (bweschke) 2020-10-02 17:19:21.418-0500 Another crash from earlier this evening. By: BJ Weschke (bweschke) 2020-10-02 17:20:25.001-0500 Just uploaded another new crash tarball that occurred a little over 20 mins ago. By: Joshua C. Colp (jcolp) 2020-10-05 05:52:59.584-0500 Do you have the console log leading up to this latest crash? It almost seems as though a stasis topic failed to allocate and wasn't properly handled. By: BJ Weschke (bweschke) 2020-10-05 10:04:07.438-0500 full log from 17:40 - 17:59 for 20201002 crash By: BJ Weschke (bweschke) 2020-10-05 10:04:48.909-0500 I gave a little bit of misinformation though with this second crash. I mistakenly reported that this was Asterisk 16.13.0 in the second crash. It is not. It is GIT-16-b3360fa5df. By: Joshua C. Colp (jcolp) 2020-10-07 04:56:28.879-0500 That version appears to be from February, since it's been so long it's not something we'd likely investigate unless it happened on a recent version. Has the other crash occurred under 16.13.0? By: Asterisk Team (asteriskteam) 2020-10-21 12:00:00.935-0500 Suspended due to lack of activity. This issue will be automatically re-opened if the reporter posts a comment. If you are not the reporter and would like this re-opened please create a new issue instead. If the new issue is related to this one a link will be created during the triage process. Further information on issue tracker usage can be found in the Asterisk Issue Guidlines [1]. [1] https://wiki.asterisk.org/wiki/display/AST/Asterisk+Issue+Guidelines