| Summary: | ASTERISK-28568: Potential memory leak on core reload | ||
| Reporter: | Michael Maier (micha) | Labels: | |
| Date Opened: | 2019-10-06 08:04:33 | Date Closed: | 2019-10-21 12:00:03 |
| Priority: | Minor | Regression? | |
| Status: | Closed/Complete | Components: | Configs/Basic-PBX |
| Versions: | 16.5.0 | Frequency of Occurrence | Constant |
| Related Issues: | |||
| Environment: | CentOS 7 64Bit x86_64 | Attachments: | ( 0) etc_asterisk.tar.gz ( 1) memory-consumption-reload.csv |
| Description: | Executing "core reload"s seem to consume more and more memory.
This is an example done with the attached simple configuration (based on FreePBX 14). Memory usage is measured with ps aux | grep sbin/asterisk | grep -v grep The memory value in the attached sheet is based on the RSS column of ps. Most reloads are done without any change. If there are more trunks / extensions, routes, ... the memory usage is growing much faster. | ||
| Comments: | By: Asterisk Team (asteriskteam) 2019-10-06 08:04:35.200-0500 Thanks for creating a report! The issue has entered the triage process. That means the issue will wait in this status until a Bug Marshal has an opportunity to review the issue. Once the issue has been reviewed you will receive comments regarding the next steps towards resolution. A good first step is for you to review the [Asterisk Issue Guidelines|https://wiki.asterisk.org/wiki/display/AST/Asterisk+Issue+Guidelines] if you haven't already. The guidelines detail what is expected from an Asterisk issue report. Then, if you are submitting a patch, please review the [Patch Contribution Process|https://wiki.asterisk.org/wiki/display/AST/Patch+Contribution+Process]. Please note that once your issue enters an open state it has been accepted. As Asterisk is an open source project there is no guarantee or timeframe on when your issue will be looked into. If you need expedient resolution you will need to find and pay a suitable developer. Asking for an update on your issue will not yield any progress on it and will not result in a response. All updates are posted to the issue when they occur. By: Michael Maier (micha) 2019-10-06 08:09:32.647-0500 It's a csv file containing the memory usage after each core reload. By: Michael Maier (micha) 2019-10-06 08:10:31.046-0500 That's the config based on FreePBX 14 By: George Joseph (gjoseph) 2019-10-07 06:26:49.625-0500 After so many reloads, 18Kb isn't really unreasonable. With a lot of activity, the memory allocated to Asterisk by the operating system becomes fragmented into smaller contiguous chunks so if Asterisk needs a larger chunk than is available, the glibc runtime has to ask the operating system for more memory. Try this... If you configure Asterisk with --enable-devmode there should be a CLI command called "malloc trim". What it does is try to return free memory to the operating system. Run it a few times in quick succession then check the resident memory utilization and see what happens. By: Corey Farrell (coreyfarrell) 2019-10-07 09:55:00.215-0500 [~gjoseph] the numbers are in kb so that's actually an 18mb increase. That said we're looking at RSS which shows the amount of memory that is actually in RAM (excluding swapped memory). Normally the reload functions would not need to be in RSS but executing `core reload` causes the reload functions of every module to execute, thus preventing them from being swapped out. I don't think RSS is useful for identifying memory leaks since it doesn't tell us how much we've allocated, it tells us what the kernel thinks should not be swapped. If VSZ were to increase by significant amounts after each reload that could be interesting, though not helpful to isolate the cause. Since core reload acts on the whole system we would need information provided by MALLOC_DEBUG or other output which can identify the source of different allocations. Note some increase of VSZ after the first could reloads is expected, this is the result of memory fragmentation (in stringfields for example). By: George Joseph (gjoseph) 2019-10-07 10:08:38.739-0500 Ha yeah, 18mb. Sorry! By: Asterisk Team (asteriskteam) 2019-10-21 12:00:02.991-0500 Suspended due to lack of activity. This issue will be automatically re-opened if the reporter posts a comment. If you are not the reporter and would like this re-opened please create a new issue instead. If the new issue is related to this one a link will be created during the triage process. Further information on issue tracker usage can be found in the Asterisk Issue Guidlines [1]. [1] https://wiki.asterisk.org/wiki/display/AST/Asterisk+Issue+Guidelines | ||