[Home]

Summary:ASTERISK-28916: Memory leak with Asterisk 16 and malformed REGISTER requests
Reporter:nappsoft (nappsoft)Labels:
Date Opened:2020-05-25 01:44:16Date Closed:2020-05-25 04:42:52
Priority:MajorRegression?
Status:Closed/CompleteComponents:Resources/res_pjsip
Versions:16.7.0 16.9.0 16.10.0 Frequency of
Occurrence		Frequent
Related
Issues:
Environment:Attachments:( 0) mmlog
Description:We had several asterisk systems running out of memory yesterday during a DOS attack. All of these systems were running with Asterisk >=16.7.0, some with PJSIP 2.9, some already with 2.10

Other machines with asterisk 13 have been attacked as well, however without running into any troubles. (That's why I guess that it's rather an asterisk issue than a PJSIP issue).

The register messages with which the systems got attacked were obviously broken and looked like bellow (IP replaced with xx.xx). What should be noticed:

- there was no CRLF after the headers
- the Content-Type of the register is set to applicatoin/sdp
- the user-agent was empty

REGISTER sip:220@x.x.x.x SIP/2.0
To: 220 <sip:220@x.x.x.x>
From:  <sip:220@x.x.x.x>;tag=0c26cd11
Via: SIP/2.0/UDP x.x.x.x:53716;branch=s8rinbit1zv039o5imke6y3vyvi91fizpvjnepn6l3kh7a9u6t2isdw89uhuqui2hb825f5;rport
Call-ID: e08c2ff23aa7495abd86575f1a294b1b
CSeq: 1 REGISTER
Contact: <sip:220@x.x.x.x:53716>
User-Agent:
Max-forwards: 70
Allow: INVITE, ACK, CANCEL, BYE, REFER
Content-Type: application/sdp

Comments:By: Asterisk Team (asteriskteam) 2020-05-25 01:44:17.511-0500

Thanks for creating a report! The issue has entered the triage process. That means the issue will wait in this status until a Bug Marshal has an opportunity to review the issue. Once the issue has been reviewed you will receive comments regarding the next steps towards resolution.

A good first step is for you to review the [Asterisk Issue Guidelines|https://wiki.asterisk.org/wiki/display/AST/Asterisk+Issue+Guidelines] if you haven't already. The guidelines detail what is expected from an Asterisk issue report.

Then, if you are submitting a patch, please review the [Patch Contribution Process|https://wiki.asterisk.org/wiki/display/AST/Patch+Contribution+Process].

Please note that once your issue enters an open state it has been accepted. As Asterisk is an open source project there is no guarantee or timeframe on when your issue will be looked into. If you need expedient resolution you will need to find and pay a suitable developer. Asking for an update on your issue will not yield any progress on it and will not result in a response. All updates are posted to the issue when they occur.
By: nappsoft (nappsoft) 2020-05-25 03:21:17.642-0500

I just tested a bit with a script sending register requests to the system with the following results:

- memory usage seems to grow quite fast when I am able to send enough register requests to the system so that asterisk consumes 100% cpu (so the queue for answering the register requests grows more and more I guess)
- I am able to let asterisk consume 400mb after 20-30 seconds sending register requests to the system
- with asterisk 13 I do not manage to let asterisk consume more than 100mb, not even when sending the requests from localhost

So it seems like asterisk 16 would need by far more performance for register requests?
By: nappsoft (nappsoft) 2020-05-25 04:12:13.741-0500

Attached the output with MALLOC_DEBUG enabled.

Please note: this is a testsystem, that is not busy:

- 40 aors
- currently only one registered phone
- two outbound registrations

However: with MALLOC_DEBUG enabled I am not able to get a similar excessive memory usage (even though I manage asterisk to use 100% cpu as well), it somehow stops at about 140mb usage while it will go up till OOM without MALLOC_DEBUG, don't know why...
By: nappsoft (nappsoft) 2020-05-25 04:42:30.332-0500

You can suspend this for now, I am sorry for the noise: it seems like I could work around this issue when using malloc-ng with musl libc rather than the old malloc (for whatever reason).