[Home]

Summary:ASTERISK-24241: crash: CDRs recursively attempt to update Party B information in a multi-party bridge, overrunning the stack
Reporter:Deepak Singh Rawat (dsr)Labels:
Date Opened:2014-08-17 08:43:37Date Closed:2014-09-05 17:06:25
Priority:CriticalRegression?
Status:Closed/CompleteComponents:CDR/General
Versions:12.4.0 Frequency of
Occurrence
Frequent
Related
Issues:
is related toASTERISK-24208 Channels with CDR Information Remain Active Even After ConfBrige Is Ended
Environment:uname -a: Linux 2.6.32-358.el6.x86_64 #1 SMP Fri Feb 22 00:31:26 UTC 2013 x86_64 x86_64 x86_64 GNU/Linux OS: CentOS release 6.4 (Final) Memory: 6 GBAttachments:( 0) ASTERISK-24241-12.diff
( 1) backtrace.txt
( 2) segfaultLog.txt
Description:PFA the backtrace. Asterisk 12.4 crashes randomly. For some reason it's also using a lot of memory. For just 50 channels it eats up most of the system memory. We are using ConfBridge for conferencing. Please let me know if you need any other details.
Comments:By: Matt Jordan (mjordan) 2014-08-17 16:05:20.577-0500

CDRs are apparently getting into an infinite recursion. That'd be a "bad" thing.

Please get a DEBUG log with 'cdr set debug on' enabled. Instructions on getting a DEBUG log can be found on the wiki:

https://wiki.asterisk.org/wiki/display/AST/Collecting+Debug+Information

By: Deepak Singh Rawat (dsr) 2014-08-17 16:27:28.528-0500

Matt should I reproduce the crash and then send the debug log or can I send it by running only few calling sessions?

By: Deepak Singh Rawat (dsr) 2014-08-17 16:47:09.897-0500

PFA debug log after cdr debugging was enabled. I have masked the number getting called.

By: Deepak Singh Rawat (dsr) 2014-08-17 16:54:52.420-0500

I have added the cdr debug log.

By: Deepak Singh Rawat (dsr) 2014-08-20 17:01:09.116-0500

[~mjordan] Any update on this issue? Could it be due to a bug in our dialplan? Will upgrading to 12.5 help?

By: Rusty Newton (rnewton) 2014-08-26 09:35:46.308-0500

Did you collect the log running all the way up to the crash?

Is the backtrace posted the backtrace that correlates with the log?

By: Deepak Singh Rawat (dsr) 2014-08-26 09:48:55.929-0500

[~rnewton] I created the backtrace from the crash dump created by asterisk when it crashed. The segfaultLog.txt contains part of the log generated when I enabled cdr debugging.

By: Matt Jordan (mjordan) 2014-09-01 10:57:18.387-0500

The patch attached (ASTERISK-24241-12.diff) should fix this issue. It prevents the CDR code from getting stuck in an infinite loop of generating more CDRs when a channel:
* Enters a multi-party bridge
* Leaves said bridge
* Re-enters the bridge

By: Matt Jordan (mjordan) 2014-09-01 10:57:36.392-0500

If you can, please test the attached patch and let me know if it fixes the issue. Thanks!

By: Deepak Singh Rawat (dsr) 2014-09-01 11:02:14.783-0500

Thank you very much team for the patch! I will apply it today and will send you feedback in a couple of days after monitoring the usage. Along with the crash I also noticed unusually high memory usage by Asterisk. I assumed that it's due to this bug only. If the memory usage comes down and I see no crashes then I will assume that his patch worked.

By: Matt Jordan (mjordan) 2014-09-01 11:40:25.123-0500

Since it was generating additional CDRs like crazy (until things crashed), then yes, you'd see memory get consumed very quickly without this patch. That doesn't mean there can't be some other issue at play if you do observe memory increasing still - depending on the symptoms, that may be a different issue altogether.

Thanks for testing!

By: Matt Jordan (mjordan) 2014-09-05 16:47:35.630-0500

Just as an FYI - other folks were seeing this issue, so I've gone ahead and merged it into the branch. If you still have a problem, please comment here and I'll be happy to re-open the issue.

By: Deepak Singh Rawat (dsr) 2014-09-06 01:35:25.483-0500

[~mjordan] Matt since I applied this patch asterisk has not crashed. I have also noticed significant decrease in memory usage. I think this patch has fixed the issue we were facing.