[Home]

Summary:ASTERISK-24379: Testsuite: Process REF_DEBUG logs, fail any test that leaks
Reporter:Corey Farrell (coreyfarrell)Labels:
Date Opened:2014-10-01 03:26:06Date Closed:2014-10-30 22:23:20
Priority:MajorRegression?
Status:Closed/CompleteComponents:Tests/testsuite
Versions:SVN 11.13.0 12.6.0 13.0.0-beta2 Frequency of
Occurrence
Related
Issues:
Environment:Attachments:
Description:The testsuite should process REF_DEBUG logs if produced, fail any test where Asterisk leaked references.
Comments:By: Matt Jordan (mjordan) 2014-10-01 03:28:21.919-0500

I think this is a good start, but I also think we're a long way from being able to enable this globally.

We know that {{res_pjsip}} doesn't unload cleanly, which means that just having it loaded will cause any test to fail. I don't think that necessarily should hold up this improvement by any stretch, but I do think it means that this can't be enabled on the build agents until we get to a point where that isn't the case.
By: Corey Farrell (coreyfarrell) 2014-10-01 03:36:07.573-0500

Agreed, this would mean for Asterisk 12+ we would need to choose between {{\-\-enable REF_DEBUG \-\-disable res_pjsip}}, or {{\-\-disable REF_DEBUG}}.  That said I think it would be useful to enable on the build agents for Asterisk 11, any leaks found in that version would likely apply to 12+ as well.
By: Corey Farrell (coreyfarrell) 2014-10-01 18:27:45.951-0500

It's worth noting that this has the side effect of failing any test that does not successfully {{core stop gracefully}}.  I'm just ran testsuite with this patch for {{tests/channels/SIP}}.  The following is a list of leaked object count per instance of asterisk:
{code}
$ grep `find logs/channels/SIP -name refs.txt|sort` -e 'Leaked Object '|sed -e 's/:.*//'|uniq -c
 34059 logs/channels/SIP/hangupcause/run_1/ast1/var/log/asterisk/refs.txt
    11 logs/channels/SIP/message_from_call/run_1/ast1/var/log/asterisk/refs.txt
    10 logs/channels/SIP/noload_res_srtp_attempt_srtp/run_1/ast2/var/log/asterisk/refs.txt
 34063 logs/channels/SIP/noload_res_srtp/run_1/ast1/var/log/asterisk/refs.txt
 34062 logs/channels/SIP/noload_res_srtp/run_1/ast2/var/log/asterisk/refs.txt
    10 logs/channels/SIP/SDP_offer_answer/run_1/ast1/var/log/asterisk/refs.txt
 34066 logs/channels/SIP/secure_bridge_media/run_1/ast1/var/log/asterisk/refs.txt
 34046 logs/channels/SIP/secure_bridge_media/run_1/ast2/var/log/asterisk/refs.txt
 34059 logs/channels/SIP/sip_cause/run_1/ast1/var/log/asterisk/refs.txt
 34046 logs/channels/SIP/sip_srtp/run_1/ast1/var/log/asterisk/refs.txt
 34065 logs/channels/SIP/sip_srtp/run_1/ast2/var/log/asterisk/refs.txt
    11 logs/channels/SIP/subscribe/run_1/ast1/var/log/asterisk/refs.txt
{code}

Totals: 81 tests run, 4 failed due to actual reference leaks, 8 failed due to shutdown timeout, 28 skipped tests (I don't have pjsua or the correct sipp).  So it looks like we can't enable {{REF_DEBUG}} with this patch for the default build agent job even for Asterisk 11, as it will cause failures in every run that are due to testsuite errors rather than Asterisk leaks.  I do think there would be value in having an alternate job run against Asterisk 11 (maybe once per week) to give the community a published report of all tests that don't result in clean shutdown free of reference leaks.