Summary:	ASTERISK-23244: [patch] testsuite: run-local fails to recognize modules
Reporter:	Corey Farrell (coreyfarrell)	Labels:
Date Opened:	2014-02-03 02:47:00.000-0600	Date Closed:	2014-02-10 12:50:03.000-0600
Priority:	Major	Regression?
Status:	Closed/Complete	Components:	Tests/testsuite
Versions:	SVN	Frequency of Occurrence	Constant
Related Issues:
Environment:		Attachments:	( 0) issueA23244_testsuite-run-locals-local-modules.patch ( 1) testsuite-run-local.patch ( 2) testsuite-run-local-r2.patch
Description:	When lib/python/asterisk/test_config.py checks for an asterisk module, it fails to adjust path by $AST_TEST_ROOT.  This prevents asterisk modules from being found from within run-local. Additionally when 'run-local setup' configures asterisk, it does not enable dev-mode, TEST_FRAMEWORK or DONT_OPTIMIZE.  This change can be avoided by running configure yourself before run-local setup, but a large number of tests require TEST_FRAMEWORK.  Since run-local doesn't currently work, now seems like a good time to set reasonable defaults.
Comments:	By: Walter Doekes (wdoekes) 2014-02-05 03:33:18.259-0600 The menuselect bit was ok. The path fix wasn't. The Dependency class should not have to know anything about environment variables. This updated patch may possibly fix or break other uses of .directories. That'll have to be looked over. By: Walter Doekes (wdoekes) 2014-02-05 03:35:52.892-0600 ... like these: {noformat} $ find . -name run-test -or -name \*.py | xargs grep -F .directories ./tests/channels/SIP/sip_tls_register/run-test:        self.keysdir = "%s%s" % (self.ast[0].base, self.ast[0].directories['astkeydir']) ./tests/channels/SIP/sip_tls_register/run-test:        self.etcdir = "%s%s" % (self.ast[0].base, self.ast[0].directories['astetcdir']) ./tests/cdr/cdr-tests.py:             self.test_object.ast[0].directories['astlogdir'], "cdrtest_local")) ./tests/cdr/cdr-tests.py:                 self.test_object.ast[0].directories['astlogdir'], ./tests/cdr/cdr-tests.py:                 self.test_object.ast[0].directories['astlogdir'], ./tests/cdr/sqlite3/cdr_sqlite3.py:                               ast_instance.directories['astetcdir'])) ./tests/cdr/sqlite3/cdr_sqlite3.py:                 ast_instance.directories['astlogdir'], ./tests/apps/mixmonitor_audiohook_inherit/run-test:            self.ast[0].directories['astspooldir'], "monitor/testaudio1.raw") ./tests/pbx/callerid_match/run-test:             (self.ast[0].base, self.ast[0].directories["astspooldir"])) ./tests/pbx/call-files/run-test:             (self.ast[0].base, self.ast[0].directories["astspooldir"])) ./tests/fax/sip/gateway_mix2/run-test:      shutil.copy("%s/send.tiff" % (os.path.dirname(os.path.realpath(__file__)),), "%s%s" % (self.ast[0].base, self.ast[0].directories['astdatadir'])) ./tests/fax/sip/voxfax_g711/run-test:        shutil.copy("%s/send.tiff" % (os.path.dirname(os.path.realpath(__file__)),), "%s%s" % (self.ast[1].base, self.ast[1].directories['astdatadir'])) ./tests/fax/sip/gateway_timeout2/run-test:      shutil.copy("%s/send.tiff" % (os.path.dirname(os.path.realpath(__file__)),), "%s%s" % (self.ast[0].base, self.ast[0].directories['astdatadir'])) ./tests/fax/sip/gateway_g711_t38/run-test:      shutil.copy("%s/send.tiff" % (os.path.dirname(os.path.realpath(__file__)),), "%s%s" % (self.ast[0].base, self.ast[0].directories['astdatadir'])) ./tests/fax/sip/gateway_mix3/run-test:      shutil.copy("%s/send.tiff" % (os.path.dirname(os.path.realpath(__file__)),), "%s%s" % (self.ast[0].base, self.ast[0].directories['astdatadir'])) ./tests/fax/sip/gateway_native_t38/run-test:      shutil.copy("%s/send.tiff" % (os.path.dirname(os.path.realpath(__file__)),), "%s%s" % (self.ast[0].base, self.ast[0].directories['astdatadir'])) ./tests/fax/sip/gateway_timeout1/run-test:      shutil.copy("%s/send.tiff" % (os.path.dirname(os.path.realpath(__file__)),), "%s%s" % (self.ast[0].base, self.ast[0].directories['astdatadir'])) ./tests/fax/sip/gateway_no_t38/run-test:      shutil.copy("%s/send.tiff" % (os.path.dirname(os.path.realpath(__file__)),), "%s%s" % (self.ast[0].base, self.ast[0].directories['astdatadir'])) ./tests/fax/sip/gateway_mix4/run-test:      shutil.copy("%s/send.tiff" % (os.path.dirname(os.path.realpath(__file__)),), "%s%s" % (self.ast[0].base, self.ast[0].directories['astdatadir'])) ./tests/fax/sip/gateway_timeout4/run-test:      shutil.copy("%s/send.tiff" % (os.path.dirname(os.path.realpath(__file__)),), "%s%s" % (self.ast[0].base, self.ast[0].directories['astdatadir'])) ./tests/fax/sip/gateway_timeout5/run-test:      shutil.copy("%s/send.tiff" % (os.path.dirname(os.path.realpath(__file__)),), "%s%s" % (self.ast[0].base, self.ast[0].directories['astdatadir'])) ./tests/fax/sip/gateway_mix1/run-test:      shutil.copy("%s/send.tiff" % (os.path.dirname(os.path.realpath(__file__)),), "%s%s" % (self.ast[0].base, self.ast[0].directories['astdatadir'])) ./tests/fax/sip/gateway_timeout3/run-test:      shutil.copy("%s/send.tiff" % (os.path.dirname(os.path.realpath(__file__)),), "%s%s" % (self.ast[0].base, self.ast[0].directories['astdatadir'])) ./tests/fax/sip/gateway_t38_g711/run-test:      shutil.copy("%s/send.tiff" % (os.path.dirname(os.path.realpath(__file__)),), "%s%s" % (self.ast[0].base, self.ast[0].directories['astdatadir'])) ./tests/fax/sip/gateway_native_t38_ced/run-test:      shutil.copy("%s/send.tiff" % (os.path.dirname(os.path.realpath(__file__)),), "%s%s" % (self.ast[0].base, self.ast[0].directories['astdatadir'])) ./tests/fax/pjsip/t38/run-test:      shutil.copy("%s/send.tiff" % (os.path.dirname(os.path.realpath(__file__)),), "%s%s" % (self.ast[0].base, self.ast[0].directories['astdatadir'])) ./tests/fax/pjsip/gateway_native_t38/run-test:      shutil.copy("%s/send.tiff" % (os.path.dirname(os.path.realpath(__file__)),), "%s%s" % (self.ast[0].base, self.ast[0].directories['astdatadir'])) ./tests/fax/pjsip/gateway_t38_g711/run-test:      shutil.copy("%s/send.tiff" % (os.path.dirname(os.path.realpath(__file__)),), "%s%s" % (self.ast[0].base, self.ast[0].directories['astdatadir'])) ./lib/python/asterisk/cdr.py:                     ast_instance.directories['astlogdir'], ./lib/python/asterisk/cel.py:                 self.test_object.ast[0].directories['astlogdir'], ./lib/python/asterisk/asterisk.py:        self.directories = {} ./lib/python/asterisk/asterisk.py:                    self.directories[var] = val ./lib/python/asterisk/voicemail.py:        self.voicemail_directory = (self.__ast.directories['astspooldir'] + ./lib/python/asterisk/voicemail.py:        file_path = (self.__ast.base + self.__ast.directories['astetcdir'] + ./lib/python/asterisk/test_config.py:        if "astmoddir" not in Dependency.ast.directories: ./lib/python/asterisk/test_config.py:        module = "%s/%s.so" % (Dependency.ast.directories["astmoddir"], name) {noformat} By: Corey Farrell (coreyfarrell) 2014-02-05 11:19:03.441-0600 Point taken about the environmental variable, it's actually saved to self.localtest_root in asterisk.py anyways. Unfortunately your change breaks all tests in run-local mode, it causes asterisk to fail startup (asterisk.conf has invalid paths).  I had some success prefixing with self.base instead of self.localtest_root.  This causes the asterisk.base + asterisk.directories['..'] to be double-prefixed.  I'll have more time to work on this tomorrow since it looks like this change cannot be done without retesting both local and non-local mode. By: Walter Doekes (wdoekes) 2014-02-05 13:56:11.256-0600 That's funny, because I ran a single test from run-local (to test that the patch fixed this modules bug) and that worked.. odd. It fails far from "all tests" over here. It mostly starts failing when we get to cdr's that are not found in one of those double paths. {noformat} $ ./run-local run Running tests for Asterisk SVN-branch-11-r403450 ... --> Running test 'tests/example' ... Making sure Asterisk isn't running ... Making sure SIPp isn't running... Running ['tests/example/run-test'] ... Checking to see if Asterisk is installed ... Asterisk not found! Test ['tests/example/run-test', 'SVN-branch-11-r403450'] failed --> tests/dynamic-modules ... skipped 'See ASTERISK-19567' --> Running test 'tests/manager/login' ... Making sure Asterisk isn't running ... Making sure SIPp isn't running... Running ['tests/manager/login/run-test'] ... Test ['tests/manager/login/run-test', 'SVN-branch-11-r403450'] passed --> Running test 'tests/manager/action-events-response' ... Making sure Asterisk isn't running ... Making sure SIPp isn't running... Running ['tests/manager/action-events-response/run-test'] ... testing with brokeneventsaction off (default) sending 'EventMask: ' sending 'EventMask: ON' sending 'EventMask: yes' sending 'EventMask: all' sending 'EventMask: all,user' sending 'EventMask: system,user,agent' sending 'EventMask: off' sending 'EventMask: none' sending 'EventMask: yeah whatever' sending 'EventMask: 1' testing with brokeneventsaction on sending 'EventMask: ' sending 'EventMask: ON' sending 'EventMask: yes' sending 'EventMask: all' sending 'EventMask: all,user' sending 'EventMask: system,user,agent' sending 'EventMask: off' sending 'EventMask: none' sending 'EventMask: yeah whatever' sending 'EventMask: 1' test passed Test ['tests/manager/action-events-response/run-test', 'SVN-branch-11-r403450'] passed --> Running test 'tests/manager/authlimit' ... Making sure Asterisk isn't running ... Making sure SIPp isn't running... Running ['tests/manager/authlimit/run-test'] ... starting asterisk connecting 5 clients to asterisk attempting to connect one more, this should fail connecting and authenticating 5 clients to asterisk attempting to connect one more, this should succeed test passed Test ['tests/manager/authlimit/run-test', 'SVN-branch-11-r403450'] passed --> Running test 'tests/manager/authtimeout' ... Making sure Asterisk isn't running ... Making sure SIPp isn't running... Running ['tests/manager/authtimeout/run-test'] ... starting asterisk testing timeout of an unauthenticated session testing timeout of an authenticated session (should not timeout) test passed Test ['tests/manager/authtimeout/run-test', 'SVN-branch-11-r403450'] passed --> Running test 'tests/manager/acl-login' ... Making sure Asterisk isn't running ... Making sure SIPp isn't running... Running ['tests/manager/acl-login/run-test'] ... [Feb 05 20:51:35] ERROR[23119]: __main__:92 ami_connect: AMI 1: Login was allowed and it shouldn't have been. [Feb 05 20:51:35] ERROR[23119]: __main__:92 ami_connect: AMI 2: Login was allowed and it shouldn't have been. [Feb 05 20:51:35] ERROR[23119]: __main__:92 ami_connect: AMI 3: Login was allowed and it shouldn't have been. [Feb 05 20:51:35] ERROR[23119]: __main__:92 ami_connect: AMI 9: Login was allowed and it shouldn't have been. [Feb 05 20:51:35] ERROR[23119]: __main__:92 ami_connect: AMI 10: Login was allowed and it shouldn't have been. [Feb 05 20:51:35] ERROR[23119]: __main__:92 ami_connect: AMI 11: Login was allowed and it shouldn't have been. [Feb 05 20:51:36] ERROR[23119]: __main__:149 evaluate_success: Not all logins that were expected to be allowed were completed. Missed: [5] [Feb 05 20:51:36] ERROR[23119]: __main__:153 evaluate_success: An AMI login was allowed which was not expected. Test ['tests/manager/acl-login/run-test', 'SVN-branch-11-r403450'] failed --> Cannot run test 'tests/manager/bridge_actions' --- --> Minimum Version: 12.0.0 (False) --- --> Tags: ['bridge'] --> Running test 'tests/manager/danger/dangerous' ... Making sure Asterisk isn't running ... Making sure SIPp isn't running... Running ['tests/manager/danger/dangerous/run-test'] ... Test ['tests/manager/danger/dangerous/run-test', 'SVN-branch-11-r403450'] passed --> Running test 'tests/manager/danger/safe' ... Making sure Asterisk isn't running ... Making sure SIPp isn't running... Running ['tests/manager/danger/safe/run-test'] ... [Feb 05 20:51:40] ERROR[23258]: __main__:38 on_get_var: Dangerous execution failed. Expected: '' Actual: 'hi' Test ['tests/manager/danger/safe/run-test', 'SVN-branch-11-r403450'] failed --> Running test 'tests/cdr/console_dial_sip_answer' ... Making sure Asterisk isn't running ... Making sure SIPp isn't running... Running ['./lib/python/asterisk/test_runner.py', 'tests/cdr/console_dial_sip_answer'] ... [Feb 05 20:51:44] ERROR[23321]: astcsv:101 __init__: IOError 2[No such file or directory] while opening file '/tmp/ast_test_B8OyYD/tmp/61308166a96b6a52379d2a1f00eec667/ast1//tmp/ast_test_B8OyYD/var/log/asterisk/cdr-csv/cdrtest_local.csv' [Feb 05 20:51:44] ERROR[23321]: astcsv:106 __init__: Unable to open file '/tmp/ast_test_B8OyYD/tmp/61308166a96b6a52379d2a1f00eec667/ast1//tmp/ast_test_B8OyYD/var/log/asterisk/cdr-csv/cdrtest_local.csv' [Feb 05 20:51:44] WARNING[23321]: astcsv:128 match: CSV MATCH FAILED, different number of records, self=1 and other=0 [Feb 05 20:51:44] ERROR[23321]: cdr:91 match_cdrs: cdrtest_local.csv: actual did not match expected. [Feb 05 20:51:44] ERROR[23321]: astcsv:101 __init__: IOError 2[No such file or directory] while opening file '/tmp/ast_test_B8OyYD/tmp/61308166a96b6a52379d2a1f00eec667/ast1//tmp/ast_test_B8OyYD/var/log/asterisk/cdr-csv/cdrtest_sip.csv' [Feb 05 20:51:44] ERROR[23321]: astcsv:106 __init__: Unable to open file '/tmp/ast_test_B8OyYD/tmp/61308166a96b6a52379d2a1f00eec667/ast1//tmp/ast_test_B8OyYD/var/log/asterisk/cdr-csv/cdrtest_sip.csv' [Feb 05 20:51:44] WARNING[23321]: astcsv:128 match: CSV MATCH FAILED, different number of records, self=1 and other=0 [Feb 05 20:51:44] ERROR[23321]: cdr:91 match_cdrs: cdrtest_sip.csv: actual did not match expected. Test ['./lib/python/asterisk/test_runner.py', 'tests/cdr/console_dial_sip_answer', 'SVN-branch-11-r403450'] failed --> Running test 'tests/cdr/console_dial_sip_busy' ... Making sure Asterisk isn't running ... Making sure SIPp isn't running... Running ['./lib/python/asterisk/test_runner.py', 'tests/cdr/console_dial_sip_busy'] ... [Feb 05 20:51:48] ERROR[23389]: astcsv:101 __init__: IOError 2[No such file or directory] while opening file '/tmp/ast_test_B8OyYD/tmp/fba94a8c9553442b2fd5822586fad31e/ast1//tmp/ast_test_B8OyYD/var/log/asterisk/cdr-csv/cdrtest_local.csv' [Feb 05 20:51:48] ERROR[23389]: astcsv:106 __init__: Unable to open file '/tmp/ast_test_B8OyYD/tmp/fba94a8c9553442b2fd5822586fad31e/ast1//tmp/ast_test_B8OyYD/var/log/asterisk/cdr-csv/cdrtest_local.csv' [Feb 05 20:51:48] WARNING[23389]: astcsv:128 match: CSV MATCH FAILED, different number of records, self=1 and other=0 [Feb 05 20:51:48] ERROR[23389]: cdr:91 match_cdrs: cdrtest_local.csv: actual did not match expected. Test ['./lib/python/asterisk/test_runner.py', 'tests/cdr/console_dial_sip_busy', 'SVN-branch-11-r403450'] failed {noformat} By: Corey Farrell (coreyfarrell) 2014-02-05 14:41:18.642-0600 Your right, I'm not sure what I did earlier.  I just reapplied your patch and it seems to work for most tests I've tried. By: Corey Farrell (coreyfarrell) 2014-02-06 10:39:23.449-0600 testsuite-run-local-r2.patch is what I believe is the lowest impact way to get run-local working. Prefixing .directories with self.localtest_root caused problems since it's included in self.base.  It made it impossible to fix the tests that use ast.base + ast.directories["..."]. Module dependencies are checked before the asterisk instance folder structure is created, so it has to be treated differently than .directories for run-local but not runtests.py.  This is why I created self.original_astmoddir. I've tested this with tests/channels/SIP and tests/cdr in both local and system mode. By: Walter Doekes (wdoekes) 2014-02-06 14:55:26.690-0600 Looks good to me!