KaneAI - World’s First E2E Software Testing Agent.Book a Demo Join TestMu Conference

How to use test_external_network_visibility method in tempest

Best Python code snippet using tempest_python

exclude_tests.py

Source:exclude_tests.py

1#!/usr/bin/python2import click3import subprocess4JUJU='juju'5JUJU_RC='admin-openrc.sh'6DIRECTOR='director'7DIRECTOR_RC='overcloudrc'8# Dictionary that shows which tempest tests to exclude, by release9excluded_test_matrix = {10    'stable/newton': [11        ('tempest.api.network.test_floating_ips.FloatingIPTestJSON.test_floating_ip_update_different_router',12         'Test uses overlapping CIDRs in the same VRF'),13        ('tempest.api.network.test_networks.NetworksTest.test_external_network_visibility',14         'BUG: https://github.com/noironetworks/support/issues/916'),15        ('tempest.api.network.test_security_groups.SecGroupTest.test_create_show_delete_security_group_rule',16         'BUG: https://github.com/noironetworks/support/issues/710'),17        ('tempest.scenario.test_network_basic_ops.TestNetworkBasicOps.test_subnet_details',18         "BIG: https://github.com/noironetworks/support/issues/915"),19        ('tempest.scenario.test_network_basic_ops.TestNetworkBasicOps.test_update_router_admin_state',20         'BUG: https://github.com/noironetworks/support/issues/491'),21        ('neutron.tests.tempest.api.test_floating_ips_negative.FloatingIPNegativeTestJSON.test_associate_floatingip_with_port_with_floatingip',22         'Test uses overlapping CIDRs in the same VRF'),23        ('neutron.tests.tempest.api.test_timestamp.TestTimeStampWithSecurityGroup.test_create_sg_with_timestamp',24         'BUG: https://github.com/noironetworks/support/issues/710'),25        ('neutron.tests.tempest.api.test_timestamp.TestTimeStampWithSecurityGroup.test_create_sgrule_with_timestamp',26         'BUG: https://github.com/noironetworks/support/issues/710'),27        ('neutron.tests.tempest.api.test_timestamp.TestTimeStampWithSecurityGroup.test_show_sg_attribute_with_timestamp',28         'BUG: https://github.com/noironetworks/support/issues/710'),29        ('neutron.tests.tempest.api.test_timestamp.TestTimeStampWithSecurityGroup.test_show_sgrule_attribute_with_timestamp',30         'BUG: https://github.com/noironetworks/support/issues/710'),31        ('neutron.tests.tempest.api.test_timestamp.TestTimeStampWithSecurityGroup.test_update_sg_with_timestamp',32         'BUG: https://github.com/noironetworks/support/issues/710')33    ],34    'stable/ocata': [35        ('tempest.api.network.test_floating_ips.FloatingIPTestJSON.test_floating_ip_update_different_router',36         'Test uses overlapping CIDRs in the same VRF'),37        ('tempest.api.network.test_networks.NetworksTest.test_external_network_visibility',38         'BUG: https://github.com/noironetworks/support/issues/916'),39        ('tempest.api.network.test_security_groups.SecGroupTest.test_create_show_delete_security_group_rule',40         'BUG: https://github.com/noironetworks/support/issues/710'),41        ('tempest.scenario.test_network_basic_ops.TestNetworkBasicOps.test_subnet_details',42         "BIG: https://github.com/noironetworks/support/issues/915"),43        ('tempest.scenario.test_network_basic_ops.TestNetworkBasicOps.test_update_router_admin_state',44         'BUG: https://github.com/noironetworks/support/issues/491'),45        ('neutron.tests.tempest.api.test_floating_ips_negative.FloatingIPNegativeTestJSON.test_associate_floatingip_with_port_with_floatingip',46         'Test uses overlapping CIDRs in the same VRF'),47        ('neutron.tests.tempest.api.test_timestamp.TestTimeStampWithSecurityGroup.test_create_sg_with_timestamp',48         'BUG: https://github.com/noironetworks/support/issues/710'),49        ('neutron.tests.tempest.api.test_timestamp.TestTimeStampWithSecurityGroup.test_create_sgrule_with_timestamp',50         'BUG: https://github.com/noironetworks/support/issues/710'),51        ('neutron.tests.tempest.api.test_timestamp.TestTimeStampWithSecurityGroup.test_show_sg_attribute_with_timestamp',52         'BUG: https://github.com/noironetworks/support/issues/710'),53        ('neutron.tests.tempest.api.test_timestamp.TestTimeStampWithSecurityGroup.test_show_sgrule_attribute_with_timestamp',54         'BUG: https://github.com/noironetworks/support/issues/710'),55        ('neutron.tests.tempest.api.test_timestamp.TestTimeStampWithSecurityGroup.test_update_sg_with_timestamp',56         'BUG: https://github.com/noironetworks/support/issues/710')57    ],58    'stable/pike': [59        ('tempest.api.network.test_floating_ips.FloatingIPTestJSON.test_floating_ip_update_different_router',60         'Test uses overlapping CIDRs in the same VRF'),61        ('tempest.api.network.test_networks.NetworksTest.test_external_network_visibility',62         'BUG: https://github.com/noironetworks/support/issues/916'),63        ('tempest.api.network.test_security_groups.SecGroupTest.test_create_show_delete_security_group_rule',64         'BUG: https://github.com/noironetworks/support/issues/710'),65        ('tempest.scenario.test_network_basic_ops.TestNetworkBasicOps.test_subnet_details',66         "BIG: https://github.com/noironetworks/support/issues/915"),67        ('tempest.scenario.test_network_basic_ops.TestNetworkBasicOps.test_update_router_admin_state',68         'BUG: https://github.com/noironetworks/support/issues/491'),69        ('neutron.tests.tempest.api.test_floating_ips_negative.FloatingIPNegativeTestJSON.test_associate_floatingip_with_port_with_floatingip',70         'Test uses overlapping CIDRs in the same VRF'),71        ('neutron.tests.tempest.api.test_timestamp.TestTimeStampWithSecurityGroup.test_create_sg_with_timestamp',72         'BUG: https://github.com/noironetworks/support/issues/710'),73        ('neutron.tests.tempest.api.test_timestamp.TestTimeStampWithSecurityGroup.test_create_sgrule_with_timestamp',74         'BUG: https://github.com/noironetworks/support/issues/710'),75        ('neutron.tests.tempest.api.test_timestamp.TestTimeStampWithSecurityGroup.test_show_sg_attribute_with_timestamp',76         'BUG: https://github.com/noironetworks/support/issues/710'),77        ('neutron.tests.tempest.api.test_timestamp.TestTimeStampWithSecurityGroup.test_show_sgrule_attribute_with_timestamp',78         'BUG: https://github.com/noironetworks/support/issues/710'),79        ('neutron.tests.tempest.api.test_timestamp.TestTimeStampWithSecurityGroup.test_update_sg_with_timestamp',80         'BUG: https://github.com/noironetworks/support/issues/710')81    ],82    'stable/queens': [83        ('tempest.scenario.test_network_basic_ops.TestNetworkBasicOps.test_subnet_details',84         "BIG: https://github.com/noironetworks/support/issues/915"),85        ('tempest.scenario.test_network_basic_ops.TestNetworkBasicOps.test_update_router_admin_state',86         'BUG: https://github.com/noironetworks/support/issues/491'),87        ('tempest.scenario.test_network_basic_ops.TestNetworkBasicOps.test_port_security_macspoofing_port',88         'Why is this failing?'),89        ('tempest.api.network.test_floating_ips.FloatingIPTestJSON.test_floating_ip_update_different_router',90         'Test uses overlapping CIDRs in the same VRF'),91        ('tempest.api.compute.servers.test_attach_interfaces.AttachInterfacesTestJSON.test_create_list_show_delete_interfaces_by_fixed_ip',92         'Upstream bug: https://bugs.launchpad.net/tempest/+bug/1790864'),93        ('neutron_tempest_plugin.api.test_revisions.TestRevisions.test_update_network_constrained_by_revision',94         'Why is this test failing?'),95        ('neutron_tempest_plugin.api.test_timestamp.TestTimeStampWithL3.test_show_floatingip_attribute_with_timestamp',96         'Why is this test failing?'),97        ('neutron_tempest_plugin.scenario.test_connectivity.NetworkConnectivityTest.test_connectivity_through_2_routers',98         'The apic_aim mechanism driver does not support transit routes (connect networks to same neutron router instead)'),99        ('neutron_tempest_plugin.scenario.test_internal_dns.InternalDNSTest.test_dns_domain_and_name',100         'This is an upstream bug (non-optimized has same failure)')101    ],102}103class Excluder(object):104    def __init__(self, undercloud_type=DIRECTOR):105        if undercloud_type == DIRECTOR:106            self.KEY = 'source ~/' + DIRECTOR_RC + ' && '107        elif undercloud_type == JUJU:108            self.KEY = 'source ~/' + JUJU_RC + ' && '109        self.KEY = 'source ~/' + DIRECTOR_RC + ' && '110        self.version = self.get_openstack_version()111    def get_openstack_version(self):112        # we rely on the openstack client repo113        cmd1 = "cd python-openstackclient/ && "114        cmd2 = "git status | grep 'branch stable\|eol' "115        cmd3 = "| awk 'NF>1{print $NF}' && cd .."116        cmd = self.KEY + cmd1 + cmd2 + cmd3117        print cmd118        subprocess.check_output(['bash','-c', cmd])119        version_string = subprocess.check_output(['bash','-c', cmd])120        version_list = [version.strip()121                        for version in version_string.split("\n") if version]122        version = version_list[0]123        if version == 'newton-eol':124            version = 'stable/newton'125        return version126        127    def get_test_info(self, test):128        test_pieces = test.split(".")129        path = "-".join(test_pieces[0].split("_")) + "/" + "/".join(test_pieces[0:-2]) + ".py"130        test_class = test_pieces[-2:-1][0]131        test_name = test.split(".")[-1:][0]132        return path, test_class, test_name133    def exclude_tests(self):134        for test, reason in excluded_test_matrix[self.version]:135            path, test_class, test_name = self.get_test_info(test)136            def add_line(file_name, before_line, new_line):137                # find the first instance of the string138                index = None139                all_lines = None140                with open(file_name, "r") as fd:141                    all_lines = fd.readlines()142                for count in range(len(all_lines)):143                    line = all_lines[count]144                    if before_line in line:145                        strip_line = line[:-1]146                        indent_len = len(strip_line) - len(strip_line.strip())147                        indent = line[:indent_len] if indent_len else ''148                        new_line = indent + new_line149                        index = count150                        break151                if index:152                    all_lines.insert(index, new_line)153                    with open(file_name, "w+") as fd:154                        contents = "".join(all_lines)155                        fd.write(contents)156      157            cmd = "egrep testtools %s" % path158            try:159                subprocess.check_output(['bash','-c', cmd])160            except Exception as e:161                # not present, so add testtools162                add_line(path, "import", "import testtools\n")163                pass164            # need to escape backslashes165            166            reason_str = '@testtools.skip("' + reason + '")\n'167            # Skip this test168            add_line(path, 'def ' + test_name, reason_str)169@click.command()170@click.option('--undercloud-type', default='director',171              help='Type of undercloud (juju or director)')172def exclude_tests(undercloud_type):173    excluder = Excluder(undercloud_type=undercloud_type)174    excluder.exclude_tests()175    click.echo("Iniitialization complete, and tempest config generated")176if __name__ == '__main__':...

test_bugcheck.py

Source:test_bugcheck.py

1# Licensed under the Apache License, Version 2.0 (the "License");2# you may not use this file except in compliance with the License.3# You may obtain a copy of the License at4#5#    http://www.apache.org/licenses/LICENSE-2.06#7# Unless required by applicable law or agreed to in writing, software8# distributed under the License is distributed on an "AS IS" BASIS,9# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or10# implied.11# See the License for the specific language governing permissions and12# limitations under the License.13#14# flake8: noqa15# If extensions (or modules to document with autodoc) are in another directory,16# add these directories to sys.path here. If the directory is relative to the17# documentation root, use os.path.abspath to make it absolute, like shown here.18from unittest import mock19import os20import tempfile21import unittest22import xmlrpclib23from bugcheck import BugVerifyCmd24from bugcheck import BugzillaConnector25from bugcheck import CLOSED26from bugcheck import INVALID27from bugcheck import LaunchpadConnector28from bugcheck import OPEN29from bugcheck import VerifyBug30class TestLaunchpadConnector(unittest.TestCase):31    @mock.patch('launchpadlib.launchpad.Launchpad.login_anonymously')32    def test_get_bug_status(self, launchpad_mock):33        lp_connector = LaunchpadConnector()34        bugs = launchpad_mock.return_value.bugs35        bug_tasks = bugs.__getitem__().bug_tasks36        item = bug_tasks.__getitem__()37        for status in ['Fix Released', 'Fix Committed', 'Invalid']:38            item.status = status39            self.assertEqual(lp_connector.get_bug_status(1693838), CLOSED)40        item.status = 'No idea'41        self.assertEqual(lp_connector.get_bug_status(1693838), OPEN)42        bugs.__getitem__.side_effect = KeyError()43        self.assertEqual(lp_connector.get_bug_status(1693838), INVALID)44class TestBugzillaConnector(unittest.TestCase):45    @mock.patch('bugzilla.Bugzilla')46    def test_get_bug_status(self, bugzilla_mock):47        bz_connector = BugzillaConnector()48        bug = bugzilla_mock.return_value.getbug49        bug.return_value.status = 'CLOSED'50        self.assertEqual(bz_connector.get_bug_status(123), CLOSED)51        bz_status = ['ASSIGNED', 'NEEDINFO', 'NEW', 'REOPENED', 'RESOLVED',52                     'UNCONFIRMED', 'VERIFIRED']53        for status in bz_status:54            bug.return_value.status = status55            self.assertEqual(bz_connector.get_bug_status(123), OPEN)56        bug.side_effect = xmlrpclib.Fault(faultCode=102,57                                          faultString='Permission')58        self.assertEqual(bz_connector.get_bug_status(123), OPEN)59        bug.side_effect = xmlrpclib.Fault(faultCode=42,60                                          faultString='Other fault')61        self.assertEqual(bz_connector.get_bug_status(123), INVALID)62class TestVerifyBug(unittest.TestCase):63    @mock.patch('launchpadlib.launchpad.Launchpad.login_anonymously')64    @mock.patch('bugzilla.Bugzilla')65    def setUp(self, bz_mock, lp_mock):66        self.v_bug = VerifyBug()67    def test__get_id_from_url(self):68        self.assertEqual(self.v_bug._get_id_from_url(69            'https://bugs.launchpad.net/tripleo/+bug/1577769'), 1577769)70        self.assertEqual(self.v_bug._get_id_from_url(71            'https://bugzilla.redhat.com/show_bug.cgi?id=1380187'), 1380187)72    def test__get_connector(self):73        self.assertIsInstance(self.v_bug._get_connector(74            'https://bugs.launchpad.net/tripleo/+bug/1577769'),75            LaunchpadConnector)76        self.assertIsInstance(self.v_bug._get_connector(77            'https://bugzilla.redhat.com/show_bug.cgi?id=1380187'),78            BugzillaConnector)79        self.assertRaises(ValueError, self.v_bug._get_connector,80                          'https://review.opendev.org')81    @mock.patch('bugcheck.VerifyBug.check_bug_status')82    def test_is_bug_open(self, bug_status_mock):83        for status in [CLOSED, INVALID]:84            bug_status_mock.return_value = status85            self.assertEqual(self.v_bug.is_bug_open(86                'https://bugzilla.redhat.com/show_bug.cgi?id=1380187'), False)87        bug_status_mock.return_value = OPEN88        self.assertEqual(self.v_bug.is_bug_open(89            'https://bugzilla.redhat.com/show_bug.cgi?id=1380187'), True)90class TestBugVerifyCmd(unittest.TestCase):91    def setUp(self):92        self.fd_file, self.tmp_file = tempfile.mkstemp()93        self._populate_skip_file()94        self.known_failures = [95            {'test': '.*test_external_network_visibility',96             'reason': 'Tempest test "external network visibility" fails',97             'lp': 'https://bugs.launchpad.net/tripleo/+bug/1577769'},98            {'test': 'tempest.api.data_processing',99             'reason': 'tempest.api.data_processing tests failing on newton',100             'bz': 'https://bugzilla.redhat.com/show_bug.cgi?id=1357667'},101            {'test': 'neutron.tests.tempest.api.test_revisions.TestRevisions',102             'reason': 'New test, need investigation'}]103        self.txt_output = ('# Tempest test "external network visibility" '104                           'fails\n'105                           '.*test_external_network_visibility\n'106                           '# tempest.api.data_processing tests failing on '107                           'newton\n'108                           'tempest.api.data_processing\n'109                           '# New test, need investigation\n'110                           'neutron.tests.tempest.api.test_revisions.'111                           'TestRevisions\n')112        self.yaml_output = ('---\nknown_failures:\n'113                            '- lp: https://bugs.launchpad.net/tripleo/+bug/'114                            '1577769\n'115                            '  reason: Tempest test "external network '116                            'visibility" fails\n'117                            '  test: .*test_external_network_visibility\n'118                            '- bz: https://bugzilla.redhat.com/show_bug.cgi'119                            '?id=1357667\n'120                            '  reason: tempest.api.data_processing tests '121                            'failing on newton\n'122                            '  test: tempest.api.data_processing\n'123                            '- reason: New test, need investigation\n'124                            '  test: neutron.tests.tempest.api.test_'125                            'revisions.TestRevisions\n')126        self.cmd = BugVerifyCmd()127        self.cmd.parse_arguments(['--skip-file', self.tmp_file])128    def tearDown(self):129        os.close(self.fd_file)130        os.unlink(self.tmp_file)131    def _populate_skip_file(self):132        content = '''133        known_failures:134        - test: '.*test_external_network_visibility'135          reason: 'Tempest test "external network visibility" fails'136          lp: 'https://bugs.launchpad.net/tripleo/+bug/1577769'137        - test: 'tempest.api.data_processing'138          reason: 'tempest.api.data_processing tests failing on newton'139          bz: 'https://bugzilla.redhat.com/show_bug.cgi?id=1357667'140        - test: 'neutron.tests.tempest.api.test_revisions.TestRevisions'141          reason: 'New test, need investigation'142        '''143        self.skip_file = open(self.tmp_file, 'w')144        self.skip_file.write(content)145        self.skip_file.close()146    def test_load_skip_file(self):147        known_failures = self.cmd.load_skip_file()148        self.assertEqual(known_failures, self.known_failures)149    def test__print_txt(self):150        output = self.cmd._print_txt(self.known_failures)151        self.assertEqual(output, self.txt_output)152    def test__print_yaml(self):153        output = self.cmd._print_yaml(self.known_failures)154        self.assertEqual(output, self.yaml_output)155    @mock.patch('bugcheck.BugVerifyCmd._print_txt')156    @mock.patch('bugcheck.BugVerifyCmd._print_yaml')157    def test_get_output(self, yaml_mock, txt_mock):158        self.cmd.get_output(self.known_failures, 'txt')159        self.cmd.get_output(self.known_failures, 'yaml')160        yaml_mock.assert_called_once()161        txt_mock.assert_called_once()162        self.assertRaises(ValueError,163                          self.cmd.get_output, self.known_failures, 'xml')164    def test_save_output(self):165        fd, tmp_f = tempfile.mkstemp()166        cmd = BugVerifyCmd()167        cmd.parse_arguments(['--skip-file', self.tmp_file, '--to-file', tmp_f])168        cmd.save_output(self.known_failures, 'txt')169        output = open(tmp_f, 'r').readlines()170        expected = ['# Tempest test "external network visibility" fails\n',171                    '.*test_external_network_visibility\n',172                    '# tempest.api.data_processing tests failing on newton\n',173                    'tempest.api.data_processing\n',174                    '# New test, need investigation\n',175                    'neutron.tests.tempest.api.test_revisions.TestRevisions\n']176        self.assertEqual(output, expected)177        cmd.save_output(self.known_failures, 'yaml')178        output = open(tmp_f, 'r').readlines()179        expected = ['---\n',180                    'known_failures:\n',181                    '- lp: https://bugs.launchpad.net/tripleo/+bug/1577769\n',182                    '  reason: Tempest test "external network visibility" '183                    'fails\n',184                    '  test: .*test_external_network_visibility\n',185                    '- bz: https://bugzilla.redhat.com/show_bug.cgi?'186                    'id=1357667\n',187                    '  reason: tempest.api.data_processing tests failing on '188                    'newton\n',189                    '  test: tempest.api.data_processing\n',190                    '- reason: New test, need investigation\n',191                    '  test: neutron.tests.tempest.api.test_revisions.Tes'192                    'tRevisions\n']...

constants.py

Source:constants.py

1import re2HREF = re.compile('href="([^"]+)"')3JOBRE = re.compile('[a-z0-9]{7}/')4TESTRE = re.compile('(tempest\.[^ \(\)]+)')5TIMEST = re.compile('(\d{4}-\d{2}-\d{2} \d{2}:\d{2}):\d{2}\.\d+ \|')6TITLE = re.compile('<title>(.*?)</title>')7FAILED = "... FAILED"8OK = "... ok"9ERROR = "... ERROR"10SKIPPED = "... SKIPPED"11TESTS = {12    'tempest.scenario.test_volume_boot_pattern.*':13        'http://bugzilla.redhat.com/1272289',14    'tempest.api.identity.*v3.*':15        'https://bugzilla.redhat.com/1266947',16    '.*test_external_network_visibility':17        'https://bugs.launchpad.net/tripleo/+bug/1577769',...

Automation Testing Tutorials

Learn to execute automation testing from scratch with LambdaTest Learning Hub. Right from setting up the prerequisites to run your first automation test, to following best practices and diving deeper into advanced test scenarios. LambdaTest Learning Hubs compile a list of step-by-step guides to help you be proficient with different test automation frameworks i.e. Selenium, Cypress, TestNG etc.