How to use isiterable method in Pytest

Yes, it was added in version 2.5. The expression syntax is:

a if condition else b

First condition is evaluated, then exactly one of either a or b is evaluated and returned based on the Boolean value of condition. If condition evaluates to True, then a is evaluated and returned but b is ignored, or else when b is evaluated and returned but a is ignored.

This allows short-circuiting because when condition is true only a is evaluated and b is not evaluated at all, but when condition is false only b is evaluated and a is not evaluated at all.

For example:

>>> 'true' if True else 'false'
'true'
>>> 'true' if False else 'false'
'false'

Note that conditionals are an expression, not a statement. This means you can't use statements such as pass, or assignments with = (or "augmented" assignments like +=), within a conditional expression:

>>> pass if False else pass
  File "<stdin>", line 1
    pass if False else pass
         ^
SyntaxError: invalid syntax

>>> # Python parses this as `x = (1 if False else y) = 2`
>>> # The `(1 if False else x)` part is actually valid, but
>>> # it can't be on the left-hand side of `=`.
>>> x = 1 if False else y = 2
  File "<stdin>", line 1
SyntaxError: cannot assign to conditional expression

>>> # If we parenthesize it instead...
>>> (x = 1) if False else (y = 2)
  File "<stdin>", line 1
    (x = 1) if False else (y = 2)
       ^
SyntaxError: invalid syntax

(In 3.8 and above, the := "walrus" operator allows simple assignment of values as an expression, which is then compatible with this syntax. But please don't write code like that; it will quickly become very difficult to understand.)

Similarly, because it is an expression, the else part is mandatory:

# Invalid syntax: we didn't specify what the value should be if the 
# condition isn't met. It doesn't matter if we can verify that
# ahead of time.
a if True

You can, however, use conditional expressions to assign a variable like so:

x = a if True else b

Or for example to return a value:

# Of course we should just use the standard library `max`;
# this is just for demonstration purposes.
def my_max(a, b):
    return a if a > b else b

Think of the conditional expression as switching between two values. We can use it when we are in a 'one value or another' situation, where we will do the same thing with the result, regardless of whether the condition is met. We use the expression to compute the value, and then do something with it. If you need to do something different depending on the condition, then use a normal if statement instead.

Keep in mind that it's frowned upon by some Pythonistas for several reasons:

• The order of the arguments is different from those of the classic condition ? a : b ternary operator from many other languages (such as C, C++, Go, Perl, Ruby, Java, JavaScript, etc.), which may lead to bugs when people unfamiliar with Python's "surprising" behaviour use it (they may reverse the argument order).
• Some find it "unwieldy", since it goes contrary to the normal flow of thought (thinking of the condition first and then the effects).
• Stylistic reasons. (Although the 'inline if' can be really useful, and make your script more concise, it really does complicate your code)

If you're having trouble remembering the order, then remember that when read aloud, you (almost) say what you mean. For example, x = 4 if b > 8 else 9 is read aloud as x will be 4 if b is greater than 8 otherwise 9.

Official documentation:

• Conditional expressions
• Is there an equivalent of C’s ”?:” ternary operator?

Figured it out - needed to define a __get__ method in order to create a MethodType binding like so:

def __get__(self, obj, objtype=None):
    return MethodType(self, obj, objtype)

which creates a MethodType object when invoking the method on an object that freezes the self argument.

Yes, it is possible.

If you have the test and all the fixtures in 1 file: test.py

import pytest

@pytest.fixture
def foo():
    return "foo"

@pytest.fixture
def bar(foo):
    return foo, "bar"

def test_foo_bar(bar):
    expected = ("foo", "bar")
    assert bar == expected

and run pytest test.py then Success!!!

======================================= test session starts ========================================
platform darwin -- Python 3.6.8, pytest-4.3.0
collected 1 item                                                                                   

test.py .                                                                                    [100%]

===================================== 1 passed in 0.02 seconds =====================================

But if you put the fixtures in a different file: test_foo_bar.py

from test import bar

def test_foo_bar(bar):
    expected = ("foo", "bar")
    assert bar == expected

and run pytest test_foo_bar.py expecting (like I did) that importing only the bar fixture is enough since on importing it would already have executed the foo fixture then you get the error you are getting.

======================================= test session starts ========================================
platform darwin -- Python 3.6.8, pytest-4.3.0
collected 1 item                                                                                   

test2.py E                                                                                   [100%]

============================================== ERRORS ==============================================
__________________________________ ERROR at setup of test_foo_bar __________________________________
file .../test_foo_bar.py, line 3
  def test_foo_bar(bar):
.../test.py, line 7
  @pytest.fixture
  def bar(foo):
E       fixture 'foo' not found
>       available fixtures: TIMEOUT, bar, cache, capfd, capfdbinary, caplog, capsys, capsysbinary, cov, doctest_namespace, monkeypatch, no_cover, once_without_docker, pytestconfig, record_property, record_xml_attribute, recwarn, tmp_path, tmp_path_factory, tmpdir, tmpdir_factory
>       use 'pytest --fixtures [testpath]' for help on them.

.../test.py:7
===================================== 1 error in 0.03 seconds ======================================

To fix this also import the foo fixture in the test_foo_bar.py module.

In general if 'a' in c.lower() is the best way to go about this. In Python3 you'd use if 'a' in c.casefold().

If you're certain that you're only ever testing one character at a time, I don't see why you couldn't use the if c in 'aA', but it seems like a narrow use case. Imagine if you needed to make sure that all the letters in a word were between a and y (your employer hates the letter "Z").

charset = 'abcdefghijklmnopqrstuvwxyABCDEFGHIJKLMNOPQRSTUVWXY'
all(c in charset for c in word)

compared to:

word.lower() == word

lst1, lst2 = zip(*zipped_list)

should give you the unzipped list.

*zipped_list unpacks the zipped_list object. it then passes all the tuples from the zipped_list object to zip, which just packs them back up as they were when you passed them in.

so if:

a = [1,2,3]
b = [4,5,6]

then zipped_list = zip(a,b) gives you:

[(1,4), (2,5), (3,6)]

and *zipped_list gives you back

(1,4), (2,5), (3,6)

zipping that with zip(*zipped_list) gives you back the two collections:

[(1, 2, 3), (4, 5, 6)]

You cannot selectively disable arbitrary plugins for selected tests. The plugins are loaded at a much earlier stage — when the pytest starts. And the plugins actually define what pytest does and how (i.e., command line options, test collection, filtering, etc).

In other words, it is too late to redefine the pytest's internal structure when it gets to the test execution.

Your best case is, indeed, to mark your tests with @pytest.mark.nocatchlog, and execute them separately:

pytest -m 'nocatchlog' -p no:catchlog  # problematic tests with no plugin
pytest -m 'not nocatchlog`             # all other tests

If those tests not under your control, i.e. if you cannot add marks, then you can only filter by expressions like -k test_logging or -k 'not test_logging' (i.e. by part of their node id).

Specifically for this pytest-catchlog plugin, you can make the same hooks as it does, and remove its log handler from the root logger (assuming that no other loggers were used explicitly):

conftest.py:

import pytest

def _disable_catchlog(item):
    logger = logging.getLogger()
    if item.catch_log_handler in logger.handlers:
        logger.handlers.remove(item.catch_log_handler)

@pytest.hookimpl(hookwrapper=True, trylast=True)
def pytest_runtest_setup(item):
    _disable_catchlog(item)
    yield

@pytest.hookimpl(hookwrapper=True, trylast=True)
def pytest_runtest_call(item):
    _disable_catchlog(item)
    yield

@pytest.hookimpl(hookwrapper=True, trylast=True)
def pytest_runtest_teardown(item):
    _disable_catchlog(item)
    yield

DataFrame.iterrows is a generator which yields both the index and row (as a Series):

import pandas as pd

df = pd.DataFrame({'c1': [10, 11, 12], 'c2': [100, 110, 120]})
df = df.reset_index()  # make sure indexes pair with number of rows

for index, row in df.iterrows():
    print(row['c1'], row['c2'])

10 100
11 110
12 120

If you do not declare my_fixture as a fixture, then the test will not be able to use it as a fixture:

import pytest

def my_fixture():
    data = {'x': 1, 'y': 2, 'z': 3}
    return data

def test_my_fixture(my_fixture):
    assert my_fixture['x'] == 1

This script leads to an error:

  def test_my_fixture(my_fixture):
E       fixture 'my_fixture' not found
>       available fixtures: cache, capfd, capsys, doctest_namespace, monkeypatch, pytestconfig, record_xml_property, recwarn, tmpdir, tmpdir_factory
>       use 'pytest --fixtures [testpath]' for help on them.

The purpose of the fixtures is to prepare something before the test(s). And to kill it afterward. And NOT to be part of the test (e.g. in logging, reporting, ways of exception handling, etc). It is kind of a pre-condition, created by something else, and just given to the test.

If you just declare it as a function, and use it as a function, it is a function, not a fixture. And its failures become the failures of the test, not the general framework run.

Consider this example:

import pytest

@pytest.fixture()
def my_fixture():
    data = {'x': 1, 'y': 2, 'z': 3}
    print('prepare')
    yield data
    print('destroy')

def test_my_fixture(my_fixture):
    print('test it')
    assert my_fixture['x'] == 1

Also in this example try uncommenting the raising lines, and see the difference. It will be ERROR vs FAILURE. It is critical for how the test results are interpreted and handled later (e.g., by the developers, QA engineers, or whoever analyses the test results).

import pytest

@pytest.fixture()
def my_fixture():
    print('prepare')
    # raise Exception('Oops')
    yield None
    print('destroy')

def test_my_fixture(my_fixture):
    # raise Exception('Booms!')
    print('test it')

In the fixture:

======================================= test session starts ========================================
collected 1 item                                                                                    

t.py E

============================================== ERRORS ==============================================
________________________________ ERROR at setup of test_my_fixture _________________________________

    @pytest.fixture()
    def my_fixture():
        data = {'x': 1, 'y': 2, 'z': 3}
        print('prepare')
>       raise Exception('Oops')
E       Exception: Oops

t.py:7: Exception
===================================== 1 error in 0.03 seconds ======================================

In the test or any function called from the test:

======================================= test session starts ========================================
collected 1 item                                                                                    

t.py F

============================================= FAILURES =============================================
_________________________________________ test_my_fixture __________________________________________

my_fixture = {'x': 1, 'y': 2, 'z': 3}

    def test_my_fixture(my_fixture):
>       raise Exception('Booms!')
E       Exception: Booms!

t.py:12: Exception
===================================== 1 failed in 0.03 seconds =====================================

Also note that the fixtures have scopes: session, module, function. It will be difficult to demonstrate in a short example, but in that case, the fixture will be prepared and destroyed only once (usually only once) for multiple tests within that scope. This is important for the heavy-weight fixtures, such as the database pre-population. Such fixtures usually reside in the conftest.py pseudo-plugin in the directory structure (NB: this is not a regular module to be imported! by design, at least).

The problem is that pytest searches for //pytest.ini, //tox.ini, //setup.cfg, and //setup.py. Each of them caused either genericpath.exists() or genericpath.isfile() to consume about 2.5 seconds.

On fix is to add the lines below to genericpath.exists() and genericpath.isfile() to skip those four specific paths.

if path.startswith(r'//'):
    return False

An alternative fix would be to modify _pytest/config.py so it does not form those double-slash in the paths to search.

The code used to find out the exact problem is pasted below. Set myshow = True so it will show how time is consumed for every file it is searching for.

$ diff -u /usr/lib/python2.6/genericpath.py genericpath.py
--- /usr/lib/python2.6/genericpath.py   2012-06-09 08:33:12.000000000 -0700
+++ genericpath.py      2015-06-11 11:46:33.674285900 -0700
@@ -9,14 +9,29 @@
 __all__ = ['commonprefix', 'exists', 'getatime', 'getctime', 'getmtime',
            'getsize', 'isdir', 'isfile']

+myshow = False
+import time as mytime
+mybasetime = mytime.time()
+def myshowtime():
+    currenttime = mytime.time()
+    tmdiff = currenttime - mybasetime
+    global mybasetime
+    mybasetime = currenttime
+    return tmdiff

 # Does a path exist?
 # This is false for dangling symbolic links on systems that support them.
 def exists(path):
     """Test whether a path exists.  Returns False for broken symbolic links"""
+    pretime = myshowtime()
+    if path.startswith(r'//'):
+        if myshow: print "\n  genericpath exists  %8.3f %8.3f False " % (pretime, myshowtime()), " ", path, "\n"
+        return False
     try:
         st = os.stat(path)
+        if myshow: print "\n  genericpath exists  %8.3f %8.3f True  " % (pretime, myshowtime()), " ", path, "\n"
     except os.error:
+        if myshow: print "\n  genericpath exists  %8.3f %8.3f False " % (pretime, myshowtime()), " ", path, "\n"
         return False
     return True

@@ -25,9 +40,15 @@
 # for the same path ono systems that support symlinks
 def isfile(path):
     """Test whether a path is a regular file"""
+    pretime = myshowtime()
+    if path.startswith(r'//'):
+        if myshow: print "\n  genericpath isfile  %8.3f %8.3f False " % (pretime, myshowtime()), " ", path, "\n"
+        return False
     try:
         st = os.stat(path)
+        if myshow: print "\n  genericpath isfile  %8.3f %8.3f True  " % (pretime, myshowtime()), " ", path, "\n"
     except os.error:
+        if myshow: print "\n  genericpath isfile  %8.3f %8.3f False " % (pretime, myshowtime()), " ", path, "\n"
         return False
     return stat.S_ISREG(st.st_mode)