How to use test_as_errors method in Pytest

Fully Pythonic way to read a file is the following:

with open(...) as f:
    for line in f:
        <do something with line>

The with statement handles opening and closing the file, including if an exception is raised in the inner block. The for line in f treats the file object f as an iterable, which automatically uses buffered IO and memory management so you don't have to worry about large files.

There should be one -- and preferably only one -- obvious way to do it.

well few hit and trial and i got my answer.

I changed following

albums_list = map(lambda x:[x['album_id'],x['title'], x['language'], x['seokey'], x['release_date'],','.join(map(lambda y:y['name'], x.get('artists',[])[:2])) ,x['trackcount']], albums)
        tabledata = [['S No.', 'Album Title', 'Album Language', 'Release Date', 'Artists', 'Track Count']]
        for idx, value in enumerate(albums_list):
            tabledata.append([str(idx), value[1], value[2], value[4], value[5], value[6]])

with

        albums_list = map(lambda x:[x['album_id'],x['title'], x['language'], x['seokey'], x['release_date']], albums)
        tabledata = [['S No.', 'Album Title', 'Album Language', 'Release Date']]
        for idx, value in enumerate(albums_list):
            tabledata.append([str(idx), value[1], value[2], value[4]])

and it worked fine now :)

In the client link you provided, states that the client is an instance of the django.test.Client so in reality it doesn't do anything special there and shouldn't be a problem.

You need to setup your environment as you correctly stated.
Let's have a look at the error now:

from the setup_test_environment() source code:

if hasattr(_TestState, 'saved_data'):
      # Executing this function twice would overwrite the saved values.
      raise RuntimeError(
          "setup_test_environment() was already called and can't be called "
          "again without first calling teardown_test_environment()."
)

And that is what raises your RuntimeError above.

Let's look now at the teardown_test_environment() method:

...
del _TestState.saved_data

So it deletes the culprit of the aforementioned exception.

Thus:

from django.test.utils import teardown_test_environment, setup_test_environment

try:
    # If setup_test_environment haven't been called previously this
    # will produce an AttributeError.
    teardown_test_environment()
except AttributeError:
    pass

setup_test_environment() 

...

Fixed.

The answer appears to be

1. Uninstall python via the windows apps and features
2. Remove the c:\program files\python38 directory
3. Remove the ..\AppData\Roaming\Python directory
4. Reinstall

Not sure what the "root" problem was, but a total wipe of python fixed it. Note that the python windows installer does not remove enough python stuff.

Thank you @nneonneo for getting me thinking in the right direction.

I could not find a suitable pytest hook to access the overall test result yet.

You don't need one; just collect the test results yourself. This is the blueprint I usually use when in need of accessing the test results in batch:

# conftest.py
import pytest


def pytest_sessionstart(session):
    session.results = dict()


@pytest.hookimpl(tryfirst=True, hookwrapper=True)
def pytest_runtest_makereport(item, call):
    outcome = yield
    result = outcome.get_result()

    if result.when == 'call':
        item.session.results[item] = result

Now all test results are stored under session.results dict; example usage:

# conftest.py (continued)

def pytest_sessionfinish(session, exitstatus):
    print()
    print('run status code:', exitstatus)
    passed_amount = sum(1 for result in session.results.values() if result.passed)
    failed_amount = sum(1 for result in session.results.values() if result.failed)
    print(f'there are {passed_amount} passed and {failed_amount} failed tests')

Running the tests will yield:

$ pytest -sv
================================== test session starts ====================================
platform darwin -- Python 3.6.4, pytest-3.7.1, py-1.5.3, pluggy-0.7.1 -- /Users/hoefling/.virtualenvs/stackoverflow/bin/python3.6
cachedir: .pytest_cache
rootdir: /Users/hoefling/projects/private/stackoverflow/so-51711988, inifile:
collected 3 items

test_spam.py::test_spam PASSED
test_spam.py::test_eggs PASSED
test_spam.py::test_fail FAILED
run status code: 1
there are 2 passed and 1 failed tests


======================================== FAILURES =========================================
_______________________________________ test_fail _________________________________________

    def test_fail():
>       assert False
E       assert False

test_spam.py:10: AssertionError
=========================== 1 failed, 2 passed in 0.05 seconds ============================

In case the overall pytest exit code (exitstatus) is sufficient info (info about # passed, # failed, etc. not required) use the following:

# conftest.py

def pytest_sessionfinish(session, exitstatus):
    print()
    print('run status code:', exitstatus)

Accessing the error

You can access the error details from the call.excinfo object:

@pytest.hookimpl(tryfirst=True, hookwrapper=True)
def pytest_runtest_makereport(item, call):
    outcome = yield
    result = outcome.get_result()
    if result.outcome == "failed":
        exception = call.excinfo.value
        exception_class = call.excinfo.type
        exception_class_name = call.excinfo.typename
        exception_type_and_message_formatted = call.excinfo.exconly()
        exception_traceback = call.excinfo.traceback

I figured it out finally. I had this environment variable set:

PYTEST_ADDOPTS=-m networks and level1

I was experimenting with env variables a while back but never had an issue until I did a reboot. Hopefully this helps someone in the future not be an idiot like myself lol.

By default, py.test captures the result of standard out so that it can control how it prints it out. If it didn't do this, it would spew out a lot of text without the context of what test printed that text.

However, if a test fails, it will include a section in the resulting report that shows what was printed to standard out in that particular test.

For example,

def test_good():
    for i in range(1000):
        print(i)

def test_bad():
    print('this should fail!')
    assert False

Results in the following output:

>>> py.test tmp.py
============================= test session starts ==============================
platform darwin -- Python 2.7.6 -- py-1.4.20 -- pytest-2.5.2
plugins: cache, cov, pep8, xdist
collected 2 items

tmp.py .F

=================================== FAILURES ===================================
___________________________________ test_bad ___________________________________

    def test_bad():
        print('this should fail!')
>       assert False
E       assert False

tmp.py:7: AssertionError
------------------------------- Captured stdout --------------------------------
this should fail!
====================== 1 failed, 1 passed in 0.04 seconds ======================

Note the Captured stdout section.

If you would like to see print statements as they are executed, you can pass the -s flag to py.test. However, note that this can sometimes be difficult to parse.

>>> py.test tmp.py -s
============================= test session starts ==============================
platform darwin -- Python 2.7.6 -- py-1.4.20 -- pytest-2.5.2
plugins: cache, cov, pep8, xdist
collected 2 items

tmp.py 0
1
2
3
... and so on ...
997
998
999
.this should fail!
F

=================================== FAILURES ===================================
___________________________________ test_bad ___________________________________

    def test_bad():
        print('this should fail!')
>       assert False
E       assert False

tmp.py:7: AssertionError
====================== 1 failed, 1 passed in 0.02 seconds ======================

Thanks for your answer @vinzBad!

It seems the pytest-extensions are not the culprit. Installed all of them in a blank repository. They only increase the time for pytest --version from 0.3s to at most 0.5s.

However, as the cProfile above demonstrated, even for pytest --version all source files seem to be imported. And the import just seems to take ~5s as determined by time python -c "import segmentation". This might be a non-optimizable time, since many libraries are imported. Definitely not a pytest issue though.

Though classmethod and staticmethod are quite similar, there's a slight difference in usage for both entities: classmethod must have a reference to a class object as the first parameter, whereas staticmethod can have no parameters at all.

Example

class Date(object):
    
    def __init__(self, day=0, month=0, year=0):
        self.day = day
        self.month = month
        self.year = year

    @classmethod
    def from_string(cls, date_as_string):
        day, month, year = map(int, date_as_string.split('-'))
        date1 = cls(day, month, year)
        return date1

    @staticmethod
    def is_date_valid(date_as_string):
        day, month, year = map(int, date_as_string.split('-'))
        return day <= 31 and month <= 12 and year <= 3999

date2 = Date.from_string('11-09-2012')
is_date = Date.is_date_valid('11-09-2012')

Explanation

Let's assume an example of a class, dealing with date information (this will be our boilerplate):

class Date(object):
    
    def __init__(self, day=0, month=0, year=0):
        self.day = day
        self.month = month
        self.year = year

This class obviously could be used to store information about certain dates (without timezone information; let's assume all dates are presented in UTC).

Here we have __init__, a typical initializer of Python class instances, which receives arguments as a typical instance method, having the first non-optional argument (self) that holds a reference to a newly created instance.

Class Method

We have some tasks that can be nicely done using classmethods.

Let's assume that we want to create a lot of Date class instances having date information coming from an outer source encoded as a string with format 'dd-mm-yyyy'. Suppose we have to do this in different places in the source code of our project.

So what we must do here is:

1. Parse a string to receive day, month and year as three integer variables or a 3-item tuple consisting of that variable.
2. Instantiate Date by passing those values to the initialization call.

This will look like:

day, month, year = map(int, string_date.split('-'))
date1 = Date(day, month, year)

For this purpose, C++ can implement such a feature with overloading, but Python lacks this overloading. Instead, we can use classmethod. Let's create another constructor.

    @classmethod
    def from_string(cls, date_as_string):
        day, month, year = map(int, date_as_string.split('-'))
        date1 = cls(day, month, year)
        return date1

date2 = Date.from_string('11-09-2012')

Let's look more carefully at the above implementation, and review what advantages we have here:

1. We've implemented date string parsing in one place and it's reusable now.
2. Encapsulation works fine here (if you think that you could implement string parsing as a single function elsewhere, this solution fits the OOP paradigm far better).
3. cls is the class itself, not an instance of the class. It's pretty cool because if we inherit our Date class, all children will have from_string defined also.

Static method

What about staticmethod? It's pretty similar to classmethod but doesn't take any obligatory parameters (like a class method or instance method does).

Let's look at the next use case.

We have a date string that we want to validate somehow. This task is also logically bound to the Date class we've used so far, but doesn't require instantiation of it.

Here is where staticmethod can be useful. Let's look at the next piece of code:

    @staticmethod
    def is_date_valid(date_as_string):
        day, month, year = map(int, date_as_string.split('-'))
        return day <= 31 and month <= 12 and year <= 3999

# usage:
is_date = Date.is_date_valid('11-09-2012')

So, as we can see from usage of staticmethod, we don't have any access to what the class is---it's basically just a function, called syntactically like a method, but without access to the object and its internals (fields and other methods), which classmethod does have.