How to use pytest_runtest_logfinish method in Pytest

Short answer

To achieve exactly what you described, you need 3 handlers (which you have in your example), and set their default level to the corresponding level at the top. No need to set the handler's levels separately.

import logging
from logging import handlers

def my_logger(module_name, log_file, level):
    logger = logging.getLogger(module_name)
    logger.setLevel(level)

    # Create handlers
    c_handler = logging.StreamHandler()
    f_handler = handlers.RotatingFileHandler(filename=log_file)

    # Create formatters and add it to handlers
    c_format = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(lineno)d - %(message)s')
    f_format = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(lineno)d - %(message)s')
    c_handler.setFormatter(c_format)
    f_handler.setFormatter(f_format)

    # Add handlers to the logger
    logger.addHandler(c_handler)
    logger.addHandler(f_handler)

    return logger

if __name__ == "__main__":
    logger_info = my_logger("test_info", "info.log", logging.INFO)
    logger_debug = my_logger("test_debug", "debug.log", logging.DEBUG)
    logger_error = my_logger("test_error", "error.log", logging.ERROR)

    logger_info.info('Info test ...')
    logger_debug.debug('Debug test ...')
    logger_error.error('Error test ...')

Output:

python test_logger2.py 
2020-12-04 16:57:00,771 - test_info - INFO - 29 - Info test ...
2020-12-04 16:57:00,771 - test_debug - DEBUG - 30 - Debug test ...
2020-12-04 16:57:00,771 - test_error - ERROR - 31 - Error test ...

cat info.log 
2020-12-04 16:57:00,771 - test_info - INFO - 29 - Info test ...

cat debug.log 
2020-12-04 16:57:00,771 - test_debug - DEBUG - 30 - Debug test ...

cat error.log 
2020-12-04 16:57:00,771 - test_error - ERROR - 31 - Error test ...

Long answer:

1. Why 3 copies?
   You are calling your my_logger function 3 times, and each time you call it a new file handler as well as a NEW stream handler added to the logger. That's why you see 3 copies on your console (3 stream handlers). Plus all your handlers are set to DEBUG level. That's why all three logger prints out any log that you have given. You don't want a ERROR handler to process/print logs at DEBUG/INFO levels, so you should set its level to ERROR.

2. I don't think this is the standard approach for logging. You should instead have single logger with 4 handlers (stream, file_debug, file_info, file_error). Additionally, a debug log file should include all logs, and an info log file should include info logs and error logs. Details below.

import logging
from logging import handlers

def main():

    logger = logging.getLogger()
    c_format = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(lineno)d - %(message)s')

    # You need to set the default level to the lowest (DEBUG)
    logger.setLevel(logging.DEBUG)

    c_handler = logging.StreamHandler()
    c_handler.setLevel(logging.DEBUG)
    c_handler.setFormatter(c_format)

    f1_handler = handlers.RotatingFileHandler("debug.log")
    f1_handler.setLevel(logging.DEBUG)
    f1_handler.setFormatter(c_format)

    f2_handler = logging.handlers.RotatingFileHandler("info.log")
    f2_handler.setLevel(logging.INFO)
    f2_handler.setFormatter(c_format)

    f3_handler = logging.handlers.RotatingFileHandler("error.log")
    f3_handler.setLevel(logging.ERROR)
    f3_handler.setFormatter(c_format)

    logger.addHandler(c_handler)
    logger.addHandler(f1_handler)
    logger.addHandler(f2_handler)
    logger.addHandler(f3_handler)

    logger.debug("A debug line")
    logger.info("An info line")
    logger.error("An error line")

if __name__ == "__main__":
    main()

The output is :

python test_logger.py 
2020-12-04 16:48:56,247 - root - DEBUG - 32 - A debug line
2020-12-04 16:48:56,248 - root - INFO - 33 - An info line
2020-12-04 16:48:56,248 - root - ERROR - 34 - An error line

cat debug.log 
2020-12-04 16:49:06,673 - root - DEBUG - 32 - A debug line
2020-12-04 16:49:06,673 - root - INFO - 33 - An info line
2020-12-04 16:49:06,673 - root - ERROR - 34 - An error line

cat info.log 
2020-12-04 16:49:06,673 - root - INFO - 33 - An info line
2020-12-04 16:49:06,673 - root - ERROR - 34 - An error line

cat error.log 
2020-12-04 16:49:06,673 - root - ERROR - 34 - An error line

Here you see your debug.log file also contains the logs from other levels, and your info.log file contain error logs too. Because that's the whole rationale behind the log levels. The lower level should also keep track of the logs of higher levels (DEBUG < INFO < WARNING < ERROR). That's why what you want is not a standard way of doing it, but can perfectly be achieved as described in the short answer.

Starting with Requests version 2.4.2, you can use the json= parameter (which takes a dictionary) instead of data= (which takes a string) in the call:

>>> import requests
>>> r = requests.post('http://httpbin.org/post', json={"key": "value"})
>>> r.status_code
200
>>> r.json()
{'args': {},
 'data': '{"key": "value"}',
 'files': {},
 'form': {},
 'headers': {'Accept': '*/*',
             'Accept-Encoding': 'gzip, deflate',
             'Connection': 'close',
             'Content-Length': '16',
             'Content-Type': 'application/json',
             'Host': 'httpbin.org',
             'User-Agent': 'python-requests/2.4.3 CPython/3.4.0',
             'X-Request-Id': 'xx-xx-xx'},
 'json': {'key': 'value'},
 'origin': 'x.x.x.x',
 'url': 'http://httpbin.org/post'}

If you look closely into PyCharm sources, from the whole pytest output, PyCharm uses a single line the to parse the data for displaying in the Click to see difference dialog. This is the AssertionError: <message> line:

def test_spam():
>       assert v1 == v2
E       AssertionError: assert {'foo': 'bar'} == {'foo': 'baz'}
E         Differing items:
E         {'foo': 'bar'} != {'foo': 'baz'}
E         Use -v to get the full diff

If you want to see the full diff line without truncation, you need to customize this line in the output. For a single test, this can be done by adding a custom message to the assert statement:

def test_eggs():
    assert a == b, '{0} != {1}'.format(a, b)

If you want to apply this behaviour to all tests, define custom pytest_assertrepr_compare hook. In the conftest.py file:

# conftest.py
def pytest_assertrepr_compare(config, op, left, right):
    if op in ('==', '!='):
        return ['{0} {1} {2}'.format(left, op, right)]

The equality comparison of the values will now still be stripped when too long; to show the complete line, you still need to increase the verbosity with -vv flag.

Now the equality comparison of the values in the AssertionError line will not be stripped and the full diff is displayed in the Click to see difference dialog, highlighting the diff parts:

The problem is that pytest searches for //pytest.ini, //tox.ini, //setup.cfg, and //setup.py. Each of them caused either genericpath.exists() or genericpath.isfile() to consume about 2.5 seconds.

On fix is to add the lines below to genericpath.exists() and genericpath.isfile() to skip those four specific paths.

if path.startswith(r'//'):
    return False

An alternative fix would be to modify _pytest/config.py so it does not form those double-slash in the paths to search.

The code used to find out the exact problem is pasted below. Set myshow = True so it will show how time is consumed for every file it is searching for.

$ diff -u /usr/lib/python2.6/genericpath.py genericpath.py
--- /usr/lib/python2.6/genericpath.py   2012-06-09 08:33:12.000000000 -0700
+++ genericpath.py      2015-06-11 11:46:33.674285900 -0700
@@ -9,14 +9,29 @@
 __all__ = ['commonprefix', 'exists', 'getatime', 'getctime', 'getmtime',
            'getsize', 'isdir', 'isfile']

+myshow = False
+import time as mytime
+mybasetime = mytime.time()
+def myshowtime():
+    currenttime = mytime.time()
+    tmdiff = currenttime - mybasetime
+    global mybasetime
+    mybasetime = currenttime
+    return tmdiff

 # Does a path exist?
 # This is false for dangling symbolic links on systems that support them.
 def exists(path):
     """Test whether a path exists.  Returns False for broken symbolic links"""
+    pretime = myshowtime()
+    if path.startswith(r'//'):
+        if myshow: print "\n  genericpath exists  %8.3f %8.3f False " % (pretime, myshowtime()), " ", path, "\n"
+        return False
     try:
         st = os.stat(path)
+        if myshow: print "\n  genericpath exists  %8.3f %8.3f True  " % (pretime, myshowtime()), " ", path, "\n"
     except os.error:
+        if myshow: print "\n  genericpath exists  %8.3f %8.3f False " % (pretime, myshowtime()), " ", path, "\n"
         return False
     return True

@@ -25,9 +40,15 @@
 # for the same path ono systems that support symlinks
 def isfile(path):
     """Test whether a path is a regular file"""
+    pretime = myshowtime()
+    if path.startswith(r'//'):
+        if myshow: print "\n  genericpath isfile  %8.3f %8.3f False " % (pretime, myshowtime()), " ", path, "\n"
+        return False
     try:
         st = os.stat(path)
+        if myshow: print "\n  genericpath isfile  %8.3f %8.3f True  " % (pretime, myshowtime()), " ", path, "\n"
     except os.error:
+        if myshow: print "\n  genericpath isfile  %8.3f %8.3f False " % (pretime, myshowtime()), " ", path, "\n"
         return False
     return stat.S_ISREG(st.st_mode)

This isn't a solution to your specific problem, but I'm putting it here because this thread is the top Google result for "SSL: CERTIFICATE_VERIFY_FAILED", and it lead me on a wild goose chase.

If you have installed Python 3.6 on OSX and are getting the "SSL: CERTIFICATE_VERIFY_FAILED" error when trying to connect to an https:// site, it's probably because Python 3.6 on OSX has no certificates at all, and can't validate any SSL connections. This is a change for 3.6 on OSX, and requires a post-install step, which installs the certifi package of certificates. This is documented in the file ReadMe.rtf, which you can find at /Applications/Python\ 3.6/ReadMe.rtf (see also the file Conclusion.rtf, and the script build-installer.py that generates the macOS installer).

The ReadMe will have you run the post-install script at

/Applications/Python\ 3.10/Install\ Certificates.command (Terminal App, this command alone should, fix the issue. Be sure to update the file path using your current subversion.)

(its source is install_certificates.command), which:

• first installs the Python package certifi, and
• then creates a symbolic link from the OpenSSL certificates file to the certificates file installed by the package certifi.

Release notes have some more info: https://www.python.org/downloads/release/python-360/

On newer versions of Python, there is more documentation about this:

• https://github.com/python/cpython/blob/e05a703848473b0365886dcc593cbddc46609f29/Mac/BuildScript/resources/ReadMe.rtf#L22-L34
• https://github.com/python/cpython/blob/e05a703848473b0365886dcc593cbddc46609f29/Mac/BuildScript/resources/Conclusion.rtf#L15-L19
• https://github.com/python/cpython/blob/e05a703848473b0365886dcc593cbddc46609f29/Mac/BuildScript/resources/Welcome.rtf#L23-L25
• https://github.com/python/cpython/blob/e05a703848473b0365886dcc593cbddc46609f29/Mac/BuildScript/resources/install_certificates.command
• https://github.com/python/cpython/blob/e05a703848473b0365886dcc593cbddc46609f29/Mac/BuildScript/README.rst
• https://github.com/python/cpython/blob/e05a703848473b0365886dcc593cbddc46609f29/Mac/BuildScript/build-installer.py#L239-L246

Here's a solution that's part of the standard library:

from distutils.dir_util import copy_tree
copy_tree("/a/b/c", "/x/y/z")

See this similar question.

Copy directory contents into a directory with python

• Reference - https://docs.python.org/3/distutils/apiref.html#distutils.dir_util.copy_tree

Do not attempt to remove any Apple-supplied system Python which are in /System/Library and /usr/bin, as this may break your whole operating system.

NOTE: The steps listed below do not affect the Apple-supplied Python 2.7; they only remove a third-party Python framework, like those installed by python.org installers.

The complete list is documented here. Basically, all you need to do is the following:

1. Remove the third-party Python 2.7 framework

    sudo rm -rf /Library/Frameworks/Python.framework/Versions/2.7
   

2. Remove the Python 2.7 applications directory

    sudo rm -rf "/Applications/Python 2.7"
   

3. Remove the symbolic links, in /usr/local/bin, that point to this Python version. See them using

    ls -l /usr/local/bin | grep '../Library/Frameworks/Python.framework/Versions/2.7' 
   

and then run the following command to remove all the links:

    cd /usr/local/bin/
    ls -l /usr/local/bin | grep '../Library/Frameworks/Python.framework/Versions/2.7' | awk '{print $9}' | tr -d @ | xargs rm

4. If necessary, edit your shell profile file(s) to remove adding /Library/Frameworks/Python.framework/Versions/2.7 to your PATH environment file. Depending on which shell you use, any of the following files may have been modified: ~/.bash_login, ~/.bash_profile, ~/.cshrc, ~/.profile, ~/.tcshrc, ~/.zshrc, and/or ~/.zprofile.

As MasterAndrey has mentioned, pytest_ignore_collect should do the trick. Important to note that you should put conftest.py to root folder (the one you run tests from).
Example:

import sys

def pytest_ignore_collect(path):
    if sys.version_info[0] > 2:
        if str(path).endswith("__py2.py"):
            return True
    else:
        if str(path).endswith("__py3.py"):
            return True

Since pytest v4.3.0 there is also --ignore-glob flag which allows to ignore by pattern. Example: pytest --doctest-modules --ignore-glob="*__py3.py" dir/

According to the docs on pytest_load_initial_conftests:

Note: This hook will not be called for conftest.py files, only for setuptools plugins.

https://docs.pytest.org/en/latest/reference/reference.html#pytest.hookspec.pytest_load_initial_conftests

Probably it shouldn't be mentioned on that page that you found.

Edit: update docs url