How to use test_addini_paths method in Pytest

I'm going to mention one of the new features of Python 3.6 - f-strings.

They can evaluate expressions,

>>> eval('f"{().__class__.__base__}"', {'__builtins__': None}, {})
"<class 'object'>"

but the attribute access won't be detected by Python's tokenizer:

0,0-0,0:            ENCODING       'utf-8'        
1,0-1,1:            ERRORTOKEN     "'"            
1,1-1,27:           STRING         'f"{().__class__.__base__}"'
2,0-2,0:            ENDMARKER      '' 

It used to be a required part of a package (old, pre-3.3 "regular package", not newer 3.3+ "namespace package").

Here's the documentation.

Python defines two types of packages, regular packages and namespace packages. Regular packages are traditional packages as they existed in Python 3.2 and earlier. A regular package is typically implemented as a directory containing an __init__.py file. When a regular package is imported, this __init__.py file is implicitly executed, and the objects it defines are bound to names in the package’s namespace. The __init__.py file can contain the same Python code that any other module can contain, and Python will add some additional attributes to the module when it is imported.

But just click the link, it contains an example, more information, and an explanation of namespace packages, the kind of packages without __init__.py.

Instead of using the pytest-cov plugin, use coverage to run pytest:

coverage run -m pytest ....

That way, coverage will be started before pytest.

Printing a progress bar isn't difficult. The common method is to use the carriage return character (\r):

from __future__ import print_function

import sys

import fcntl
import struct
import termios

class ProgressBar(object):
    def __init__(self):
        pass

    def get_terminal_width(self):
        try:
            return struct.unpack('hh', fcntl.ioctl(sys.stdout.fileno(), termios.TIOCGWINSZ, '\x00' * 4))[1]
        except IOError:
            return 120

    def display(self, percentage):
        terminal_width = self.get_terminal_width()

        stars = '*' * int((percentage / 100.0) * terminal_width)

        print(stars, end='\r')  # Print out your progress and jump to the beginning of the line
        sys.stdout.flush()      # Flush stdout to make it show up immediately

if __name__ == '__main__':
    import time
    bar = ProgressBar()

    for i in range(100):
        bar.display(i)
        time.sleep(0.03)

I'll leave it you to figure out the string manipulations you need to do to get the progress bar showing up in the middle.

As already discussed, tensorflow doesn't provide its own way to cross-validate the model. The recommended way is to use KFold. It's a bit tedious, but doable. Here's a complete example of cross-validating MNIST model with tensorflow and KFold:

from sklearn.model_selection import KFold
import tensorflow as tf
from tensorflow.examples.tutorials.mnist import input_data

# Parameters
learning_rate = 0.01
batch_size = 500

# TF graph
x = tf.placeholder(tf.float32, [None, 784])
y = tf.placeholder(tf.float32, [None, 10])
W = tf.Variable(tf.zeros([784, 10]))
b = tf.Variable(tf.zeros([10]))
pred = tf.nn.softmax(tf.matmul(x, W) + b)
cost = tf.reduce_mean(-tf.reduce_sum(y*tf.log(pred), reduction_indices=1))
optimizer = tf.train.GradientDescentOptimizer(learning_rate).minimize(cost)
correct_prediction = tf.equal(tf.argmax(pred, 1), tf.argmax(y, 1))
accuracy = tf.reduce_mean(tf.cast(correct_prediction, tf.float32))
init = tf.global_variables_initializer()

mnist = input_data.read_data_sets("data/mnist-tf", one_hot=True)
train_x_all = mnist.train.images
train_y_all = mnist.train.labels
test_x = mnist.test.images
test_y = mnist.test.labels

def run_train(session, train_x, train_y):
  print "\nStart training"
  session.run(init)
  for epoch in range(10):
    total_batch = int(train_x.shape[0] / batch_size)
    for i in range(total_batch):
      batch_x = train_x[i*batch_size:(i+1)*batch_size]
      batch_y = train_y[i*batch_size:(i+1)*batch_size]
      _, c = session.run([optimizer, cost], feed_dict={x: batch_x, y: batch_y})
      if i % 50 == 0:
        print "Epoch #%d step=%d cost=%f" % (epoch, i, c)

def cross_validate(session, split_size=5):
  results = []
  kf = KFold(n_splits=split_size)
  for train_idx, val_idx in kf.split(train_x_all, train_y_all):
    train_x = train_x_all[train_idx]
    train_y = train_y_all[train_idx]
    val_x = train_x_all[val_idx]
    val_y = train_y_all[val_idx]
    run_train(session, train_x, train_y)
    results.append(session.run(accuracy, feed_dict={x: val_x, y: val_y}))
  return results

with tf.Session() as session:
  result = cross_validate(session)
  print "Cross-validation result: %s" % result
  print "Test accuracy: %f" % session.run(accuracy, feed_dict={x: test_x, y: test_y})

OK, I'm not 100% sure why but adding the sts decorator seems to have fixed this:

@mock_sts
def test_publish(sns):
    resp = sns.create_topic(Name="sdfsdfsdfsd")

I figured that out from this article but I'm still unclear on how it works: https://www.serverlessops.io/blog/aws-lambda-serverless-development-workflow-part2-testing-debugging

Is this because boto needs to use sts so I need to mock that out too? I use a credentials file with profiles to access AWS from laptop

Edit

You also, MUST use yield to return the client. Using return here gave me an sts error. I'd like to understand this better as well. I assume I need to use yield because it's a generator?

@pytest.fixture
def sns(aws_credentials):
    with mock_sns():
        # using return here causes below error
        return boto3.resource("sns", region_name="us-east-1")

Error when not using yield:

botocore.exceptions.ClientError: An error occurred (InvalidClientTokenId) when calling the CreateTopic operation: The security token included in the request is invalid

Here's a unit test that fails if ResourceWarning is generated by the code in with catch_warning() statement:

#!/usr/bin/env python3
import gc
import socket
import unittest
import warnings

class Test(unittest.TestCase):
    def test_resource_warning(self):
        s = socket.socket()
        ####s.close() #XXX uncomment to pass the test

        # generate resource warning when s is deleted
        with warnings.catch_warnings(record=True) as w:
            warnings.resetwarnings() # clear all filters
            warnings.simplefilter('ignore') # ignore all
            warnings.simplefilter('always', ResourceWarning) # add filter
            del s        # remove reference
            gc.collect() # run garbage collection (for pypy3)
            self.assertFalse(w and str(w[-1])) # test fails if there
                                               # are warnings

if __name__=="__main__":
    unittest.main()

In short:

NLTK is not perfect. In fact, no model is perfect.

Note:

As of NLTK version 3.1, default pos_tag function is no longer the old MaxEnt English pickle.

It is now the perceptron tagger from @Honnibal's implementation, see nltk.tag.pos_tag

>>> import inspect
>>> print inspect.getsource(pos_tag)
def pos_tag(tokens, tagset=None):
    tagger = PerceptronTagger()
    return _pos_tag(tokens, tagset, tagger) 

Still it's better but not perfect:

>>> from nltk import pos_tag
>>> pos_tag("The quick brown fox jumps over the lazy dog".split())
[('The', 'DT'), ('quick', 'JJ'), ('brown', 'NN'), ('fox', 'NN'), ('jumps', 'VBZ'), ('over', 'IN'), ('the', 'DT'), ('lazy', 'JJ'), ('dog', 'NN')]

At some point, if someone wants TL;DR solutions, see https://github.com/alvations/nltk_cli

In long:

Try using other tagger (see https://github.com/nltk/nltk/tree/develop/nltk/tag) , e.g.:

• HunPos
• Stanford POS
• Senna

Using default MaxEnt POS tagger from NLTK, i.e. nltk.pos_tag:

>>> from nltk import word_tokenize, pos_tag
>>> text = "The quick brown fox jumps over the lazy dog"
>>> pos_tag(word_tokenize(text))
[('The', 'DT'), ('quick', 'NN'), ('brown', 'NN'), ('fox', 'NN'), ('jumps', 'NNS'), ('over', 'IN'), ('the', 'DT'), ('lazy', 'NN'), ('dog', 'NN')]

Using Stanford POS tagger:

$ cd ~
$ wget http://nlp.stanford.edu/software/stanford-postagger-2015-04-20.zip
$ unzip stanford-postagger-2015-04-20.zip
$ mv stanford-postagger-2015-04-20 stanford-postagger
$ python
>>> from os.path import expanduser
>>> home = expanduser("~")
>>> from nltk.tag.stanford import POSTagger
>>> _path_to_model = home + '/stanford-postagger/models/english-bidirectional-distsim.tagger'
>>> _path_to_jar = home + '/stanford-postagger/stanford-postagger.jar'
>>> st = POSTagger(path_to_model=_path_to_model, path_to_jar=_path_to_jar)
>>> text = "The quick brown fox jumps over the lazy dog"
>>> st.tag(text.split())
[(u'The', u'DT'), (u'quick', u'JJ'), (u'brown', u'JJ'), (u'fox', u'NN'), (u'jumps', u'VBZ'), (u'over', u'IN'), (u'the', u'DT'), (u'lazy', u'JJ'), (u'dog', u'NN')]

Using HunPOS (NOTE: the default encoding is ISO-8859-1 not UTF8):

$ cd ~
$ wget https://hunpos.googlecode.com/files/hunpos-1.0-linux.tgz
$ tar zxvf hunpos-1.0-linux.tgz
$ wget https://hunpos.googlecode.com/files/en_wsj.model.gz
$ gzip -d en_wsj.model.gz 
$ mv en_wsj.model hunpos-1.0-linux/
$ python
>>> from os.path import expanduser
>>> home = expanduser("~")
>>> from nltk.tag.hunpos import HunposTagger
>>> _path_to_bin = home + '/hunpos-1.0-linux/hunpos-tag'
>>> _path_to_model = home + '/hunpos-1.0-linux/en_wsj.model'
>>> ht = HunposTagger(path_to_model=_path_to_model, path_to_bin=_path_to_bin)
>>> text = "The quick brown fox jumps over the lazy dog"
>>> ht.tag(text.split())
[('The', 'DT'), ('quick', 'JJ'), ('brown', 'JJ'), ('fox', 'NN'), ('jumps', 'NNS'), ('over', 'IN'), ('the', 'DT'), ('lazy', 'JJ'), ('dog', 'NN')]

Using Senna (Make sure you've the latest version of NLTK, there were some changes made to the API):

$ cd ~
$ wget http://ronan.collobert.com/senna/senna-v3.0.tgz
$ tar zxvf senna-v3.0.tgz
$ python
>>> from os.path import expanduser
>>> home = expanduser("~")
>>> from nltk.tag.senna import SennaTagger
>>> st = SennaTagger(home+'/senna')
>>> text = "The quick brown fox jumps over the lazy dog"
>>> st.tag(text.split())
[('The', u'DT'), ('quick', u'JJ'), ('brown', u'JJ'), ('fox', u'NN'), ('jumps', u'VBZ'), ('over', u'IN'), ('the', u'DT'), ('lazy', u'JJ'), ('dog', u'NN')]

Or try building a better POS tagger:

• Ngram Tagger:
• Affix/Regex Tagger:
• Build Your Own Brill (Read the code it's a pretty fun tagger, http://www.nltk.org/_modules/nltk/tag/brill.html), see
• Perceptron Tagger: https://honnibal.wordpress.com/2013/09/11/a-good-part-of-speechpos-tagger-in-about-200-lines-of-python/
• LDA Tagger: http://scm.io/blog/hack/2015/02/lda-intentions/

Complains about pos_tag accuracy on stackoverflow include:

• POS tagging - NLTK thinks noun is adjective
• python NLTK POS tagger not behaving as expected
• How to obtain better results using NLTK pos tag
• pos_tag in NLTK does not tag sentences correctly

Issues about NLTK HunPos include:

• How do I tag textfiles with hunpos in nltk?
• Does anyone know how to configure the hunpos wrapper class on nltk?

Issues with NLTK and Stanford POS tagger include:

• trouble importing stanford pos tagger into nltk
• Java Command Fails in NLTK Stanford POS Tagger
• Error using Stanford POS Tagger in NLTK Python
• How to improve speed with Stanford NLP Tagger and NLTK
• Nltk stanford pos tagger error : Java command failed
• Instantiating and using StanfordTagger within NLTK
• Running Stanford POS tagger in NLTK leads to "not a valid Win32 application" on Windows

1) First of all, you can declare those fixtures not only in conftest.py, but in every Python module you want. And you can import that module. Also you can use fixtures in the same way as you used setUp method:

@pytest.fixture(scope='class')
def input(request):
    request.cls.varA = 1
    request.cls.varB = 2
    request.cls.varC = 3
    request.cls.modified_varA = 2

@pytest.usefixtures('input')
class TestClass:
    def test_1(self):
        do_something_with_self.varA, self.varB

    def test_2(self):
        do_something_with_self_modified_varA, self.varC

or you can define separate variables in separate fixtures:

def fixture_a():
    return varA

def fixture_b():
    return varB

def fixture_c():
    return varC

def fixture_mod_A():
    return modified_varA

or make one fixture which returns all the variables (why not?) or even make indirect parametrized fixture which returns variables by your choice (quite confusing way):

@pytest.fixture()
def parametrized_input(request):
   vars = {'varA': 1, 'varB': 2, 'varC': 3}
   var_names = request.param
   return (vars[var_name] for var_name in var_names)

@pytest.mark.parametrize('parametrized_input', [('varA', 'varC')], indirect=True)
def test_1(parametrized_input)
   varA, varC = parametrized_input
   ...

Or even you can make fixture factory which will make fixtures for you on the fly. Sounds curiously when you have only 5 tests and 5 configurations of variables, but when you get hundreds of both, it can be useful.

3) Of course you can. But I recommend you not to import this file directly, but use command line option pointing what file to import. In this case you can choose another file with variables without changing your code.

4) I use classes in my tests because I migrated from nosetest. I didn't mention any problem with using classes in pytest.

5) In that case I propose you to do the following: fist make the function with desired actions:

def some_actions(a, b):
    # some actions here
    ...
    return c

then use it both in test and fixture:

def test():
    assert some_actions(1,2) == 10

@pytest.fixture()
def some_fixture():
     return some_actions(1,2)