How to use shouldAllowPassingNullArgument method of org.mockitousage.bugs.CovariantOverrideTest class

You should be getting it as a compile-time error, not an exception (unless the actual exception is that you've got an unresolved compile-time error).

Just importing org.mockito.Matchers means you can use the name Matchers to mean org.mockito.Matchers anywhere in the class. If you want to import the methods, you need a static wildcard import:

import static org.mockito.Matchers.*;

Or specific methods:

import static org.mockito.Matchers.anyString;
import static org.mockito.Matchers.anyList;

Or you could just qualify the method name in the calling code instead:

Mockito.when(nrClient.uploadFiles("DF49ACBC8", Matchers.anyList(), "dl"))
       .thenReturn("");

Ok, first things first: I never used Mockito once in a life. But I was curious, so I spent several hours to figure this out and I guess there is much to improve.

So to get this working, we need:

1. The aforementioned (by @jewelsea) JUnit Threading Rule.
2. A custom MockMaker implementation, wrapping the default CglibMockMaker.
3. Wire the things together.

So 1+2 is this:

public class JavaFXMockMaker implements MockMaker {

    private final MockMaker wrapped = new CglibMockMaker();
    private boolean jfxIsSetup;

    private void doOnJavaFXThread(Runnable pRun) throws RuntimeException {
        if (!jfxIsSetup) {
            setupJavaFX();
            jfxIsSetup = true;
        }
        final CountDownLatch countDownLatch = new CountDownLatch(1);
        Platform.runLater(() -> {
            pRun.run();
            countDownLatch.countDown();
        });

        try {
            countDownLatch.await();
        } catch (InterruptedException e) {
            throw new RuntimeException(e);
        }
    }

    protected void setupJavaFX() throws RuntimeException {
        final CountDownLatch latch = new CountDownLatch(1);
        SwingUtilities.invokeLater(() -> {
            new JFXPanel(); // initializes JavaFX environment
            latch.countDown();
        });

        try {
            latch.await();
        } catch (InterruptedException e) {
            throw new RuntimeException(e);
        }
    }

    @Override
    public <T> T createMock(MockCreationSettings<T> settings, MockHandler handler) {
        AtomicReference<T> result = new AtomicReference<>();
        Runnable run = () -> result.set(wrapped.createMock(settings, handler));
        doOnJavaFXThread(run);
        return result.get();
    }

    @Override
    public MockHandler getHandler(Object mock) {
        AtomicReference<MockHandler> result = new AtomicReference<>();
        Runnable run = () -> result.set(wrapped.getHandler(mock));
        doOnJavaFXThread(run);
        return result.get();
    }

    @Override
    public void resetMock(Object mock, MockHandler newHandler, @SuppressWarnings("rawtypes") MockCreationSettings settings) {
        Runnable run = () -> wrapped.resetMock(mock, newHandler, settings); 
        doOnJavaFXThread(run);
    }

}

Number 3 is just following the manual:

1. Copy the fully qualified class name of our MockMaker, eg. org.awesome.mockito.JavaFXMockMaker.
2. Create a file "mockito-extensions/org.mockito.plugins.MockMaker". The content of this file is exactly a one line with the qualified name.

Happy testing & kudos to Andy Till for his threading rule.

Warning: This implementation kind of hard-codes the MockMaker to use the CglibMockMaker which might not be be what you want in every case (see the JavaDocs).

UPDATE: My answer is quite old. There is a built-in method in Mockito now to insert the delay directly: AnswersWithDelay. See Bogdan's response for more details.

There is already a CallsRealMethods Answer that you can extend and decorate with your delay:

public class CallsRealMethodsWithDelay extends CallsRealMethods {

    private final long delay;

    public CallsRealMethodsWithDelay(long delay) {
        this.delay = delay;
    }

    public Object answer(InvocationOnMock invocation) throws Throwable {
        Thread.sleep(delay);
        return super.answer(invocation);
    }

}

And then use it like that:

MyObject foobar = Mockito.spy(new MyObject(param1, param2, param3));
Mockito.doAnswer(new CallsRealMethodsWithDelay(2000))
           .when(foobar).myRealMethodName();

You can of course also use a static method to make everything even more beautiful:

public static Stubber doAnswerWithRealMethodAndDelay(long delay) {
    return Mockito.doAnswer(new CallsRealMethodsWithDelay(delay));
}

And use it like:

doAnswerWithRealMethodAndDelay(2000)
           .when(foobar).myRealMethodName();

You need to annotate ProductService with @InjectMocks:

@Autowired
@InjectMocks
private ProductService productService;

This will inject the ClientService mock into your ProductService.

Mockito 2 now supports final classes and methods!

But for now that's an "incubating" feature. It requires some steps to activate it which are described in What's New in Mockito 2:

Mocking of final classes and methods is an incubating, opt-in feature. It uses a combination of Java agent instrumentation and subclassing in order to enable mockability of these types. As this works differently to our current mechanism and this one has different limitations and as we want to gather experience and user feedback, this feature had to be explicitly activated to be available ; it can be done via the mockito extension mechanism by creating the file src/test/resources/mockito-extensions/org.mockito.plugins.MockMaker containing a single line:

mock-maker-inline

After you created this file, Mockito will automatically use this new engine and one can do :

 final class FinalClass {
   final String finalMethod() { return "something"; }
 }

 FinalClass concrete = new FinalClass(); 

 FinalClass mock = mock(FinalClass.class);
 given(mock.finalMethod()).willReturn("not anymore");

 assertThat(mock.finalMethod()).isNotEqualTo(concrete.finalMethod());

In subsequent milestones, the team will bring a programmatic way of using this feature. We will identify and provide support for all unmockable scenarios. Stay tuned and please let us know what you think of this feature!

It is possible for third-party plugins to register custom MockMaker implementations that create classes differently to the default behavior via Mockito's SPI. As a part of the specified behavior, since Mockito 2.0, a MockMaker can decide if a class is mockable, such as final classes in the case of PowerMock. In order to integrate such a custom behavior, PowerMock registers a custom MockMaker that is then used by Mockito.

In Mockito 2.0, the API for a custom MockMaker had to be extended but PowerMock does not yet adapt the API extensions. Therefore, this new method is not yet implemented by the PowerMock MockMaker what results in an AbstractMethodError.

Mockito 1.8.1 introduced anyVararg() matcher:

when(a.b(anyInt(), anyInt(), Matchers.<String>anyVararg())).thenReturn(b);

Also see history for this: https://code.google.com/archive/p/mockito/issues/62

Edit new syntax after deprecation:

when(a.b(anyInt(), anyInt(), ArgumentMatchers.<String>any())).thenReturn(b);

The cause of your problem is that Spock makes methods it spies on synchronized. Particularly, the method MockController.handle(), through which all such calls go, is synchronized. You'll easily notice it if you add a pause and some output to your getStrings() method.

public List<String> getStrings() throws InterruptedException {
    System.out.println(Thread.currentThread().getId() + " goes to sleep");
    Thread.sleep(1000);
    System.out.println(Thread.currentThread().getId() + " awoke");
    if (strings == null) {
        strings = loadStrings();
    }
    return strings;
}

This way Spock inadvertently fixes your concurrency problem. Mockito obviously uses another approach.

A couple of other thoughts on your tests:

First, you don't do much to ensure that all your threads have come to the getStrings() call at the same moment, thus decreasing the probability of collisions. Long time may pass between threads start (long enough for the first one to complete the call before others start it). A better approach would be to use some synchronization primitive to remove the influence of threads startup time. For instance, a CountDownLatch may be of use here:

given:
def final CountDownLatch latch = new CountDownLatch(10)
def loader = Spy(ExampleLoader)
def threads = (0..<10).collect { new Thread({
    latch.countDown()
    latch.await()
    loader.getStrings()
})}

Of course, within Spock it will make no difference, it's just an example on how to do it in general.

Second, the problem with concurrency tests is that they never guarantee that your program is thread safe. The best you can hope for is that such test will show you that your program is broken. But even if the test passes, it doesn't prove thread safety. To increase chances of finding concurrency bugs you may want to run the test many times and gather statistics. Sometimes, such tests only fail once in several thousands or even once in several hundreds thousands of runs. Your class is simple enough to make guesses about its thread safety, but such approach will not work with more complicated cases.

I also faced the same problem.

And finally, I found argThat() will return null, and normally the argument in the function in kotlin, does not accept null type.

The source code of argThat from ArgumentMatchers.java

public static <T> T argThat(ArgumentMatcher<T> matcher) {
    reportMatcher(matcher);
    return null;
}

You can see that it return null. So when we mock the function, it will throw IllegalStateException, because argThat returns null and argument can't be null.

It mean that if your function is:

fun doSomething(arg1: String): Int {
    // do something
}

When you mock it like that:

Mockito.`when`(
    doSomething(Mockito.argThat<String> { true })
).thenReturn(100)

It will throw IllegalStateException

So you should change your function like that:

fun doSomething(arg1: String?): Int {
    // do something
}

Change the "String" to "String?", make it accept null type.

My solution is to define the argument with class? so that it can accept null, but I don't know if it is a great solution