How to use validateReturnValueFor method of org.mockito.internal.stubbing.answers.DefaultAnswerValidator class

Let me quote the official documentation:

Important gotcha on spying real objects!

Sometimes it's impossible to use when(Object) for stubbing spies. Example:

List list = new LinkedList();
List spy = spy(list);

// Impossible: real method is called so spy.get(0) throws IndexOutOfBoundsException (the list is yet empty)
when(spy.get(0)).thenReturn("foo");

// You have to use doReturn() for stubbing
doReturn("foo").when(spy).get(0);

In your case it goes something like:

doReturn(resultsIWant).when(myClassSpy).method1();

On my end if I build your project using the pom.xml you've showed us, with apache-poi declared after jfreechart, then as you've mentioned I get a corrupt JAR. Swapping the order of these two dependencies indeed gives me a correct JAR.

I've some previous experience with the maven-shade-plugin and when I used it I had trouble with the created JAR's META-INF directory, so I've checked that for defects.

I've tried to copy (using Total Commander) the META-INF directory somewhere on my local filesystem and what I've noticed is that there were errors while copying the files in META-INF/licences/. If I've tried to copy those somewhere individually and it worked but I couldn't copy the whole thing. My conclusion was that the JAR/ZIP archive is corrupt.

What I did was I entered that JAR in Total Commander (Ctrl+PgDown over the JAR file) and I renamed thirdpartylicenses.txt to thirdpartylicenses.txt.wtf. Doing this Total Commander offers to save this and it repackages the JAR (I've the Total7zip Total Commander packer plugin installed - if anyone tries this and it doesn't work try it with this installed).

After this. It works.

(I've also tried to repack the whole thing without renaming anything using Cygwin's unzip/zip commands, but that didn't work the new archive was still corrupt. Total Commander or the plugin I've mentioned does some magic.)

I guess the maven-shade-plugin merely creates a corrupt or invalid ZIP/JAR archive. I'm not sure why and maybe what I've described won't work for anyone else but I thought I'd mention it so maybe I could help.

---

I couldn't just leave this alone, so I've dug deeper and I think I've found the answer.

The bad JAR contains 65608 entries. The good JAR contains 65450 entries.

Guess what's the upper limit of the number of entries for a plain ZIP file? Yeah. The Wiki article speaks about a ZIP64 format which overcomes this limitation.

The good JAR has less entries because the actual dependencies change because of the position of the dependency declarations in your pom.xml. (As described by this answer.)

I've counted the entries like this.

Collections.list(new JarFile("...").entries()).size();

I was using Java 7 which seems to support the new ZIP64 format, so maybe if someone tries to count the entries in the bad JAR using Java 5 or 6 will receive an error (I'm not sure though).

I also tried to run the JAR exploded. I've unpacked the whole JAR into a directory and run the whole thing like this.

java -cp <dir/ com.mycompany.test.Start

Worked like a charm.

---

Bottom line. Don't overuse the maven-shade-plugin.

I have a project at work where I build my project like this.

• I copy my project's dependencies using the maven-dependency-plugin. Checkout the copy-dependencies goal. This copies your dependencies into target/dependency if I remember correctly.

• Using the maven-jar-plugin I add these dependencies to my final JAR's MANIFEST.MF as Class-Path entries using these options in the plugins configuration.

  <classpathPrefix>dependency/</classpathPrefix>
  <addClasspath>true</addClasspath>
  

  So I'll have Class-Path entries like dependency/<artifactId>-<version>.jar, etc.

• After this I'm using the maven-assembly-plugin to create a distribution ZIP which contains my final JAR and the whole dependency/ folder.
• When I deploy my application I can run it like java -jar final.jar.

Primarily I've opted to use this solution, because in my project I use some Bouncy Castle JARs which JARs have some extravagant this and that inside their META-INF directory. When I used the maven-shade-plugin to create my final runnable JAR all hell broke loose and I got nasty that method couldn't be found and this signature is not quite right errors.

You should be doing something like this also. This Maven shade business is too shady to be useful (pun intended).

---

Here's a blog post on the whole process that I've tried to describe just above (thanks to baba), maybe it'll help someone in the future.

You could provide a default Answer in the construction of your mock that always throws an exception. Then every call that is stubbed will act like usual. Everything outside those paths will throw an exception. Something like this:

final String arg = "some arg";
Collection<Object> object = mock(Collection.class, new Answer<Object>() {
    @Override
    public Object answer(InvocationOnMock invocation) throws Throwable {
        throw new IllegalArgumentException("You cannot invoke " + invocation.getMethod() +
                                    " with " + Arrays.toString(invocation.getArguments()));
    }
});
doReturn(true).when(object).add(arg);

object.add(arg); // Goes ok
object.add("azertyuiop"); // Throws the exception

The following code sample from this Mockito documentation illustrates how to mock a void method:

doThrow(new RuntimeException()).when(mockedList).clear();

// following throws RuntimeException:
mockedList.clear();

For void methods, I think you need to use the doThrow syntax.

So in your case it would be:

doThrow(BookingException.builder().build())
      .when(booking)
      .validate(any());

Mockito matchers are static methods and calls to those methods, which stand in for arguments during calls to when and verify.

Hamcrest matchers (archived version) (or Hamcrest-style matchers) are stateless, general-purpose object instances that implement Matcher<T> and expose a method matches(T) that returns true if the object matches the Matcher's criteria. They are intended to be free of side effects, and are generally used in assertions such as the one below.

/* Mockito */  verify(foo).setPowerLevel(gt(9000));
/* Hamcrest */ assertThat(foo.getPowerLevel(), is(greaterThan(9000)));

Mockito matchers exist, separate from Hamcrest-style matchers, so that descriptions of matching expressions fit directly into method invocations: Mockito matchers return T where Hamcrest matcher methods return Matcher objects (of type Matcher<T>).

Mockito matchers are invoked through static methods such as eq, any, gt, and startsWith on org.mockito.Matchers and org.mockito.AdditionalMatchers. There are also adapters, which have changed across Mockito versions:

• For Mockito 1.x, Matchers featured some calls (such as intThat or argThat) are Mockito matchers that directly accept Hamcrest matchers as parameters. ArgumentMatcher<T> extended org.hamcrest.Matcher<T>, which was used in the internal Hamcrest representation and was a Hamcrest matcher base class instead of any sort of Mockito matcher.
• For Mockito 2.0+, Mockito no longer has a direct dependency on Hamcrest. Matchers calls phrased as intThat or argThat wrap ArgumentMatcher<T> objects that no longer implement org.hamcrest.Matcher<T> but are used in similar ways. Hamcrest adapters such as argThat and intThat are still available, but have moved to MockitoHamcrest instead.

Regardless of whether the matchers are Hamcrest or simply Hamcrest-style, they can be adapted like so:

/* Mockito matcher intThat adapting Hamcrest-style matcher is(greaterThan(...)) */
verify(foo).setPowerLevel(intThat(is(greaterThan(9000))));

In the above statement: foo.setPowerLevel is a method that accepts an int. is(greaterThan(9000)) returns a Matcher<Integer>, which wouldn't work as a setPowerLevel argument. The Mockito matcher intThat wraps that Hamcrest-style Matcher and returns an int so it can appear as an argument; Mockito matchers like gt(9000) would wrap that entire expression into a single call, as in the first line of example code.

What matchers do/return

when(foo.quux(3, 5)).thenReturn(true);

When not using argument matchers, Mockito records your argument values and compares them with their equals methods.

when(foo.quux(eq(3), eq(5))).thenReturn(true);    // same as above
when(foo.quux(anyInt(), gt(5))).thenReturn(true); // this one's different

When you call a matcher like any or gt (greater than), Mockito stores a matcher object that causes Mockito to skip that equality check and apply your match of choice. In the case of argumentCaptor.capture() it stores a matcher that saves its argument instead for later inspection.

Matchers return dummy values such as zero, empty collections, or null. Mockito tries to return a safe, appropriate dummy value, like 0 for anyInt() or any(Integer.class) or an empty List<String> for anyListOf(String.class). Because of type erasure, though, Mockito lacks type information to return any value but null for any() or argThat(...), which can cause a NullPointerException if trying to "auto-unbox" a null primitive value.

Matchers like eq and gt take parameter values; ideally, these values should be computed before the stubbing/verification starts. Calling a mock in the middle of mocking another call can interfere with stubbing.

Matcher methods can't be used as return values; there is no way to phrase thenReturn(anyInt()) or thenReturn(any(Foo.class)) in Mockito, for instance. Mockito needs to know exactly which instance to return in stubbing calls, and will not choose an arbitrary return value for you.

Implementation details

Matchers are stored (as Hamcrest-style object matchers) in a stack contained in a class called ArgumentMatcherStorage. MockitoCore and Matchers each own a ThreadSafeMockingProgress instance, which statically contains a ThreadLocal holding MockingProgress instances. It's this MockingProgressImpl that holds a concrete ArgumentMatcherStorageImpl. Consequently, mock and matcher state is static but thread-scoped consistently between the Mockito and Matchers classes.

Most matcher calls only add to this stack, with an exception for matchers like and, or, and not. This perfectly corresponds to (and relies on) the evaluation order of Java, which evaluates arguments left-to-right before invoking a method:

when(foo.quux(anyInt(), and(gt(10), lt(20)))).thenReturn(true);
[6]      [5]  [1]       [4] [2]     [3]

This will:

1. Add anyInt() to the stack.
2. Add gt(10) to the stack.
3. Add lt(20) to the stack.
4. Remove gt(10) and lt(20) and add and(gt(10), lt(20)).
5. Call foo.quux(0, 0), which (unless otherwise stubbed) returns the default value false. Internally Mockito marks quux(int, int) as the most recent call.
6. Call when(false), which discards its argument and prepares to stub method quux(int, int) identified in 5. The only two valid states are with stack length 0 (equality) or 2 (matchers), and there are two matchers on the stack (steps 1 and 4), so Mockito stubs the method with an any() matcher for its first argument and and(gt(10), lt(20)) for its second argument and clears the stack.

This demonstrates a few rules:

• Mockito can't tell the difference between quux(anyInt(), 0) and quux(0, anyInt()). They both look like a call to quux(0, 0) with one int matcher on the stack. Consequently, if you use one matcher, you have to match all arguments.

• Call order isn't just important, it's what makes this all work. Extracting matchers to variables generally doesn't work, because it usually changes the call order. Extracting matchers to methods, however, works great.

  int between10And20 = and(gt(10), lt(20));
  /* BAD */ when(foo.quux(anyInt(), between10And20)).thenReturn(true);
  // Mockito sees the stack as the opposite: and(gt(10), lt(20)), anyInt().
  
  public static int anyIntBetween10And20() { return and(gt(10), lt(20)); }
  /* OK */  when(foo.quux(anyInt(), anyIntBetween10And20())).thenReturn(true);
  // The helper method calls the matcher methods in the right order.
  

• The stack changes often enough that Mockito can't police it very carefully. It can only check the stack when you interact with Mockito or a mock, and has to accept matchers without knowing whether they're used immediately or abandoned accidentally. In theory, the stack should always be empty outside of a call to when or verify, but Mockito can't check that automatically. You can check manually with Mockito.validateMockitoUsage().

• In a call to when, Mockito actually calls the method in question, which will throw an exception if you've stubbed the method to throw an exception (or require non-zero or non-null values). doReturn and doAnswer (etc) do not invoke the actual method and are often a useful alternative.

• If you had called a mock method in the middle of stubbing (e.g. to calculate an answer for an eq matcher), Mockito would check the stack length against that call instead, and likely fail.

• If you try to do something bad, like stubbing/verifying a final method, Mockito will call the real method and also leave extra matchers on the stack. The final method call may not throw an exception, but you may get an InvalidUseOfMatchersException from the stray matchers when you next interact with a mock.

Common problems

• InvalidUseOfMatchersException:

  • Check that every single argument has exactly one matcher call, if you use matchers at all, and that you haven't used a matcher outside of a when or verify call. Matchers should never be used as stubbed return values or fields/variables.

  • Check that you're not calling a mock as a part of providing a matcher argument.

  • Check that you're not trying to stub/verify a final method with a matcher. It's a great way to leave a matcher on the stack, and unless your final method throws an exception, this might be the only time you realize the method you're mocking is final.

• NullPointerException with primitive arguments: (Integer) any() returns null while any(Integer.class) returns 0; this can cause a NullPointerException if you're expecting an int instead of an Integer. In any case, prefer anyInt(), which will return zero and also skip the auto-boxing step.

• NullPointerException or other exceptions: Calls to when(foo.bar(any())).thenReturn(baz) will actually call foo.bar(null), which you might have stubbed to throw an exception when receiving a null argument. Switching to doReturn(baz).when(foo).bar(any()) skips the stubbed behavior.

General troubleshooting

• Use MockitoJUnitRunner, or explicitly call validateMockitoUsage in your tearDown or @After method (which the runner would do for you automatically). This will help determine whether you've misused matchers.

• For debugging purposes, add calls to validateMockitoUsage in your code directly. This will throw if you have anything on the stack, which is a good warning of a bad symptom.

Here is what I did to test my code using Mockito and Apache HttpBuilder:

Class under test:

import java.io.BufferedReader;
import java.io.IOException;

import javax.ws.rs.core.Response.Status;

import org.apache.http.HttpResponse;
import org.apache.http.client.HttpClient;
import org.apache.http.client.methods.HttpGet;
import org.apache.http.impl.client.HttpClientBuilder;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

public class StatusApiClient {
private static final Logger LOG = LoggerFactory.getLogger(StatusApiClient.class);

    private String targetUrl = "";
    private HttpClient client = null;
    HttpGet httpGet = null;

    public StatusApiClient(HttpClient client, HttpGet httpGet) {
        this.client = client;
        this.httpGet = httpGet;
    }

    public StatusApiClient(String targetUrl) {
        this.targetUrl = targetUrl;
        this.client = HttpClientBuilder.create().build();
        this.httpGet = new HttpGet(targetUrl);
    }

    public boolean getStatus() {
        BufferedReader rd = null;
        boolean status = false;
        try{
            LOG.debug("Requesting status: " + targetUrl);


            HttpResponse response = client.execute(httpGet);

            if(response.getStatusLine().getStatusCode() == Status.OK.getStatusCode()) {
                LOG.debug("Is online.");
                status = true;
            }

        } catch(Exception e) {
            LOG.error("Error getting the status", e);
        } finally {
            if (rd != null) {
                try{
                    rd.close();
                } catch (IOException ioe) {
                    LOG.error("Error while closing the Buffered Reader used for reading the status", ioe);
                }
            }   
        }

        return status;
    }
}

Test:

import java.io.IOException;

import org.apache.http.HttpResponse;
import org.apache.http.StatusLine;
import org.apache.http.client.ClientProtocolException;
import org.apache.http.client.HttpClient;
import org.apache.http.client.methods.HttpGet;
import org.apache.http.conn.HttpHostConnectException;
import org.junit.Assert;
import org.junit.Test;
import org.mockito.Mockito;

public class StatusApiClientTest extends Mockito {

    @Test
    public void should_return_true_if_the_status_api_works_properly() throws ClientProtocolException, IOException {
        //given:
        HttpClient httpClient = mock(HttpClient.class);
        HttpGet httpGet = mock(HttpGet.class);
        HttpResponse httpResponse = mock(HttpResponse.class);
        StatusLine statusLine = mock(StatusLine.class);

        //and:
        when(statusLine.getStatusCode()).thenReturn(200);
        when(httpResponse.getStatusLine()).thenReturn(statusLine);
        when(httpClient.execute(httpGet)).thenReturn(httpResponse);

        //and:
        StatusApiClient client = new StatusApiClient(httpClient, httpGet);

        //when:
        boolean status = client.getStatus();

        //then:
        Assert.assertTrue(status);
    }

    @Test
    public void should_return_false_if_status_api_do_not_respond() throws ClientProtocolException, IOException {
        //given:
        HttpClient httpClient = mock(HttpClient.class);
        HttpGet httpGet = mock(HttpGet.class);
        HttpResponse httpResponse = mock(HttpResponse.class);
        StatusLine statusLine = mock(StatusLine.class);

        //and:
        when(httpClient.execute(httpGet)).thenThrow(HttpHostConnectException.class);

        //and:
        StatusApiClient client = new StatusApiClient(httpClient, httpGet);

        //when:
        boolean status = client.getStatus();

        //then:
        Assert.assertFalse(status);
    }

}

What do you think folks, do I need to improve something? (Yeah, I know, the comments. That is something I brought from my Spock background :D)

From the docs:

Sometimes we need to stub with different return value/exception for the same method call. Typical use case could be mocking iterators. Original version of Mockito did not have this feature to promote simple mocking. For example, instead of iterators one could use Iterable or simply collections. Those offer natural ways of stubbing (e.g. using real collections). In rare scenarios stubbing consecutive calls could be useful, though:

when(mock.someMethod("some arg"))
   .thenThrow(new RuntimeException())
  .thenReturn("foo");

//First call: throws runtime exception:
mock.someMethod("some arg");

//Second call: prints "foo"
System.out.println(mock.someMethod("some arg"));

So in your case, you'd want:

when(myMock.doTheCall())
   .thenReturn("You failed")
   .thenReturn("Success");

I think the reason may be that mock object libraries typically create mocks by dynamically creating classes at runtime (using cglib). This means they either implement an interface at runtime (that's what EasyMock does if I'm not mistaken), or they inherit from the class to mock (that's what Mockito does if I'm not mistaken). Both approaches do not work for static members, since you can't override them using inheritance.

The only way to mock statics is to modify a class' byte code at runtime, which I suppose is a little more involved than inheritance.

That's my guess at it, for what it's worth...