How to use testFailure method of org.mockito.runners.ConsoleSpammingMockitoJUnitRunner class

I found this trying to solve a similar issue creating a Mockito stub with a Map parameter. I didn't want to write a custom matcher for the Map in question and then I found a more elegant solution: use the additional matchers in hamcrest-library with mockito's argThat:

when(mock.search(argThat(hasEntry("xpath", "PRICE"))).thenReturn("$100.00");

If you need to check against multiple entries then you can use other hamcrest goodies:

when(mock.search(argThat(allOf(hasEntry("xpath", "PRICE"), hasEntry("otherKey", "otherValue")))).thenReturn("$100.00");

This starts to get long with non-trivial maps, so I ended up extracting methods to collect the entry matchers and stuck them in our TestUtils:

import static org.hamcrest.Matchers.allOf;
import static org.hamcrest.Matchers.anyOf;
import static org.hamcrest.Matchers.hasEntry;

import java.util.ArrayList;
import java.util.List;
import java.util.Map;

import org.hamcrest.Matcher;
---------------------------------
public static <K, V> Matcher<Map<K, V>> matchesEntriesIn(Map<K, V> map) {
    return allOf(buildMatcherArray(map));
}

public static <K, V> Matcher<Map<K, V>> matchesAnyEntryIn(Map<K, V> map) {
    return anyOf(buildMatcherArray(map));
}

@SuppressWarnings("unchecked")
private static <K, V> Matcher<Map<? extends K, ? extends V>>[] buildMatcherArray(Map<K, V> map) {
    List<Matcher<Map<? extends K, ? extends V>>> entries = new ArrayList<Matcher<Map<? extends K, ? extends V>>>();
    for (K key : map.keySet()) {
        entries.add(hasEntry(key, map.get(key)));
    }
    return entries.toArray(new Matcher[entries.size()]);
}

So I'm left with:

when(mock.search(argThat(matchesEntriesIn(map))).thenReturn("$100.00");
when(mock.search(argThat(matchesAnyEntryIn(map))).thenReturn("$100.00");

There's some ugliness associated with the generics and I'm suppressing one warning, but at least it's DRY and hidden away in the TestUtil.

One last note, beware the embedded hamcrest issues in JUnit 4.10. With Maven, I recommend importing hamcrest-library first and then JUnit 4.11 (now 4.12) and exclude hamcrest-core from JUnit just for good measure:

<dependency>
    <groupId>org.hamcrest</groupId>
    <artifactId>hamcrest-library</artifactId>
    <version>1.3</version>
    <scope>test</scope>
</dependency>
<dependency>
    <groupId>junit</groupId>
    <artifactId>junit</artifactId>
    <version>4.12</version>
    <scope>test</scope>
    <exclusions>
        <exclusion>
            <groupId>org.hamcrest</groupId>
            <artifactId>hamcrest-core</artifactId>
        </exclusion>
    </exclusions>
</dependency>
<dependency>
    <groupId>org.mockito</groupId>
    <artifactId>mockito-all</artifactId>
    <version>1.9.5</version>
    <scope>test</scope>
</dependency>

---

Edit: Sept 1, 2017 - Per some of the comments, I updated my answer to show my Mockito dependency, my imports in the test util, and a junit that is running green as of today:

import static blah.tool.testutil.TestUtil.matchesAnyEntryIn;
import static blah.tool.testutil.TestUtil.matchesEntriesIn;
import static org.hamcrest.MatcherAssert.assertThat;
import static org.hamcrest.Matchers.is;
import static org.mockito.Matchers.argThat;
import static org.mockito.Mockito.mock;
import static org.mockito.Mockito.when;

import java.util.HashMap;
import java.util.Map;

import org.junit.Test;

public class TestUtilTest {

    @Test
    public void test() {
        Map<Integer, String> expected = new HashMap<Integer, String>();
        expected.put(1, "One");
        expected.put(3, "Three");

        Map<Integer, String> actual = new HashMap<Integer, String>();
        actual.put(1, "One");
        actual.put(2, "Two");

        assertThat(actual, matchesAnyEntryIn(expected));

        expected.remove(3);
        expected.put(2, "Two");
        assertThat(actual, matchesEntriesIn(expected));
    }

    @Test
    public void mockitoTest() {
        SystemUnderTest sut = mock(SystemUnderTest.class);
        Map<Integer, String> expected = new HashMap<Integer, String>();
        expected.put(1, "One");
        expected.put(3, "Three");

        Map<Integer, String> actual = new HashMap<Integer, String>();
        actual.put(1, "One");

        when(sut.search(argThat(matchesAnyEntryIn(expected)))).thenReturn("Response");
        assertThat(sut.search(actual), is("Response"));
    }

    protected class SystemUnderTest {
        // We don't really care what this does
        public String search(Map<Integer, String> map) {
            if (map == null) return null;
            return map.get(0);
        }
    }
}

The first issue is that you have to use spyTemp object to expect something from Mockito. Here it is not the same as test. spyTemp is wrapped by the Mockito object temp.

Another issue is that you stub only methodB(), but you are trying to run methodA(). Yes in your implementation of methodA() you call methodB(), but you call this.methodB(), not spyTemp.methodB(). Here you have to understand that mocking would work only when you call it on the instance of temp. It's wrapped by a Mockito proxy which catches your call, and if you have overriden some method, it will call your new implementation instead of the original one. But since the original method is called, inside it you know nothing about Mockito proxy. So your "overriden" method would be called only when you run spyTemp.methodB()

This should work:

Mockito.doReturn(true).when(spyTemp).methodB(Mockito.any()); 
boolean status = spyTemp.methodA("XYZ");

TL; DR:

Yes, it's pretty safe to use and I'd recommend using it. (May 2022)

---

Original Answer

Just started using Lombok today. So far I like it, but one drawback I didn't see mentioned was refactoring support.

If you have a class annotated with @Data, it will generate the getters and setters for you based on the field names. If you use one of those getters in another class, then decide the field is poorly named, it will not find usages of those getters and setters and replace the old name with the new name.

I would imagine this would have to be done via an IDE plug-in and not via Lombok.

UPDATE (Jan 22 '13)
After using Lombok for 3 months, I still recommend it for most projects. I did, however, find another drawback that is similar to the one listed above.

If you have a class, say MyCompoundObject.java that has 2 members, both annotated with @Delegate, say myWidgets and myGadgets, when you call myCompoundObject.getThingies() from another class, it's impossible to know if it's delegating to the Widget or Gadget because you can no longer jump to source within the IDE.

Using the Eclipse "Generate Delegate Methods..." provides you with the same functionality, is just as quick and provides source jumping. The downside is it clutters your source with boilerplate code that take the focus off the important stuff.

UPDATE 2 (Feb 26 '13)
After 5 months, we're still using Lombok, but I have some other annoyances. The lack of a declared getter & setter can get annoying at times when you are trying to familiarize yourself with new code.

For example, if I see a method called getDynamicCols() but I don't know what it's about, I have some extra hurdles to jump to determine the purpose of this method. Some of the hurdles are Lombok, some are the lack of a Lombok smart plugin. Hurdles include:

• Lack of JavaDocs. If I javadoc the field, I would hope the getter and setter would inherit that javadoc through the Lombok compilation step.
• Jump to method definition jumps me to the class, but not the property that generated the getter. This is a plugin issue.
• Obviously you are not able to set a breakpoint in a getter/setter unless you generate or code the method.
• NOTE: This Reference Search is not an issue as I first thought it was. You do need to be using a perspective that enables the Outline view though. Not a problem for most developers. My problem was I am using Mylyn which was filtering my Outline view, so I didn't see the methods. Lack of References search. If I want to see who's calling getDynamicCols(args...), I have to generate or code the setter to be able to search for references.

UPDATE 3 (Mar 7 '13)
Learning to use the various ways of doing things in Eclipse I guess. You can actually set a conditional breakpoint (BP) on a Lombok generated method. Using the Outline view, you can right-click the method to Toggle Method Breakpoint. Then when you hit the BP, you can use the debugging Variables view to see what the generated method named the parameters (usually the same as the field name) and finally, use the Breakpoints view to right-click the BP and select Breakpoint Properties... to add a condition. Nice.

UPDATE 4 (Aug 16 '13)
Netbeans doesn't like it when you update your Lombok dependencies in your Maven pom. The project still compiles, but files get flagged for having compilation errors because it can't see the methods Lombok is creating. Clearing the Netbeans cache resolves the issue. Not sure if there is a "Clean Project" option like there is in Eclipse. Minor issue, but wanted to make it known.

UPDATE 5 (Jan 17 '14)
Lombok doesn't always play nice with Groovy, or at least the groovy-eclipse-compiler. You might have to downgrade your version of the compiler. Maven Groovy and Java + Lombok

UPDATE 6 (Jun 26 '14)
A word of warning. Lombok is slightly addictive and if you work on a project where you can't use it for some reason, it will annoy the piss out of you. You may be better off just never using it at all.

UPDATE 7 (Jul 23 '14)
This is a bit of an interesting update because it directly addresses the safety of adopting Lombok that the OP asked about.

As of v1.14, the @Delegate annotation has been demoted to an Experimental status. The details are documented on their site (Lombok Delegate Docs).

The thing is, if you were using this feature, your backout options are limited. I see the options as:

• Manually remove @Delegate annotations and generate/handcode the delegate code. This is a little harder if you were using attributes within the annotation.
• Delombok the files that have the @Delegate annotation and maybe add back in the annotations that you do want.
• Never update Lombok or maintain a fork (or live with using experiential features).
• Delombok your entire project and stop using Lombok.

As far as I can tell, Delombok doesn't have an option to remove a subset of annotations; it's all or nothing at least for the context of a single file. I opened a ticket to request this feature with Delombok flags, but I wouldn't expect that in the near future.

UPDATE 8 (Oct 20 '14)
If it's an option for you, Groovy offers most of the same benefits of Lombok, plus a boat load of other features, including @Delegate. If you think you'll have a hard time selling the idea to the powers that be, take a look at the @CompileStatic or @TypeChecked annotation to see if that can help your cause. In fact, the primary focus of the Groovy 2.0 release was static safety.

UPDATE 9 (Sep 1 '15)
Lombok is still being actively maintained and enhanced, which bodes well to the safety level of adoption. The @Builder annotations is one of my favorite new features.

UPDATE 10 (Nov 17 '15)
This may not seem directly related to the OP's question, but worth sharing. If you're looking for tools to help you reduce the amount of boilerplate code you write, you can also check out Google Auto - in particular AutoValue. If you look at their slide deck, the list Lombok as a possible solution to the problem they are trying to solve. The cons they list for Lombok are:

• The inserted code is invisible (you can't "see" the the methods it generates) [ed note - actually you can, but it just requires a decompiler]
• The compiler hacks are non-standard and fragile
• "In our view, your code is no longer really Java"

I'm not sure how much I agree with their evaluation. And given the cons of AutoValue that are documented in the slides, I'll be sticking with Lombok (if Groovy is not an option).

UPDATE 11 (Feb 8 '16)
I found out Spring Roo has some similar annotations. I was a little surprised to find out Roo is still a thing and finding documentation for the annotations is a bit rough. Removal also doesn't look as easy as de-lombok. Lombok seems like the safer choice.

UPDATE 12 (Feb 17 '16)
While trying to come up with justifications for why it's safe to bring in Lombok for the project I'm currently working on, I found a piece of gold that was added with v1.14 - The Configuration System! This is means you can configure a project to dis-allow certain features that your team deems unsafe or undesirable. Better yet, it can also create directory specific config with different settings. This is AWESOME.

UPDATE 13 (Oct 4 '16)
If this kind of thing matters to you, Oliver Gierke felt it was safe to add Lombok to Spring Data Rest.

UPDATE 14 (Sep 26 '17)
As pointed out by @gavenkoa in the comments on the OPs question, JDK9 compiler support isn't yet available (Issue #985). It also sounds like it's not going to be an easy fix for the Lombok team to get around.

UPDATE 15 (Mar 26 '18)
The Lombok changelog indicates as of v1.16.20 "Compiling lombok on JDK1.9 is now possible" even though #985 is still open.

Changes to accommodate JDK9, however, necessitated some breaking changes; all isolated to changes in config defaults. It's a little concerning that they introduced breaking changes, but the version only bumped the "Incremental" version number (going from v1.16.18 to v1.16.20). Since this post was about the safety, if you had a yarn/npm like build system that automatically upgraded to the latest incremental version, you might be in for a rude awakening.

UPDATE 16 (Jan 9 '19)

It seems the JDK9 issues have been resolved and Lombok works with JDK10, and even JDK11 as far as I can tell.

One thing I noticed though that was concerning from a safety aspect is the fact that the change log going from v1.18.2 to v1.18.4 lists two items as BREAKING CHANGE!? I'm not sure how a breaking change happens in a semver "patch" update. Could be an issue if you use a tool that auto-updates patch versions.

UPDATE 17 (Mar 17 '21)

There is some drama unfolding between the Lombok developers and an OpenJDK developer around JDK 16. The JDK developers argue that Lombok is taking advantage of unpublished JDK internals via loopholes the JDK team would like to close, but have intentionally left open for various reasons.

They have stated their concern (about the safety of Lombok) as such:

All access to internals will remain available as before, provided that the client application explicitly allows it, acknowledging that it is knowingly taking on any maintenance (or security) issue this might entail.

While Lombok might think they're deceiving OpenJDK, all they're doing is announcing that it is their intention to deceive their own users.

There may come a day soon where Lombok will not be able to find any more creative solutions around the JDK's security restrictions. Even if they do, the safety of using Lombok in your project may be in question.

UPDATE 18 (May 11 '22)

A recent comment asked for a summary, so I put that at the top.

The short answer is it's perfectly safe to use and I'd highly recommend using it if we're writing Java code.

Given that support for JDK 17 has been out for awhile and was released less than a month after the JDK was officially released, the safety of Lombok sticking around is high. And you can always de-lombok if you need to.

As a consultant, I get to see how a lot of different companies write code. Every client I've had for the past 5 years has used Lombok. These have all been Fortune 1000 companies. It speeds development and makes it less error prone.

That said, you still need to keep up on the latest features of the JDK. Consider using the Java record keyword to make your objects immutable instead of some Lombok feature. Use Lombok where it makes sense. Use the Lombok config options to prevent usage of it in ways you don't agree with.

So unless something major happens, this will probably be my last update to this answer. Thanks for all the votes. I'm glad it helps.

The following, running under Mockito 1.9.0 and JUnit 4.8.2, does not print anything to my console:

import static org.mockito.Mockito.*;

import org.junit.Test;

public class TestNonCall {
  public class TestClass {
    public String test() {
      System.out.println("test called!");
      return "test";
    }
  }

  @Test
  public void doTest() {
    final TestClass mock = mock(TestClass.class);

    when(mock.test()).thenReturn("mock!");
  }
}

Further, if I put a breakpoint in the test() method it is never hit.

Perhaps post more code? It looks like your example is not complex enough to demonstrate the behaviour you're having problems with.

Also: are you using the latest version of Mockito?

Edit: New Thought: Are You Mocking a Final Method?

If you add a final modifier to the method you are mocking, you get the behaviour you reported.

This is because Mockito does not mock final and static methods. Instead, it delegates the calls to the real implementation.

Might your actual code be attempting to mock a final method?

If so, you can use PowerMock, which is an extension to Mockito that allows mocking final methods.

You would need to add the following annotations to your test case class:

@RunWith(PowerMockRunner.class)
@PrepareForTest(TestClass.class)
public class TestNonCall {
  // ...
}

and mock the class using the PowerMock method in your test method:

final TestClass mock = PowerMockito.mock(TestClass.class);

then proceed as usual.

Also, a slightly more terse way to get around this is to use the do syntax instead of when.

doReturn(User.class).when(methodParameter).getParameterType();

Access MainConfig.xml directly. The src/test/resources directory contents are placed in the root of your CLASSPATH.

More precisely: contents of src/test/resources are copied into target/test-classes, so if you have the following project structure:

.
└── src
    └── test
        ├── java
        │   └── foo
        │       └── C.java
        └── resources
            ├── a.xml
            └── foo
                └── b.xml

It will result with the following test CLASSPATH contents:

• /foo/C.class
• /a.xml
• /foo/b.xml

To actually access the files from Java source, use getClass().getResource("/MainConfig.xml").getFile().

Make sure you add @RunWith(PowerMockRunner.class) to the top of your class as well.

::edit:: two years later...

Don't ever use PowerMockito, you shouldn't need to.

If you do need to, you have most likely broken the SOLID principles and your design is wrong.

Fix your design instead.

You could use atLeast(0) in combination with ArgumentCaptor:

ArgumentCaptor<String> propertyKeyCaptor = ArgumentCaptor.forClass(String.class);
Mockito.verify(foo, atLeast(0)).getProperty(propertyKeyCaptor.capture(), anyString());

ArgumentCaptor<String> propertyKeyCaptor2 = ArgumentCaptor.forClass(String.class);
Mockito.verify(foo, atLeast(0)).getProperty(propertyKeyCaptor2.capture());

List<String> propertyKeyValues = propertyKeyCaptor.getAllValues();
List<String> propertyKeyValues2 = propertyKeyCaptor2.getAllValues();

assertTrue(!propertyKeyValues.isEmpty() || !propertyKeyValues2.isEmpty()); //JUnit assert -- modify for whatever testing framework you're using

It should be something like

@RunWith(SpringJUnit4ClassRunner.class)
public class aTest () {

    @Mock
    private B b;

    @Mock
    private C c;

    @Autowired
    @InjectMocks
    private A a;

}

If you want D to be Autowired dont need to do anything in your Test class. Your Autowired A should have correct instance of D. Also i think you need to use SpringJUnit4ClassRunner for Autowiring to work, with contextConfiguration set correctly. Because you are not using MockitoJunitRunner you need to initialize your mocks yourself using

MockitoAnnotations.initMocks(java.lang.Object testClass)