How to use validateTestMethods method of org.junit.runners.BlockJUnit4Runner class

Some cases where I have found it useful:

• mark a test that is incomplete, so it fails and warns you until you can finish it
• making sure an exception is thrown:

try{
  // do stuff...
  fail("Exception not thrown");
}catch(Exception e){
  assertTrue(e.hasSomeFlag());
}

Note:

Since JUnit4, there is a more elegant way to test that an exception is being thrown: Use the annotation @Test(expected=IndexOutOfBoundsException.class)

However, this won't work if you also want to inspect the exception, then you still need fail().

Part of the problem has been that I do not know every method call out there that clears the interrupt flag.

It is important to clarify that the following methods clear the interrupt flag by just calling them:

Thread.interrupted()
Thread.isInterrupted(true) -- added to your list

For this reason Thread.currentThread().isInterrupted() should always be used instead.

The following methods will clear the interrupted flag by immediately throwing InterruptedException either if they were called and then the thread was interrupted or if the thread was already interrupted and then they were called (see junit code below). So it is not the method that clears the flag, throwing the exception does.

Your initial list:

Thread.interrupted()
Thread.sleep(long)
Thread.join()
Thread.join(long)
Object.wait()
Object.wait(long)

Added to your list:

Thread.sleep(long, int)
Thread.join(int, long)
Thread.isInterrupted(true)
Object.wait(int, long)
BlockingQueue.put(...)
BlockingQueue.offer(...)
BlockingQueue.take(...)
BlockingQueue.poll(...)
Future.get(...)
Process.waitFor()
ExecutorService.invokeAll(...)
ExecutorService.invokeAny(...)
ExecutorService.awaitTermination(...)
CompletionService.poll(...)
CompletionService.take(...)
CountDownLatch.await(...)
CyclicBarrier.await(...)
Semaphore.acquire(...)
Semaphore.tryAcquire(...)
Lock.lockInteruptibly()
Lock.tryLock(...)

Please note that the proper pattern with any code that catches InterruptedException is to immediately re-interrupt the thread. We do this in case others are relying on the thread.isInterrupted() method:

try {
    ...
} catch (InterruptedException e) {
    // immediately re-interrupt the thread
    Thread.currentThread().interrupt();
    // log the exception or [likely] quit the thread
}

JUnit code that demonstrates some of this:

assertFalse(Thread.currentThread().isInterrupted());
// you can do this from another thread by saying: someThread.interrupt();
Thread.currentThread().interrupt();
// this method does _not_ clear the interrupt flag
assertTrue(Thread.currentThread().isInterrupted());
// but this one _does_ and should probably not be used
assertTrue(Thread.interrupted());
assertFalse(Thread.currentThread().isInterrupted());
Thread.currentThread().interrupt();
assertTrue(Thread.currentThread().isInterrupted());
try {
    // this throws immediately because the thread is _already_ interrupted
    Thread.sleep(1);
    fail("will never get here");
} catch (InterruptedException e) {
    // and when the InterruptedException is throw, it clears the interrupt
    assertFalse(Thread.currentThread().isInterrupted());
    // we should re-interrupt the thread so other code can use interrupt status
    Thread.currentThread().interrupt();
}
assertTrue(Thread.currentThread().isInterrupted());

For the mocks initialization, using the runner or the MockitoAnnotations.initMocks are strictly equivalent solutions. From the javadoc of the MockitoJUnitRunner :

JUnit 4.5 runner initializes mocks annotated with Mock, so that explicit usage of MockitoAnnotations.initMocks(Object) is not necessary. Mocks are initialized before each test method.

The first solution (with the MockitoAnnotations.initMocks) could be used when you have already configured a specific runner (SpringJUnit4ClassRunner for example) on your test case.

The second solution (with the MockitoJUnitRunner) is the more classic and my favorite. The code is simpler. Using a runner provides the great advantage of automatic validation of framework usage (described by @David Wallace in this answer).

Both solutions allows to share the mocks (and spies) between the test methods. Coupled with the @InjectMocks, they allow to write unit tests very quickly. The boilerplate mocking code is reduced, the tests are easier to read. For example:

@RunWith(MockitoJUnitRunner.class)
public class ArticleManagerTest {

    @Mock private ArticleCalculator calculator;
    @Mock(name = "database") private ArticleDatabase dbMock;
    @Spy private UserProvider userProvider = new ConsumerUserProvider();

    @InjectMocks private ArticleManager manager;

    @Test public void shouldDoSomething() {
        manager.initiateArticle();
        verify(database).addListener(any(ArticleListener.class));
    }

    @Test public void shouldDoSomethingElse() {
        manager.finishArticle();
        verify(database).removeListener(any(ArticleListener.class));
    }
}

Pros: The code is minimal

Cons: Black magic. IMO it is mainly due to the @InjectMocks annotation. With this annotation "you loose the pain of code" (see the great comments of @Brice)

The third solution is to create your mock on each test method. It allow as explained by @mlk in its answer to have "self contained test".

public class ArticleManagerTest {

    @Test public void shouldDoSomething() {
        // given
        ArticleCalculator calculator = mock(ArticleCalculator.class);
        ArticleDatabase database = mock(ArticleDatabase.class);
        UserProvider userProvider = spy(new ConsumerUserProvider());
        ArticleManager manager = new ArticleManager(calculator, 
                                                    userProvider, 
                                                    database);

        // when 
        manager.initiateArticle();

        // then 
        verify(database).addListener(any(ArticleListener.class));
    }

    @Test public void shouldDoSomethingElse() {
        // given
        ArticleCalculator calculator = mock(ArticleCalculator.class);
        ArticleDatabase database = mock(ArticleDatabase.class);
        UserProvider userProvider = spy(new ConsumerUserProvider());
        ArticleManager manager = new ArticleManager(calculator, 
                                                    userProvider, 
                                                    database);

        // when 
        manager.finishArticle();

        // then
        verify(database).removeListener(any(ArticleListener.class));
    }
}

Pros: You clearly demonstrate how your api works (BDD...)

Cons: there is more boilerplate code. (The mocks creation)

My recommandation is a compromise. Use the @Mock annotation with the @RunWith(MockitoJUnitRunner.class), but do not use the @InjectMocks :

@RunWith(MockitoJUnitRunner.class)
public class ArticleManagerTest {

    @Mock private ArticleCalculator calculator;
    @Mock private ArticleDatabase database;
    @Spy private UserProvider userProvider = new ConsumerUserProvider();

    @Test public void shouldDoSomething() {
        // given
        ArticleManager manager = new ArticleManager(calculator, 
                                                    userProvider, 
                                                    database);

        // when 
        manager.initiateArticle();

        // then 
        verify(database).addListener(any(ArticleListener.class));
    }

    @Test public void shouldDoSomethingElse() {
        // given
        ArticleManager manager = new ArticleManager(calculator, 
                                                    userProvider, 
                                                    database);

        // when 
        manager.finishArticle();

        // then 
        verify(database).removeListener(any(ArticleListener.class));
    }
}

Pros: You clearly demonstrate how your api works (How my ArticleManager is instantiated). No boilerplate code.

Cons: The test is not self contained, less pain of code

As noted in another answer, if you don't care about the order, you might do best to change the interface so it doesn't care about the order.

If order matters in the code but not in a specific test, you can use the ArgumentCaptor as you did. It clutters the code a bit.

If this is something you might do in multiple tests, you might do better to use appropriate Mockito Matchers or Hamcrest Matchers, or roll your own (if you don't find one that fills the need). A hamcrest matcher might be best as it can be used in other contexts besides mockito.

For this example you could create a hamcrest matcher as follows:

import org.hamcrest.BaseMatcher;
import org.hamcrest.Description;
import org.hamcrest.Matcher;

import java.util.Collections;
import java.util.HashSet;
import java.util.List;
import java.util.Set;

public class MyMatchers {
    public  static <T> Matcher<List<T>> sameAsSet(final List<T> expectedList) {
        return new BaseMatcher<List<T>>(){
            @Override
            public boolean matches(Object o) {
                List<T> actualList = Collections.EMPTY_LIST;
                try {
                    actualList = (List<T>) o;
                }
                catch (ClassCastException e) {
                    return false;
                }
                Set<T> expectedSet = new HashSet<T>(expectedList);
                Set<T> actualSet = new HashSet<T>(actualList);
                return actualSet.equals(expectedSet);
            }

            @Override
            public void describeTo(Description description) {
                description.appendText("should contain all and only elements of ").appendValue(expectedList);
            }
        };
    }
}

And then the verify code becomes:

verify(mockClassB).sendEvent(argThat(MyMatchers.sameAsSet(expectedFileList)));

If you instead created a mockito matcher, you wouldn't need the argThat, which basically wraps a hamcrest matcher in a mockito matcher.

This moves the logic of sorting or converting to set out of your test and makes it reusable.

In order to use @Rule, you require a class that implements TestRule(preferred) or MethodRule, as can be read here. Whereas @Before and @After require a new method to be written in every test case, @Rule does not because it is only an instantiation of already existing code.

So, if you would use @Before and @After for setUp() and tearDown() that you'll be using in many test cases, it is actually a better idea to use @Rule because of code reuse. If you have a test case that requires a unique @Before and/or @After, then these annotations are preferable.

For a bit more elaborate answer with a couple examples, take a look here. Ajit explains it very well.

For the future I would recommend Eran Harel's answer (refactoring moving new to factory that can be mocked). But if you don't want to change the original source code, use very handy and unique feature: spies. From the documentation:

You can create spies of real objects. When you use the spy then the real methods are called (unless a method was stubbed).

Real spies should be used carefully and occasionally, for example when dealing with legacy code.

In your case you should write:

TestedClass tc = spy(new TestedClass());
LoginContext lcMock = mock(LoginContext.class);
when(tc.login(anyString(), anyString())).thenReturn(lcMock);

The answer is now to create a @ClassRule within your suite. The rule will be invoked before or after (depending on how you implement it) each test class is run. There are a few different base classes you can extend/implement. What is nice about class rules is that if you do not implement them as anonymous classes then you can reuse the code!

Here is an article about them: http://java.dzone.com/articles/junit-49-class-and-suite-level-rules

Here is some sample code to illustrate their use. Yes, it is trivial, but it should illustrate the life-cycle well enough for you to get started.

First the suite definition:

import org.junit.*;
import org.junit.rules.ExternalResource;
import org.junit.runners.Suite;
import org.junit.runner.RunWith;


@RunWith( Suite.class )
@Suite.SuiteClasses( { 
    RuleTest.class,
} )
public class RuleSuite{

    private static int bCount = 0;
    private static int aCount = 0;

    @ClassRule
    public static ExternalResource testRule = new ExternalResource(){
            @Override
            protected void before() throws Throwable{
                System.err.println( "before test class: " + ++bCount );
                sss = "asdf";
            };

            @Override
            protected void after(){
                System.err.println( "after test class: " + ++aCount );
            };
        };


    public static String sss;
}

And now the test class definition:

import static org.junit.Assert.*;

import org.junit.ClassRule;
import org.junit.Rule;
import org.junit.Test;
import org.junit.rules.ExternalResource;

public class RuleTest {

    @Test
    public void asdf1(){
        assertNotNull( "A value should've been set by a rule.", RuleSuite.sss );
    }

    @Test
    public void asdf2(){
        assertEquals( "This value should be set by the rule.", "asdf", RuleSuite.sss );
    }
}

Think about exactly what it is you're trying to achieve. Do you want to test object identity or verify that the two objects have exactly the same data? From your suggestion of assertEquals I am guessing you want to go field-by-field.

If the objects are shallow you can use

assertThat(actual, samePropertyValuesAs(expected));

This fails when the object is a composite though. If you want to walk the entire graph you can use the sameBeanAs matcher we wrote some time back to address this issue:

assertThat(actual, sameBeanAs(expected));

If you're using AssertJ then you can also use the built-in functionality like this:

assertThat(actual).isEqualToComparingFieldByField(expected);

Updated in 06/21 - apparently this is deprecated and has been replaced with:

assertThat(actual).usingRecursiveComparison().isEqualTo(expected)

One thing you don't want to do it override the equals method for this as that might change in the future to accommodate business need.

I just don't understand why we need to have nested test class in our test.

@Nested makes really sense to organize big test classes.

Typical use case

Very often, developer teams define a test class by class to test. That is a shared good practice but it also may make your test class very big and to count several hundred of lines. You can indeed have classes to test with multiple methods to test, multiple scenarios for each one and also some initialization steps required in the unit test methods to test the scenarios.
All of these will naturally increase the test class size.
Above a threshold (maybe 500 lines or about), it becomes legitimate to ask yourself whether a refactoring is needed.

A big class (test class or not), even well organized is harder to read, maintain than multiple classes grouping things with high cohesion/relationship between.
In the unit tests cases, it can be sometime still worse because you may not find a test scenario and write a new one while it existed but you didn't manage to find it because the test class is big.

@Nested : the solution

@Nested addresses this issue by giving the possibility to group multiple test methods inside multiple nested classes of a main(outer) test class.
The test methods of all nested classes defined in the main(outer) test class are handled as any test methods. So @BeforeEach, @AfterEach, @ExtendWith... are applied for all the them.
The single exception is @BeforeAll and @AfterAll :

Only non-static nested classes (i.e. inner classes) can serve as @Nested test classes. Nesting can be arbitrarily deep, and those inner classes are considered to be full members of the test class family with one exception: @BeforeAll and @AfterAll methods do not work by default. The reason is that Java does not allow static members in inner classes. However, this restriction can be circumvented by annotating a @Nested test class with @TestInstance(Lifecycle.PER_CLASS) (see Test Instance Lifecycle).

Using @Nested combined with @DisplayName that takes a String value becomes still finer as the display name will be used for test reporting in IDEs and build tools and may contain spaces, special characters, and even emoji.

Example

I have a FooService with multiple methods and multiple scenarios. I can groups scenarios of the same concern inside nested classes of the unit test class.
Here I choose the method to test to group them (so I group by scenario) but the discriminator could be another thing if it makes sense.

For example :

public class FooServiceTest {

    Foo foo;

    // invoked for ALL test methods
    @BeforeEach
    public void beforeEach() {
         Foo foo = new Foo(...);
    }

    @Nested
    @DisplayName("findWith methods")
    class FindMethods {
        @Test
        void findWith_when_X() throws Exception {
             //...
             foo.findWith(...);
             //...
        }
        @Test
        void findWith_when_Y() throws Exception {
             //...
             foo.findWith(...);
             //...

        }
        @Test
        void findWith_when_Z() throws Exception {
             //...
             foo.findWith(...);
             //...
        }           
    }

    @Nested
    @DisplayName("findAll methods")
    class FindAllMethods {
        @Test
        void findAll_when_X() throws Exception {
             //...
             foo.findAll(...);
             //...
        }
        @Test
        void findAll_when_Y() throws Exception {
             //...
             foo.findAll(...);
             //...

        }
        @Test
        void findAll_when_Z() throws Exception {
             //...
             foo.findAll(...);
             //...
        }   
    }   

    @Nested
    @DisplayName("computeBar methods")
    class ComputeBarMethods {   
         //...

    }

    @Nested
    @DisplayName("saveOrUpdate methods")
    class SaveOrUpdateMethods { 
         //...

    }
}

Sample renderings in the IDE

Child methods of Nesteds are folded by default :

In case or test failure or on demand you can unfold child methods of Nesteds :