How to use countTestCases method of junit.extensions.RepeatedTest class

Are you fixed to JUnit? TestNG provides good multi thread testing out of the box and it's compatible with JUnit tests (you need to make a few changes). For example you could run a test like this:

@Test(threadPoolSize = 3, invocationCount = 9,  timeOut = 10000)
public void doSomething() {
...
}

This would mean that the doSomething() method will be invoked 9 times by 3 different threads.

I highly recommend TestNG.

For the mocks initialization, using the runner or the MockitoAnnotations.initMocks are strictly equivalent solutions. From the javadoc of the MockitoJUnitRunner :

JUnit 4.5 runner initializes mocks annotated with Mock, so that explicit usage of MockitoAnnotations.initMocks(Object) is not necessary. Mocks are initialized before each test method.

The first solution (with the MockitoAnnotations.initMocks) could be used when you have already configured a specific runner (SpringJUnit4ClassRunner for example) on your test case.

The second solution (with the MockitoJUnitRunner) is the more classic and my favorite. The code is simpler. Using a runner provides the great advantage of automatic validation of framework usage (described by @David Wallace in this answer).

Both solutions allows to share the mocks (and spies) between the test methods. Coupled with the @InjectMocks, they allow to write unit tests very quickly. The boilerplate mocking code is reduced, the tests are easier to read. For example:

@RunWith(MockitoJUnitRunner.class)
public class ArticleManagerTest {

    @Mock private ArticleCalculator calculator;
    @Mock(name = "database") private ArticleDatabase dbMock;
    @Spy private UserProvider userProvider = new ConsumerUserProvider();

    @InjectMocks private ArticleManager manager;

    @Test public void shouldDoSomething() {
        manager.initiateArticle();
        verify(database).addListener(any(ArticleListener.class));
    }

    @Test public void shouldDoSomethingElse() {
        manager.finishArticle();
        verify(database).removeListener(any(ArticleListener.class));
    }
}

Pros: The code is minimal

Cons: Black magic. IMO it is mainly due to the @InjectMocks annotation. With this annotation "you loose the pain of code" (see the great comments of @Brice)

The third solution is to create your mock on each test method. It allow as explained by @mlk in its answer to have "self contained test".

public class ArticleManagerTest {

    @Test public void shouldDoSomething() {
        // given
        ArticleCalculator calculator = mock(ArticleCalculator.class);
        ArticleDatabase database = mock(ArticleDatabase.class);
        UserProvider userProvider = spy(new ConsumerUserProvider());
        ArticleManager manager = new ArticleManager(calculator, 
                                                    userProvider, 
                                                    database);

        // when 
        manager.initiateArticle();

        // then 
        verify(database).addListener(any(ArticleListener.class));
    }

    @Test public void shouldDoSomethingElse() {
        // given
        ArticleCalculator calculator = mock(ArticleCalculator.class);
        ArticleDatabase database = mock(ArticleDatabase.class);
        UserProvider userProvider = spy(new ConsumerUserProvider());
        ArticleManager manager = new ArticleManager(calculator, 
                                                    userProvider, 
                                                    database);

        // when 
        manager.finishArticle();

        // then
        verify(database).removeListener(any(ArticleListener.class));
    }
}

Pros: You clearly demonstrate how your api works (BDD...)

Cons: there is more boilerplate code. (The mocks creation)

My recommandation is a compromise. Use the @Mock annotation with the @RunWith(MockitoJUnitRunner.class), but do not use the @InjectMocks :

@RunWith(MockitoJUnitRunner.class)
public class ArticleManagerTest {

    @Mock private ArticleCalculator calculator;
    @Mock private ArticleDatabase database;
    @Spy private UserProvider userProvider = new ConsumerUserProvider();

    @Test public void shouldDoSomething() {
        // given
        ArticleManager manager = new ArticleManager(calculator, 
                                                    userProvider, 
                                                    database);

        // when 
        manager.initiateArticle();

        // then 
        verify(database).addListener(any(ArticleListener.class));
    }

    @Test public void shouldDoSomethingElse() {
        // given
        ArticleManager manager = new ArticleManager(calculator, 
                                                    userProvider, 
                                                    database);

        // when 
        manager.finishArticle();

        // then 
        verify(database).removeListener(any(ArticleListener.class));
    }
}

Pros: You clearly demonstrate how your api works (How my ArticleManager is instantiated). No boilerplate code.

Cons: The test is not self contained, less pain of code

I had the same issue and it was fixed by putting the Model and Model_ class into the same package.

Am I correct in saying then, that at runtime there is no difference between an @Ignore test, a method with no annotation, and a commented out method?

An @Ignored method can be found via reflection. A method with no annotation can't (or, to be precise, it can't be identified with certainty as an ignored test method), and a commented out method does not even get into the bytecode.

Albeit I don't think there would be much practical value in finding @Ignored methods runtime, it may be useful to generate statistics / reports.

how much usefulness does the @Ignore tag really have

One thing I can think of is searchability. You can easily identify all @Ignore annotations in the source code, while unannotated or commented out tests are not so simple to find.

it might be more useful to fail the test so it's not overlooked?

If you want (and can) fix it right away, it is fine to have it fail. There are cases when you can't, but you still want to method to be around, precisely so that it does not get forgotten. Then @Ignore makes sense.

You are using mockito anyString() while calling the test method, it should be used only for verifying a mock object to ensure a certain method is called with any string parameter inside the test, but not to invoke the test itself. For your test use empty string "" instead to anyString().

Try it this way:

@Rule public ExpectedException thrown = ExpectedException.none();

@Test public void testMethod() throws Throwable {
    final String error = "error message";
    Throwable expectedCause = new IllegalStateException(error);
    thrown.expectCause(IsEqual.equalTo(expectedCause));
    throw new RuntimeException(expectedCause);
}

Consider not to check against the cause by equals but by IsInstanceOf and / or comapring the exception message if necessary. Comparing the cause by equals check the stacktrace as well, which may be more than you would like to test / check. Like this for example:

@Rule public ExpectedException thrown = ExpectedException.none();

@Test public void testMethod() throws Throwable {
    final String error = "error message";
    thrown.expectCause(IsInstanceOf.<Throwable>instanceOf(IllegalStateException.class));
    thrown.expectMessage(error);
    throw new RuntimeException(new IllegalStateException(error));
}

I had the same problem (JUnit tests failed in Maven Surefire but passed in Eclipse) and managed to solve it by setting forkMode to always in the maven surefire configuration in pom.xml:

<plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-surefire-plugin</artifactId>
    <version>2.12</version>
    <configuration>
        <forkMode>always</forkMode>
    </configuration>
</plugin>

Surefire parameters: http://maven.apache.org/plugins/maven-surefire-plugin/test-mojo.html

Edit (January 2014):

As Peter Perháč pointed out, the forkMode parameter is deprecated since Surefire 2.14. Beginning from Surefire 2.14 use this instead:

<plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-surefire-plugin</artifactId>
    <version>2.16</version>
    <configuration>
        <reuseForks>false</reuseForks>
        <forkCount>1</forkCount>
    </configuration>
</plugin>

For more information see Fork Options and Parallel Test Execution

The best way (IMO) of making your code testable is to extract the dependency of "what's the current time" into its own interface, with an implementation which uses the current system time (used normally) and an implementation which lets you set the time, advance it as you want etc.

I've used this approach in various situations, and it's worked well. It's easy to set up - just create an interface (e.g. Clock) which has a single method to give you the current instant in whatever format you want (e.g. using Joda Time, or possibly a Date).

This appears to occur because only the source code is compiling when you use mvn clean compile (I'm using maven 3.1.0 so I'm not sure if it always behaved like this).

If you run mvn test, the test code will compile as well, but then it runs the tests (which may not be immediately desirable if you're trying to run them through Eclipse.) The way around this is to add test-compile to your Maven command sequence whenever you do a mvn clean. For example, you would run mvn clean compile test-compile.