How to use run method of org.junit.runner.JUnitCore class

The easiest way for using an in memory database that is fully compatible to MySQL and can be used within JUnit test cases is imho MariaDB4j. you just need a Gradle (/Maven) dependency (http://search.maven.org/#search%7Cga%7C1%7Ca%3A%22mariaDB4j%22) and a few lines of code to start:

DB database = DB.newEmbeddedDB(3306);
database.start();
Connection connection = DriverManager.getConnection("jdbc:mysql://localhost/test", "root", "");

a startup script can be included via

database.source("path/to/resource.sql");

More information on GitHub readme: https://github.com/vorburger/MariaDB4j

EDIT: I have a add some hints to this answer: The MariaDB4j seems to add files in the systems temporary folder. So it will work in an embedded way which means there is no need to install anything and you can just use the dependency via your desired build tool. But it's not a true in-memory-only solution and therefore we cannot speak of unit tests anymore because unit tests mustn't rely on files or databases

Using the Maven Surefire plugin and JUnit 5 together requires some tweaking ...

From the docs:

The JUnit team has developed a very basic provider for Maven Surefire that lets you run JUnit 4 and JUnit Jupiter tests via mvn test. The pom.xml file in the junit5-maven-consumer project demonstrates how to use it and can serve as a starting point.

Due to a memory leak in Surefire 2.20, the junit-platform-surefire-provider currently only works with Surefire 2.19.1.

...
<build>
    <plugins>
        ...
        <plugin>
            <artifactId>maven-surefire-plugin</artifactId>
            <version>2.19</version>
            <dependencies>
                <dependency>
                    <groupId>org.junit.platform</groupId>
                    <artifactId>junit-platform-surefire-provider</artifactId>
                    <version>1.0.0</version>
                </dependency>
            </dependencies>
        </plugin>
    </plugins>
</build>
...

Ok, you've declared junit dependency for test classes only (those that are in src/test/java but you're trying to use it in main classes (those that are in src/main/java).

Either do not use it in main classes, or remove <scope>test</scope>.

You have to use a formatter element inside the junit task. The formatter will create a report file by default, but you can force it to print the results on screen. You can use two formatters: one for output to screen, another for output to file.

Note that you no longer need the attributes printsummary="yes" and showoutput="true" in the junit task. The formatter is taking care of output now.

<project name = "TestPrj" default="unittest" basedir = ".">

<target name="unittest" >
    <junit>
        <classpath>
            <pathelement location="./junit-4.8.2.jar"/>
            <pathelement location="./ant-junit4.jar"/>
        </classpath>
        <formatter type="plain" usefile="false" /> <!-- to screen -->
        <formatter type="plain" /> <!-- to file -->
        <test name = "com.mytest.unittest.SimpleTest" todir="."/>
    </junit>
</target>

</project>

Read the junit page in the ant manual for more information.

This problem happens because Android Platform (android.jar) already contains JUnit classes. IDEA test runner loads these classes and sees that they are from the old JUnit, while you are trying to use annotated tests which is a feature of the new JUnit, therefore you get the error from the test runner.

The solution is simple, open the Project Structure | Modules | Dependencies, and move the junit-4.7.jar up, so that it comes before Android 1.6 Platform in the classpath. Now the test runner will be happy as it loads the new JUnit version.

In JUnit, Parameterized tests use a special runner that ensure that the test is instantiated multiple times, so each test method is called multiple times. Mockito is a tool for writing specific unit tests, so there is no built-in ability to run the same test multiple times with different Mockito expectations.

If you want your test conditions to change, your best bet is to do one of the following:

• Parameterize your test using JUnit, with a parameter for the mock inputs you want;
• Run a loop of different parameters in your test, which unfortunately avoids the "test one thing per method" philosophy
• Extract a method that actually performs the test, and create a new @Test method for each mock you want.

Note that there's no prohibition on using mock objects as @Parameterized test parameters. If you're looking to parameterize based on mocks, you can do that, possibly creating the mock and setting the expectations in a static method on the test.

Note about runners: This Parameterized test runner conflicts with Mockito's MockitoJUnitRunner: Each test class can only have one runner. You'll want to switch to @Before and @After methods or a Mockito JUnit4 rule for your setup, if you use them both.

As an example, compressed from a different answer that explains more about Parameterized runners versus JUnit rules and lifting from the JUnit4 Parameterized Test doc page and MockitoRule doc page:

@RunWith(Parameterized.class)
public class YourComponentTest {
    @Rule public MockitoRule rule = MockitoJUnit.rule();
    @Mock YourDep mockYourDep;

    @Parameters public static Collection<Object[]> data() { /* Return the values */ }

    public YourComponentTest(Parameter parameter) { /* Save the parameter to a field */ }

    @Test public void test() { /* Use the field value in assertions */ }
}

Epsilon is your "fuzz factor," since doubles may not be exactly equal. Epsilon lets you describe how close they have to be.

If you were expecting 3.14159 but would take anywhere from 3.14059 to 3.14259 (that is, within 0.001), then you should write something like

double myPi = 22.0d / 7.0d; //Don't use this in real life!
assertEquals(3.14159, myPi, 0.001);

(By the way, 22/7 comes out to 3.1428+, and would fail the assertion. This is a good thing.)

Solution inspired by Guillaume :

In Run->Edit configuration->Defaults->JUnit->Working directory set the value $MODULE_DIR$ and Intellij will set the relative path in all junits just like Maven.

JUnit 5 provides a way out of the box.

JUnit 5 = JUnit Platform + JUnit Jupiter + JUnit Vintage

Each one is a distinct project and using all of them allows to compile and execute JUnit 4 and JUnit 5 tests in a same project.

JUnit Jupiter is the combination of the new programming model and extension model for writing tests and extensions in JUnit 5.

JUnit Vintage provides a TestEngine for running JUnit 3 and JUnit 4 based tests on the platform.

The JUnit Platform serves as a foundation for launching testing frameworks on the JVM

Update : from Maven Surefire 2.22.0

From the JUnit 5 documentation :

Starting with version 2.22.0, Maven Surefire provides native support for executing tests on the JUnit Platform.

So the configuration is much simpler.
Note that the junit-4 api dependency is optional as the engine dependencies that are now required already pull a default api version (it is the case for both junit 4 and 5).

Here is a sample pom.xml.

<project xmlns="http://maven.apache.org/POM/4.0.0"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>
    <groupId>david</groupId>
    <artifactId>jupiter-4-and-5-same-build</artifactId>
    <version>0.0.1-SNAPSHOT</version>

    <properties>
        <maven.compiler.source>1.8</maven.compiler.source>
        <maven.compiler.target>1.8</maven.compiler.target>
        <junit-jupiter.version>5.1.0</junit-jupiter.version>
        <!-- optional : if we want to use a junit4 specific version -->
        <junit.version>4.12</junit.version>
    </properties>
    <dependencies>
        <!--JUnit Jupiter Engine to depend on the JUnit5 engine and JUnit 5 API -->
        <dependency>
            <groupId>org.junit.jupiter</groupId>
            <artifactId>junit-jupiter-engine</artifactId>
            <version>${junit-jupiter.version}</version>
            <scope>test</scope>
        </dependency>
        <!--JUnit Jupiter Engine to depend on the JUnit4 engine and JUnit 4 API  -->
        <dependency>
            <groupId>org.junit.vintage</groupId>
            <artifactId>junit-vintage-engine</artifactId>
            <version>${junit-jupiter.version}</version>
        </dependency>
        <!-- Optional : override the JUnit 4 API version provided by junit-vintage-engine -->
        <dependency>
            <groupId>junit</groupId>
            <artifactId>junit</artifactId>
            <version>${junit.version}</version>
            <scope>test</scope>
        </dependency>
    </dependencies>

    <build>
        <plugins>
            <plugin>
                <artifactId>maven-surefire-plugin</artifactId>
                <version>2.22.0</version>
            </plugin>
        </plugins>
    </build>

</project>

On my GitHub space I added a working sample maven project that you can browse/clone. URL: https://github.com/ebundy/junit4-and-5-minimal-maven-project

Old way : for Maven Surefire below 2.22.0

Here is the minimal configuration to use with Maven to configure the project to compile and run both JUnit4 and JUnit5 tests :

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>
    <groupId>mygroup</groupId>
    <artifactId>minimal-conf-junit4-5</artifactId>
    <version>0.0.1-SNAPSHOT</version>

    <properties>
        <!-- JUnit 5 depends on JDK 1.8 -->
        <maven.compiler.source>1.8</maven.compiler.source>
        <maven.compiler.target>1.8</maven.compiler.target>
        <!--  JUnit dependency versions -->
        <junit.version>4.12</junit.version>
        <junit-vintage-engine>4.12.1</junit-vintage-engine>
        <junit-jupiter.version>5.0.1</junit-jupiter.version>
        <junit-platform.version>1.0.1</junit-platform.version>
    </properties>

    <dependencies>
        <!--JUnit Jupiter API to write and compile tests with JUnit5 -->
        <dependency>
            <groupId>org.junit.jupiter</groupId>
            <artifactId>junit-jupiter-api</artifactId>
            <version>${junit-jupiter.version}</version>
            <scope>test</scope>
        </dependency>
        <!-- JUnit 4 to make legacy JUnit 4 tests compile -->
        <dependency>
            <groupId>junit</groupId>
            <artifactId>junit</artifactId>
            <version>${junit.version}</version>
            <scope>test</scope>
        </dependency>
    </dependencies>

    <build>
        <plugins>
            <plugin>
                <artifactId>maven-surefire-plugin</artifactId>
                <version>2.19.1</version> <!-- matters until now-->
                <dependencies>
                    <!-- to let surefire to run JUnit 4 but also JUnit 5 tests -->
                    <dependency>
                        <groupId>org.junit.platform</groupId>
                        <artifactId>junit-platform-surefire-provider</artifactId>
                        <version>${junit-platform.version}</version>
                    </dependency>
                    <!-- JUnit vintage engine to run JUnit 3 or JUnit 4 tests -->
                    <dependency>
                        <groupId>org.junit.vintage</groupId>
                        <artifactId>junit-vintage-engine</artifactId>
                        <version>${junit-vintage-engine}</version>
                    </dependency>
                    <!-- JUnit 5 engine to run JUnit 5 tests -->
                    <dependency>
                        <groupId>org.junit.jupiter</groupId>
                        <artifactId>junit-jupiter-engine</artifactId>
                        <version>${junit-jupiter.version}</version>
                    </dependency>
                </dependencies>
            </plugin>
        </plugins>
    </build>
</project>

Now mvn test compiles and runs both JUnit 4 and JUnit 5 tests.

Note 1 : the junit-vintage-engine (4.12.1) and the junit (4.12) dependencies don't specify the same exact version.
This is not an issue at all as :

• their release are not related between them

• junit-vintage-engine is designed to run any JUnit 3 or 4 tests.
  

Note 2 : maven-surefire-plugin with the 2.19.1 version matters whatever you want to compile JUnit 5 test classes or both JUnit 4 and JUnit 5 test classes.
Next version of the plugin causes indeed some exceptions during JUnit 5 tests execution but the 2.22.0 that at last solves the issue (see the first part of the answer : "Update : from Maven Surefire 2.22.0").