How to use XMLReporter class of org.testng.reporters package

The problem here is an unfortunate constellation between TestNG, JUnit and Mockto. To fix your issue, you just need to add a dependency to JUnit 4.0 or greater (the most recent version is currently 4.12):

<dependency>
  <groupId>junit</groupId>
  <artifactId>junit</artifactId>
  <version>4.12</version>
</dependency>

Here are the details:

TestNG, which is apparently your testing framework, declares a dependency to the quite old JUnit version 3.8.1. Mockito does not declare a dependency to JUnit at all but it uses some JUnit classes that were introduced in JUnit 4.0 (!).

Edit:

The method Mockito#timeout() in your example creates a Timeout instance which in turn creates an instance of VerificationOverTimeImpl. The method VerificationOverTimeImpl#verify() handles an error of type ArgumentsAreDifferent which is a subclass of org.junit.ComparisonFailure.

From JUnit version 3.8.1 to 4.x the class hierarchy of ComparisonFailure changed to having AssertionError instead of Error as base class. The VerifiyError is caused because VerificationOverTimeImpl#handleVerifyException() requires an AssertionError but would be invoked with an Error when JUnit 3.8.1 is used.

You can use BeanShell script in your testng.xml file http://testng.org/doc/documentation-main.html#beanshell and read system variables in it, something like:

<test name="Test">
    <method-selectors>
        <method-selector>
            <script language="beanshell">
                <![CDATA[
                    String myTestGroup = System.getProperty("chooseCase");
                    groups.containsKey(myTestGroup);
                ]]>
            </script>
        </method-selector>
    </method-selectors>

    <classes>
        <class name="AppTest"/>
        <class name="AppTest2"/>
    </classes>
</test>

Ok, first things first: I never used Mockito once in a life. But I was curious, so I spent several hours to figure this out and I guess there is much to improve.

So to get this working, we need:

1. The aforementioned (by @jewelsea) JUnit Threading Rule.
2. A custom MockMaker implementation, wrapping the default CglibMockMaker.
3. Wire the things together.

So 1+2 is this:

public class JavaFXMockMaker implements MockMaker {

    private final MockMaker wrapped = new CglibMockMaker();
    private boolean jfxIsSetup;

    private void doOnJavaFXThread(Runnable pRun) throws RuntimeException {
        if (!jfxIsSetup) {
            setupJavaFX();
            jfxIsSetup = true;
        }
        final CountDownLatch countDownLatch = new CountDownLatch(1);
        Platform.runLater(() -> {
            pRun.run();
            countDownLatch.countDown();
        });

        try {
            countDownLatch.await();
        } catch (InterruptedException e) {
            throw new RuntimeException(e);
        }
    }

    protected void setupJavaFX() throws RuntimeException {
        final CountDownLatch latch = new CountDownLatch(1);
        SwingUtilities.invokeLater(() -> {
            new JFXPanel(); // initializes JavaFX environment
            latch.countDown();
        });

        try {
            latch.await();
        } catch (InterruptedException e) {
            throw new RuntimeException(e);
        }
    }

    @Override
    public <T> T createMock(MockCreationSettings<T> settings, MockHandler handler) {
        AtomicReference<T> result = new AtomicReference<>();
        Runnable run = () -> result.set(wrapped.createMock(settings, handler));
        doOnJavaFXThread(run);
        return result.get();
    }

    @Override
    public MockHandler getHandler(Object mock) {
        AtomicReference<MockHandler> result = new AtomicReference<>();
        Runnable run = () -> result.set(wrapped.getHandler(mock));
        doOnJavaFXThread(run);
        return result.get();
    }

    @Override
    public void resetMock(Object mock, MockHandler newHandler, @SuppressWarnings("rawtypes") MockCreationSettings settings) {
        Runnable run = () -> wrapped.resetMock(mock, newHandler, settings); 
        doOnJavaFXThread(run);
    }

}

Number 3 is just following the manual:

1. Copy the fully qualified class name of our MockMaker, eg. org.awesome.mockito.JavaFXMockMaker.
2. Create a file "mockito-extensions/org.mockito.plugins.MockMaker". The content of this file is exactly a one line with the qualified name.

Happy testing & kudos to Andy Till for his threading rule.

Warning: This implementation kind of hard-codes the MockMaker to use the CglibMockMaker which might not be be what you want in every case (see the JavaDocs).

By convention, the test classes must also end with "Test". If they don't, you should add

<includes>
    <include>**/*.java</include>
</includes>

to your surefire config (or whatever includes work for you and your file naming scheme).

There are three wrong things I can spot based on your question, while they might not provide a complete answer I hope they'll will be a step in a right direction:

1. src/main/java and src/test/java is not a package as you write. Its a folder layout supported by maven, basically , if you have a code in com.myorg.somepackage.Arithmetics Make sure you have the following layout at the level of actual physical folders:

src
 main
  java
    com
     myorg
       somepackage
         Arithmetics.java
 test
  java
    com
     myorg
       somepackage
         ArithmeticsTest.java

For both java files the package must be com.myorg.somepackage:

package com.myorg.somepackage;

public class Arithmetics {}

And

package com.myorg.somepackage;

public class ArithmeticsTest {}

I know it might be a wrong naming that you've used in the question, but still I have to state it because it might cause an actual issue

2. The second issue is that for some reason you seem to configure your surefire plugin to use test ng which is an alternative to junit. It can happen because testng is placed as a dependency - I can only speculate because you don't really show the full surefire plugin configure and do not provide a full list of dependencies, but you've got and idea I believe :)

This is wrong because you use the junit5 dependencies as well as the imports that correspond to the junit 5.

3. The dependencies on the junit 5 are completely messy:

You seem to have two dependencies on just the same with the different scope, its really a wrong thing to do. Make sure you use only the test scope and have all the relevant dependencies. Read here for instruction of how to configure the surefire plugin

In addition, for the sake of completeness of the answer, check that you use the recent version of surefire plugin, if its too old the chances are that it won't be able to run jupiter engine (junit 5)

You should remove RestAssured.baseURI = "localhost:8080"; as baseURI is by default localhost and if you want set port you should use RestAssured.port=8080;

It is possible but it is very a tricky hack that I don't recommend.

public class DynamicTimeOutSample {

  private final long timeout;
  private final long waitTime;

  @DataProvider
  public static Object[][] dp() {
    return new Object[][]{
        new Object[]{ 1_000, 2_000 },
        new Object[]{ 3_000, 6_000 },
    };
  }

  @Factory(dataProvider = "dp")
  public DynamicTimeOutSample(long timeout, long waitTime) {
    this.timeout = timeout;
    this.waitTime = waitTime;
  }

  @BeforeMethod
  public void setUp(ITestContext context) {
    ITestNGMethod currentTestNGMethod = null;
    for (ITestNGMethod testNGMethod : context.getAllTestMethods()) {
      if (testNGMethod.getInstance() == this) {
        currentTestNGMethod = testNGMethod;
        break;
      }
    }
    currentTestNGMethod.setTimeOut(timeout);
  }

  @Test
  public void test() throws InterruptedException {
    Thread.sleep(waitTime);
  }
}

The twice test cases of the example will fail as expected.

I created the feature request, but don't expect to have it soon: https://github.com/cbeust/testng/issues/1061

If you have no concrete implementations of the class and the methods aren't static whats the point of testing them? If you have a concrete class then you'll be testing those methods as part of the concrete class's public API.

I know what you are thinking "I don't want to test these methods over and over thats the reason I created the abstract class", but my counter argument to that is that the point of unit tests is to allow developers to make changes, run the tests, and analyze the results. Part of those changes could include overriding your abstract class's methods, both protected and public, which could result in fundamental behavioral changes. Depending on the nature of those changes it could affect how your application runs in unexpected, possibly negative ways. If you have a good unit testing suite problems arising from these types changes should be apparent at development time.

the issue may be that you are putting cl.class in your Class array. cl is a class object. Try

testng.setTestClasses(new Class[] { cl });

You can read more about the Class class here http://docs.oracle.com/javase/tutorial/reflect/class/classNew.html