How to use onExecutionStart method of org.testng.Interface IExecutionListener class

What you need to use is @BeforeGroups rather than @BeforeMethod. BeforeMethod ensures that setup1 and setup2 both run before each method. BeforeGroups would ensure that it would run only before specific group starts executing.

According to Maven Surefire Plugin - Running a Single Test :

Running a Set of Methods in a Single Test Class

As of Surefire 2.7.3, you can also run only a subset of the tests in a test class.

NOTE : This feature is supported only for Junit 4.x and TestNG. Use syntax e.g. "foo/MyTest.java", "/MyTest.java", "MyTest" for "test" parameter (see includes/excludes).**

You should use the following syntax:

mvn -Dtest=TestCircle#mytest test

So I think your command should be:

mvn -B clean install -T2C -pl sub-tests -Dtest=!com.xx.xx.test.xx.xx#canMakeCall*

It will ignore all tests methods starting with canMakeCall of test com.xx.xx.test.xx.xx.

Another option is to use a suite-of-suites. I'll preface this by saying it's been a while since I've worked with this setup, but hopefully you'll find enough detail here to at least get started.

First, you'd define two (or more) suite XML files:

<suite name="Suite1">
    <test name="test1">
        <classes>
            <class name="fully.qualified.ClassName" />
        </classes>
        <methods>
            <include name="method1" />
        </methods>
    </test>
</suite>

and...

<suite name="Suite2">
    <test name="test2">
        <classes>
            <class name="fully.qualified.ClassName" />
        </classes>
        <methods>
            <include name="method2" />
        </methods>
    </test>
</suite>

Then, you'd define a suite-of-suites XML file:

<suite name="suite of suites">
    <suite-files>
        <suite-file path="Suite1.xml" />
        <suite-file path="Suite2.xml" />
    </suite-files>
</suite>

Do note, that the suite-file path value is the relative path from the current suite-of-suites XML file to the XML file you're calling. If they're in the same directory, simply the file name will suffice.

Additionally, both XML types do support the <parameter> tag. The standard suite XML will read both at <suite> and in the <test> levels, and I've found that <parameter> tags at the <suite> level on the suite-of-suites XML file will work as well.

Finally, on execution, you'd need only pass in the suite-of-suites XML file as your suite file argument.

EDIT: Here's how I was able to make my two suites run in parallel. The trick was setting up my main() method properly.

public class TestRunner
{
    public static void main(String[] args)
    {
        TestNG testng = new TestNG();
        TestListenerAdapter adapter = new TestListenerAdapter();
        List<String> suites = new ArrayList<String>();

        testng.addListener(adapter);
        suites.add(args[0]);
        testng.setTestSuites(suites);
        testng.setParallel("parallel");
        testng.setSuiteThreadPoolSize(5);
        testng.setOutputDirectory("path to output");
        testng.run();
    }
}

Then, on the command line:

java -jar ParallelSuiteDemo.jar SuiteOfSuites.xml

Note, my jar and all xml files were in the same directory with this configuration. The command line args and <suite-file> entries would need to be configured properly if you wish you use a directory structure.

This yielded my two suite.xml files running in parallel on a Selenium Grid.

There may be better ways of doing this, to be honest. This is just what worked for me when I was trying to do something similar.

Like you said, just use this flag to turn off the default reports. The exact name depends on whether you're using the command line, ant or maven. Look this string up in the documentation that is relevant to you.

Mocking ==> https://stackoverflow.com/questions/37359/what-c-mocking-framework-to-use

Continuous Integration ==> CruiseControl.NET

JSON Serialization ==> DataContractJsonSerializer

GUI == > WPF (different than Swing but nice)

Web Server hooks ==> Mod_Mono

The problem is you have the same method name between A and B.

When TestNG is looking for configuration methods, it won't find the annotation on the setUp() method of B. In every case, you'll have to annotate your method in B because TestNG is not able to find configuration methods without annotation.

You can keep the work around you use, or you can rename one configuration method. setUp is no more mandatory.

But we can suppose it is an issue which can be solved by TestNG too. So, you should open a ticket on the project: http://github.com/cbeust/testng

JUnit 4.8 supports grouping:

public interface SlowTests {}
public interface IntegrationTests extends SlowTests {}
public interface PerformanceTests extends SlowTests {}

And then...

public class AccountTest {

    @Test
    @Category(IntegrationTests.class)
    public void thisTestWillTakeSomeTime() {
        ...
    }

    @Test
    @Category(IntegrationTests.class)
    public void thisTestWillTakeEvenLonger() {
        ...
    }

    @Test
    public void thisOneIsRealFast() {
        ...
    }
}

And lastly,

@RunWith(Categories.class)
@ExcludeCategory(SlowTests.class)
@SuiteClasses( { AccountTest.class, ClientTest.class })
public class UnitTestSuite {}

Taken from here: https://community.oracle.com/blogs/johnsmart/2010/04/25/grouping-tests-using-junit-categories-0

Also, Arquillian itself supports grouping: https://github.com/weld/core/blob/master/tests-arquillian/src/test/java/org/jboss/weld/tests/Categories.java

Found the Solution

//Create a string list and iterate the Json array to fetch the required text with the same key 

List<String> responseList = new ArrayList<String>(jsonArray.length());

for (int i = 0; i < jsonArray.length(); i++) {
                            responseList.add((jsonArray.getJSONObject(i).getString("message")));

}

//Assert the list
org.junit.Assert.assertThat(responseList, hasItems("Rest Endpoint should not be empty", "Invalid URL"));

This took me a while to figure out, so I am answering my own question.

import org.junit.Before;
import org.junit.Test;
import org.junit.runner.RunWith;
import org.mockito.Mockito;
import org.powermock.api.mockito.PowerMockito;
import org.powermock.core.classloader.annotations.PrepareForTest;
import org.powermock.modules.junit4.PowerMockRunner;

@RunWith(PowerMockRunner.class)  // important
@PrepareForTest(MachineImpl.class)  // important: do not use Thread.class here
public class MachineImplTest {

    private MachineImpl classUnderTest;

    @Before
    public void beforeEachTest() {
        classUnderTest = new MachineImpl();
    }

    @Test
    public void sleep_Pass() {
        classUnderTest.sleep(0);
        classUnderTest.sleep(-100);
        classUnderTest.sleep(+100);
    }

    @Test
    public void sleep_Pass2() {
        // We do not want to mock all methods, only specific methods, such as Thread.sleep().
        // Use spy() instead of mockStatic().
        PowerMockito.spy(Thread.class);

        // These two lines are tightly bound.
        PowerMockito.doThrow(new InterruptedException()).when(Thread.class);
        Thread.sleep(Mockito.anyLong());

        classUnderTest.sleep(0);
        classUnderTest.sleep(-100);
        classUnderTest.sleep(+100);
    }
}

If you are using TestNG, try this:

import org.mockito.Mockito;
import org.powermock.api.mockito.PowerMockito;
import org.powermock.core.classloader.annotations.PrepareForTest;
import org.powermock.modules.testng.PowerMockTestCase;

@PrepareForTest(MachineImpl.class)  // important: do not use Thread.class here
public class MachineImplTest
extends PowerMockTestCase {
    ...
}

Read more about TestNG + Mockito + PowerMock here: https://stackoverflow.com/a/35815153/257299