How to use XMLUtils class of org.testng.reporters package

Although Parameterized extends Suite, it behaves totally different - in disrespect of the Liskov substitution principle. This is because normally the constructor Suite(Class<?>, RunnerBuilder) processes the @SuiteClasses annotation. But Parameterized(Class<?>) replaces this behaviour with a processing of @Parameters.

If you want to combine the behaviour of Suite and Parameterized you have to look outside of JUnit 4. E.g. you could implement your own Runner like Adam Hawkes already mentioned in another post here.

I did the same by myself and cobbled a library together that provides you with a ParameterizedSuite Runner: https://github.com/PeterWippermann/parameterized-suite

A parameterized test suite looks like this:

@RunWith(ParameterizedSuite.class)
@SuiteClasses({OneTest.class, TwoTest.class})
public class MyParameterizedTestSuite {
    @Parameters(name = "Parameters are {0} and {1}")
    public static Object[] params() {
        return new Object[][] {{'A',1}, {'B',2}, {'C',3}};
    }

Just to let you know that i have found a solution as provided in this How to programmatically call a Maven-task

as i mentioned I did use the main class. So i did the below

-Added the maven-invoker dependency in pom.xml

<dependency>
    <groupId>org.apache.maven.shared</groupId>
    <artifactId>maven-invoker</artifactId>
  <version>2.1.1</version>
</dependency>

• Updated my main class class as below using this link- http://maven.apache.org/shared/maven-invoker/usage.html

  public class MainOne {
      public static void main(String[] args) {
          InvocationRequest request = new DefaultInvocationRequest();
          request.setPomFile( new File( "pom.xml" ));
          request.setGoals( Collections.singletonList( "install" ));
          Invoker invoker = new DefaultInvoker();
          invoker.setMavenHome(new File(System.getenv("MAVEN_HOME")));
          try {
              invoker.execute( request );
          } catch (MavenInvocationException e) {
              e.printStackTrace();
          } 
      } 
  }
  

• Then from command prompt ran the command mvn clean package shade:shade

• This created the jar in the target folder.
• Copied the jar into Project folder and ran the command java -jar myproj-0.0.1-SNAPSHOT.jar, my tests starting running perfectly!!.

the issue may be that you are putting cl.class in your Class array. cl is a class object. Try

testng.setTestClasses(new Class[] { cl });

You can read more about the Class class here http://docs.oracle.com/javase/tutorial/reflect/class/classNew.html

I've used both, but I have to agree with Justin Standard that you shouldn't really consider rewriting your existing tests to any new format. Regardless of the decision, it is pretty trivial to run both. TestNG strives to be much more configurable than JUnit, but in the end they both work equally well.

TestNG has a neat feature where you can mark tests as a particular group, and then easily run all tests of a specific group, or exclude tests of a particular group. Thus you can mark tests that run slowly as in the "slow" group and then ignore them when you want quick results. A suggestion from their documentation is to mark some subset as "checkin" tests which should be run whenever you check new files in. I never saw such a feature in JUnit, but then again, if you don't have it, you don't REALLY miss it.

For all its claims of high configuration, I did run into a corner case the a couple weeks ago where I couldn't do what I wanted to do... I wish I could remember what it is, but I wanted to bring it up so you know that it's not perfect.

The biggest advantage TestNG has is annotations... which JUnit added in version 4 anyways.

@PostConstruct would not be invoked for a test class instance used in an Arquillian test. While Arquillian performs non-contextual CDI injection into the injection points of a test class instance, it is not responsible for constructing the instance itself (JUnit or TestNG does this), and neither is construction of the test class instance managed by the CDI container or any other service container (this explains why @PostConstruct is ignored).

You're best bet therefore is to use @Before. You do raise a good point though, and it might be worth investigating whether the JUnit and TestNG runners provide hooks so that CDI or other DI providers can manage or hook into the test instance lifecycle.

There is no WebApplicationContext only an ApplicationContext unless you add the @WebAppConfiguration to your test class.

@ContextConfiguration(locations = { "classpath:/test/BeanConfig.xml" })
@RunWith(SpringJUnit4ClassRunner.class)
@WebAppConfiguration
public class CandidateControllerTest { ... }

Instead of the @RunWith annotation you can also extend one of springs convenience classes.

@ContextConfiguration(locations = { "classpath:/test/BeanConfig.xml" })
@WebAppConfiguration
public class CandidateControllerTest extends AbstractJUnit4SpringContextTests { ... }

Links

1. WebAppConfiguration javadoc
2. Reference Guide

Recent Gradle versions automatically create and wire configurations 'fooCompile' and 'fooRuntime' for each source set 'foo'.

If you are still using an older version, you can declare your own configuration and add it to the source set's compileClasspath or runtimeClasspath. Something like:

configurations {
    serviceTestCompile
}

sourceSets {
    serviceTest {
        compileClasspath = configurations.serviceTestCompile
    }
}

dependencies {
    serviceTestCompile "org.testng:testng:5.8"
}

The problem is you have the same method name between A and B.

When TestNG is looking for configuration methods, it won't find the annotation on the setUp() method of B. In every case, you'll have to annotate your method in B because TestNG is not able to find configuration methods without annotation.

You can keep the work around you use, or you can rename one configuration method. setUp is no more mandatory.

But we can suppose it is an issue which can be solved by TestNG too. So, you should open a ticket on the project: http://github.com/cbeust/testng

Try using this code:

ListenerApadter:

public class MyTestListenerAdapter extends TestListenerAdapter {
    @Override
    public void onTestFailure(ITestResult result) {
        if (result.getMethod().getRetryAnalyzer() != null) {
            MyRetryAnalyzer retryAnalyzer = (MyRetryAnalyzer)result.getMethod().getRetryAnalyzer();

            if(retryAnalyzer.isRetryAvailable()) {
                result.setStatus(ITestResult.SKIP);
            } else {
                result.setStatus(ITestResult.FAILURE);
            }
            Reporter.setCurrentTestResult(result);
        }
    }

   @Overrride
   public void onFinish(ITestContext context) {
     Iterator<ITestResult> failedTestCases =context.getFailedTests().getAllResults().iterator();
    while (failedTestCases.hasNext()) {
        System.out.println("failedTestCases");
        ITestResult failedTestCase = failedTestCases.next();
        ITestNGMethod method = failedTestCase.getMethod();
        if (context.getFailedTests().getResults(method).size() > 1) {
            System.out.println("failed test case remove as dup:" + failedTestCase.getTestClass().toString());
            failedTestCases.remove();
        } else {

            if (context.getPassedTests().getResults(method).size() > 0) {
                System.out.println("failed test case remove as pass retry:" + failedTestCase.getTestClass().toString());
                failedTestCases.remove();
            }
        }
    }
   }
}

RetryAnalizer:

public class MyRetryAnalyzer implements IRetryAnalyzer {
    private static int MAX_RETRY_COUNT = 3;

    AtomicInteger count = new AtomicInteger(MAX_RETRY_COUNT);

    public boolean isRetryAvailable() {
        return (count.intValue() > 0);
    }

    @Override
    public boolean retry(ITestResult result) {
        boolean retry = false;
        if (isRetryAvailable()) {
            System.out.println("Going to retry test case: " + result.getMethod() + ", " + (MAX_RETRY_COUNT - count.intValue() + 1) + " out of " + MAX_RETRY_COUNT);
            retry = true;
            count.decrementAndGet();
        }
        return retry;
    }
}

POM.xml -> Surefire Configuration:

This is where you should configure "overwrite" surefire listener wich has their own counters.

<plugin>
  <groupId>org.apache.maven.plugins</groupId>
  <artifactId>maven-surefire-plugin</artifactId>
  <version>2.18.1</version>
  <configuration>
    <suiteXmlFiles><suiteXmlFile>${basedir}/testng.xml</suiteXmlFile></suiteXmlFiles>
 <properties> 
   <property>
    <name>listener</name>
    <value>Utils.MyTestListenerAdapter,Utils.MyRetryAnalizer</value>
   </property>
 </properties>