How to use setReporter method of org.testng.TestNGAntTask class

You can use a combination of the IRetryAnalyzer, a Listener and a custom reporter to do what you are looking for.

IRetryAnalyzer:

public class RetryAnalyzer implements IRetryAnalyzer  { 
private int count = 0; 
// this number is actually twice the number
// of retry attempts will allow due to the retry
// method being called twice for each retry
private int maxCount = 6; 
protected Logger log;
private static Logger testbaseLog;

static {
    PropertyConfigurator.configure("test-config/log4j.properties");
    testbaseLog = Logger.getLogger("testbase.testng");
}

public RetryAnalyzer()
{
    testbaseLog.trace( " ModeledRetryAnalyzer constructor " + this.getClass().getName() );
    log = Logger.getLogger("transcript.test");
}

@Override 
public boolean retry(ITestResult result) { 
    testbaseLog.trace("running retry logic for  '" 
            + result.getName() 
            + "' on class " + this.getClass().getName() );
        if(count < maxCount) {                     
                count++;                                    
                return true; 
        } 
        return false; 
}
}

RetryListener:

public class RetryTestListener extends TestListenerAdapter  {
private int count = 0; 
private int maxCount = 3; 

@Override
public void onTestFailure(ITestResult result) {     
    Logger log = Logger.getLogger("transcript.test");
    Reporter.setCurrentTestResult(result);

    if(result.getMethod().getRetryAnalyzer().retry(result)) {    
        count++;
        result.setStatus(ITestResult.SKIP);
        log.warn("Error in " + result.getName() + " with status " 
                + result.getStatus()+ " Retrying " + count + " of 3 times");
        log.info("Setting test run attempt status to Skipped");                 
    } 
    else
    {
        count = 0;
        log.error("Retry limit exceeded for " + result.getName());
    }       

    Reporter.setCurrentTestResult(null);
}

@Override
public void onTestSuccess(ITestResult result)
{
    count = 0;
}

However, there appears to be a bug within TestNG that actually causes some of the test results to be reported as both skipped AND failed. To prevent this, I recommend that you override whatever Reporter you wish to use and include a method such as the one included below:

private IResultMap removeIncorrectlyFailedTests(ITestContext test)
{     
  List<ITestNGMethod> failsToRemove = new ArrayList<ITestNGMethod>();
  IResultMap returnValue = test.getFailedTests();

  for(ITestResult result : test.getFailedTests().getAllResults())
  {
    long failedResultTime = result.getEndMillis();          

    for(ITestResult resultToCheck : test.getSkippedTests().getAllResults())
    {
        if(failedResultTime == resultToCheck.getEndMillis())
        {
            failsToRemove.add(resultToCheck.getMethod());
            break;
        }
    }

    for(ITestResult resultToCheck : test.getPassedTests().getAllResults())
    {
        if(failedResultTime == resultToCheck.getEndMillis())
        {
            failsToRemove.add(resultToCheck.getMethod());
            break;
        }
    }           
  }

  for(ITestNGMethod method : failsToRemove)
  {
      returnValue.removeResult(method);
  }  

  return returnValue;
}

After all of this is done, you can add the reporter using using .addListener and specify the retryAnalyzer in the @Test annotation.

I think you just need to get rid of this line in both your TestNGClass2 and TestNGClass3

public static WebDriver driver;

You're already storing the driver in the base class TestNGClass1, so when you have that line in your other classes, you're basically hiding the one that is instantiated.

I'd also consider changing the base class access modifier to protected since you probably don't want classes that aren't children of that base class to access the driver.

I've used both, but I have to agree with Justin Standard that you shouldn't really consider rewriting your existing tests to any new format. Regardless of the decision, it is pretty trivial to run both. TestNG strives to be much more configurable than JUnit, but in the end they both work equally well.

TestNG has a neat feature where you can mark tests as a particular group, and then easily run all tests of a specific group, or exclude tests of a particular group. Thus you can mark tests that run slowly as in the "slow" group and then ignore them when you want quick results. A suggestion from their documentation is to mark some subset as "checkin" tests which should be run whenever you check new files in. I never saw such a feature in JUnit, but then again, if you don't have it, you don't REALLY miss it.

For all its claims of high configuration, I did run into a corner case the a couple weeks ago where I couldn't do what I wanted to do... I wish I could remember what it is, but I wanted to bring it up so you know that it's not perfect.

The biggest advantage TestNG has is annotations... which JUnit added in version 4 anyways.

This was a bug in Testng. solved in 6.9.5. Please upgrade.

You shouldn't see a NPE as a test failure. You should actually assert it is not null and provide an error message,

assertNotNull(someVal, "Some descriptive message saying why value should not be null");

Neither JSR-303 nor Hibernate Validator has any ready-made constraint that can validate each elements of Collection.

One possible solution to address this issue is to create a custom @ValidCollection constraint and corresponding validator implementation ValidCollectionValidator.

To validate each element of collection we need an instance of Validator inside ValidCollectionValidator; and to get such instance we need custom implementation of ConstraintValidatorFactory.

See if you like following solution...

Simply,

• copy-paste all these java classes (and import relavent classes);
• add validation-api, hibenate-validator, slf4j-log4j12, and testng jars on classpath;
• run the test-case.

ValidCollection

    public @interface ValidCollection {

    Class<?> elementType();

    /* Specify constraints when collection element type is NOT constrained 
     * validator.getConstraintsForClass(elementType).isBeanConstrained(); */
    Class<?>[] constraints() default {};

    boolean allViolationMessages() default true;

    String message() default "{ValidCollection.message}";

    Class<?>[] groups() default {};

    Class<? extends Payload>[] payload() default {};

}

ValidCollectionValidator

    public class ValidCollectionValidator implements ConstraintValidator<ValidCollection, Collection>, ValidatorContextAwareConstraintValidator {

    private static final Logger logger = LoggerFactory.getLogger(ValidCollectionValidator.class);

    private ValidatorContext validatorContext;

    private Class<?> elementType;
    private Class<?>[] constraints;
    private boolean allViolationMessages;

    @Override
    public void setValidatorContext(ValidatorContext validatorContext) {
        this.validatorContext = validatorContext;
    }

    @Override
    public void initialize(ValidCollection constraintAnnotation) {
        elementType = constraintAnnotation.elementType();
        constraints = constraintAnnotation.constraints();
        allViolationMessages = constraintAnnotation.allViolationMessages();
    }

    @Override
    public boolean isValid(Collection collection, ConstraintValidatorContext context) {
        boolean valid = true;

        if(collection == null) {
            //null collection cannot be validated
            return false;
        }

        Validator validator = validatorContext.getValidator();

        boolean beanConstrained = validator.getConstraintsForClass(elementType).isBeanConstrained();

        for(Object element : collection) {
            Set<ConstraintViolation<?>> violations = new HashSet<ConstraintViolation<?>> ();

            if(beanConstrained) {
                boolean hasValidCollectionConstraint = hasValidCollectionConstraint(elementType);
                if(hasValidCollectionConstraint) {
                    // elementType has @ValidCollection constraint
                    violations.addAll(validator.validate(element));
                } else {
                    violations.addAll(validator.validate(element));
                }
            } else {
                for(Class<?> constraint : constraints) {
                    String propertyName = constraint.getSimpleName();
                    propertyName = Introspector.decapitalize(propertyName);
                    violations.addAll(validator.validateValue(CollectionElementBean.class, propertyName, element));
                }
            }

            if(!violations.isEmpty()) {
                valid = false;
            }

            if(allViolationMessages) { //TODO improve
                for(ConstraintViolation<?> violation : violations) {
                    logger.debug(violation.getMessage());
                    ConstraintViolationBuilder violationBuilder = context.buildConstraintViolationWithTemplate(violation.getMessage());
                    violationBuilder.addConstraintViolation();
                }
            }

        }

        return valid;
    }

    private boolean hasValidCollectionConstraint(Class<?> beanType) {
        BeanDescriptor beanDescriptor = validatorContext.getValidator().getConstraintsForClass(beanType);
        boolean isBeanConstrained = beanDescriptor.isBeanConstrained();
        if(!isBeanConstrained) {
            return false;
        }
        Set<ConstraintDescriptor<?>> constraintDescriptors = beanDescriptor.getConstraintDescriptors(); 
        for(ConstraintDescriptor<?> constraintDescriptor : constraintDescriptors) {
            if(constraintDescriptor.getAnnotation().annotationType().getName().equals(ValidCollection.class.getName())) {
                return true;
            }
        }
        Set<PropertyDescriptor> propertyDescriptors = beanDescriptor.getConstrainedProperties();
        for(PropertyDescriptor propertyDescriptor : propertyDescriptors) {
            constraintDescriptors = propertyDescriptor.getConstraintDescriptors();
            for(ConstraintDescriptor<?> constraintDescriptor : constraintDescriptors) {
                if(constraintDescriptor.getAnnotation().annotationType().getName().equals(ValidCollection.class.getName())) {
                    return true;
                }
            }    
        }
        return false;
    }

}

ValidatorContextAwareConstraintValidator

public interface ValidatorContextAwareConstraintValidator {

    void setValidatorContext(ValidatorContext validatorContext);

}

CollectionElementBean

    public class CollectionElementBean {

    /* add more properties on-demand */
    private Object notNull;
    private String notBlank;
    private String email;

    protected CollectionElementBean() {
    }

    @NotNull
    public Object getNotNull() { return notNull; }
    public void setNotNull(Object notNull) { this.notNull = notNull; }

    @NotBlank
    public String getNotBlank() { return notBlank; }
    public void setNotBlank(String notBlank) { this.notBlank = notBlank; }

    @Email
    public String getEmail() { return email; }
    public void setEmail(String email) { this.email = email; }

}

ConstraintValidatorFactoryImpl

public class ConstraintValidatorFactoryImpl implements ConstraintValidatorFactory {

    private ValidatorContext validatorContext;

    public ConstraintValidatorFactoryImpl(ValidatorContext nativeValidator) {
        this.validatorContext = nativeValidator;
    }

    @Override
    public <T extends ConstraintValidator<?, ?>> T getInstance(Class<T> key) {
        T instance = null;

        try {
            instance = key.newInstance();
        } catch (Exception e) { 
            // could not instantiate class
            e.printStackTrace();
        }

        if(ValidatorContextAwareConstraintValidator.class.isAssignableFrom(key)) {
            ValidatorContextAwareConstraintValidator validator = (ValidatorContextAwareConstraintValidator) instance;
            validator.setValidatorContext(validatorContext);
        }

        return instance;
    }

}

Employee

public class Employee {

    private String firstName;
    private String lastName;
    private List<String> emailAddresses;

    @NotNull
    public String getFirstName() { return firstName; }
    public void setFirstName(String firstName) { this.firstName = firstName; }

    public String getLastName() { return lastName; }
    public void setLastName(String lastName) { this.lastName = lastName; }

    @ValidCollection(elementType=String.class, constraints={Email.class})
    public List<String> getEmailAddresses() { return emailAddresses; }
    public void setEmailAddresses(List<String> emailAddresses) { this.emailAddresses = emailAddresses; }

}

Team

public class Team {

    private String name;
    private Set<Employee> members;

    public String getName() { return name; }
    public void setName(String name) { this.name = name; }

    @ValidCollection(elementType=Employee.class)
    public Set<Employee> getMembers() { return members; }
    public void setMembers(Set<Employee> members) { this.members = members; }

}

ShoppingCart

public class ShoppingCart {

    private List<String> items;

    @ValidCollection(elementType=String.class, constraints={NotBlank.class})
    public List<String> getItems() { return items; }
    public void setItems(List<String> items) { this.items = items; }

}

ValidCollectionTest

public class ValidCollectionTest {

    private static final Logger logger = LoggerFactory.getLogger(ValidCollectionTest.class);

    private ValidatorFactory validatorFactory;

    @BeforeClass
    public void createValidatorFactory() {
        validatorFactory = Validation.buildDefaultValidatorFactory();
    }

    private Validator getValidator() {
        ValidatorContext validatorContext = validatorFactory.usingContext();
        validatorContext.constraintValidatorFactory(new ConstraintValidatorFactoryImpl(validatorContext));
        Validator validator = validatorContext.getValidator();
        return validator;
    }

    @Test
    public void beanConstrained() {
        Employee se = new Employee();
        se.setFirstName("Santiago");
        se.setLastName("Ennis");
        se.setEmailAddresses(new ArrayList<String> ());
        se.getEmailAddresses().add("segmail.com");
        Employee me = new Employee();
        me.setEmailAddresses(new ArrayList<String> ());
        me.getEmailAddresses().add("me@gmail.com");

        Team team = new Team();
        team.setMembers(new HashSet<Employee>());
        team.getMembers().add(se);
        team.getMembers().add(me);

        Validator validator = getValidator();

        Set<ConstraintViolation<Team>> violations = validator.validate(team);
        for(ConstraintViolation<Team> violation : violations) {
            logger.info(violation.getMessage());
        }
    }

    @Test
    public void beanNotConstrained() {
        ShoppingCart cart = new ShoppingCart();
        cart.setItems(new ArrayList<String> ());
        cart.getItems().add("JSR-303 Book");
        cart.getItems().add("");

        Validator validator = getValidator();

        Set<ConstraintViolation<ShoppingCart>> violations = validator.validate(cart, Default.class);
        for(ConstraintViolation<ShoppingCart> violation : violations) {
            logger.info(violation.getMessage());
        }
    }

}

Output

02:16:37,581  INFO main validation.ValidCollectionTest:66 - {ValidCollection.message}
02:16:38,303  INFO main validation.ValidCollectionTest:66 - may not be null
02:16:39,092  INFO main validation.ValidCollectionTest:66 - not a well-formed email address

02:17:46,460  INFO main validation.ValidCollectionTest:81 - may not be empty
02:17:47,064  INFO main validation.ValidCollectionTest:81 - {ValidCollection.message}

Note:- When bean has constraints do NOT specify the constraints attribute of @ValidCollection constraint. The constraints attribute is necessary when bean has no constraint.

I couldn't figure out how exactly you could get the JUnit 4 running with all of your dependencies, but I did notice that you're dependency on the TestNG Unitils artifact was what was causing the problem.

Note that in order to get my test working, I had to remove the repositories, the plugins other than the compiler and surefire, and the apache rampart dependencies.

The three changes I made in order to get a sample JUnit 4 test case running were:

• Remove the junitArtifactName element from the surefire configuration
• Moved the junit artifact from the dependencyManagement section to the dependency section
• Remove the unitils-testng artifact from the dependencies. You have it listed as a having a type of pom, which makes even less sense.

Note that the unitils-core is also of type pom, which doesn't make much sense.

Please add the below line to avoid the mentioned exception

@SuppressStaticInitializationFor("packageName.StaticInitializerService")

Then rest of the mocking will be similar to static void method mocking and PowerMockito.doThrow() method to thow the exception.here is the sample code which is not throwing any exception for your class

    import org.junit.Test;
    import org.junit.runner.RunWith;
    import org.powermock.api.mockito.PowerMockito;
    import org.powermock.core.classloader.annotations.PrepareForTest;
    import org.powermock.core.classloader.annotations.SuppressStaticInitializationFor;
    import org.powermock.modules.junit4.PowerMockRunner;
    @RunWith(PowerMockRunner.class)
    @PrepareForTest(StaticInitializerService.class)
    @SuppressStaticInitializationFor("PackageName.StaticInitializerService")
    public class StaticInitializerServiceTest {
    @Test 
    public void method() {
    PowerMockito.mockStatic(StaticInitializerService.class);
    }   
  }

Take a look at the dependency injection section in the documentation. It states that dependency injection can be used for example in this case:

Any @BeforeMethod (and @AfterMethod) can declare a parameter of type java.lang.reflect.Method. This parameter will receive the test method that will be called once this @BeforeMethod finishes (or after the method as run for @AfterMethod).

So basically you just have to declare a parameter of type java.lang.reflect.Method in your @BeforeMethod and you will have access to the name of the following test name. Something like:

@BeforeMethod
protected void startTest(Method method) throws Exception {
    String testName = method.getName(); 
    System.out.println("Executing test: " + testName);
}

There's also a way by using the ITestNGMethod interface (documentation), but as I'm not exactly sure on how to use it, I'll just let you have a look at it if you're interested.