How to use getDefaultFileNameForXmlReports method of org.testng.reporters.RuntimeBehavior class

I recommend to use approach from article: https://github.com/mockito/mockito/wiki/Mocking-Object-Creation

You need to change little bit a class, applying approach for mocking from article in following way:

AwsHelper

public class AwsHelper {

    private AmazonSQS sqs;

    public void sendMessageToQueue(String message) {
        sqs = defaultClient();

        SendMessageRequest sendMessageRequest = new SendMessageRequest();
        sendMessageRequest.setQueueUrl("");
        sendMessageRequest.setMessageBody(message);
        sendMessageRequest.setMessageGroupId("");

        sqs.sendMessage(sendMessageRequest);
    }

    protected AmazonSQS defaultClient() {
        return AmazonSQSClientBuilder.defaultClient();
    }
}

AwsSQSReferralsUtilTest

public class AwsSQSReferralsUtilTest {

    @Spy
    private AwsHelper awsHelper;

    @Mock
    private AmazonSQS sqs;

    @BeforeClass
    public void setUp() {
        MockitoAnnotations.initMocks(this);
    }
    
    @AfterMethod
    public void afterEachMethod() {
        Mockito.reset(awsHelper);
    }

    @Test
    public void shouldSendMessage() {
        //mocking object creation
        doReturn(sqs).when(awsHelper).defaultClient();

        when(sqs.sendMessage(any(SendMessageRequest.class))).thenReturn(new SendMessageRequest());
        awsHelper.sendMessageToQueue("");
    }

}

Make sure you call System.setProperty("property.name", "https://justatest.com/dav/bulk"); before the class being tested is instantiated, otherwise the static field will always be null.

Consider using a @BeforeClass setup method for this:

@BeforeClass
public static void setup() {
    System.setProperty("property.name", "https://justatest.com/dav/bulk");
    // Instantiate your CsvConfig instance here if applicable.
}

and then

@Test
public void getUrl(){
    System.setProperty("property.name", "https://justatest.com/dav/bulk");
    String fileUrl = csvConfig.getFileUrl();
    assertEquals(fileUrl, "https://justatest.com/dav/bulk/otherstuff");
}

You are catching NoSuchElementException and inside the catch block you are checking if it is a NoSuchElementException which does not add any value. You are testing if JDK works correctly.

To check if the element does not exist on page, you could use a method like this :

public boolean isElementPresent(By locatorKey) {
    try {
          driver.findElement(locatorKey);
          return true;
        } catch (org.openqa.selenium.NoSuchElementException e) {
          return false;
        }
}

You should then assert on the above method,

if element should not be present

Assert.assertFalse(isElementPresent(By.id("login")));

if element should be present

Assert.assertTrue(isElementPresent(By.id("login")));

A bit late, but I ran into the same problem and was able to resolve it.

The problem was a mismatch between the version of Arquillian core and Arquillian persistence.

For Arquillian Persistence version 1.0.0.Alpha7, when running with Arquillian Core 1.1.5.Final, this error was presenting itself.

If the Arquillian Core version is changed to 1.1.2.Final, the error goes away... :-D

You can use a combination of the IRetryAnalyzer, a Listener and a custom reporter to do what you are looking for.

IRetryAnalyzer:

public class RetryAnalyzer implements IRetryAnalyzer  { 
private int count = 0; 
// this number is actually twice the number
// of retry attempts will allow due to the retry
// method being called twice for each retry
private int maxCount = 6; 
protected Logger log;
private static Logger testbaseLog;

static {
    PropertyConfigurator.configure("test-config/log4j.properties");
    testbaseLog = Logger.getLogger("testbase.testng");
}

public RetryAnalyzer()
{
    testbaseLog.trace( " ModeledRetryAnalyzer constructor " + this.getClass().getName() );
    log = Logger.getLogger("transcript.test");
}

@Override 
public boolean retry(ITestResult result) { 
    testbaseLog.trace("running retry logic for  '" 
            + result.getName() 
            + "' on class " + this.getClass().getName() );
        if(count < maxCount) {                     
                count++;                                    
                return true; 
        } 
        return false; 
}
}

RetryListener:

public class RetryTestListener extends TestListenerAdapter  {
private int count = 0; 
private int maxCount = 3; 

@Override
public void onTestFailure(ITestResult result) {     
    Logger log = Logger.getLogger("transcript.test");
    Reporter.setCurrentTestResult(result);

    if(result.getMethod().getRetryAnalyzer().retry(result)) {    
        count++;
        result.setStatus(ITestResult.SKIP);
        log.warn("Error in " + result.getName() + " with status " 
                + result.getStatus()+ " Retrying " + count + " of 3 times");
        log.info("Setting test run attempt status to Skipped");                 
    } 
    else
    {
        count = 0;
        log.error("Retry limit exceeded for " + result.getName());
    }       

    Reporter.setCurrentTestResult(null);
}

@Override
public void onTestSuccess(ITestResult result)
{
    count = 0;
}

However, there appears to be a bug within TestNG that actually causes some of the test results to be reported as both skipped AND failed. To prevent this, I recommend that you override whatever Reporter you wish to use and include a method such as the one included below:

private IResultMap removeIncorrectlyFailedTests(ITestContext test)
{     
  List<ITestNGMethod> failsToRemove = new ArrayList<ITestNGMethod>();
  IResultMap returnValue = test.getFailedTests();

  for(ITestResult result : test.getFailedTests().getAllResults())
  {
    long failedResultTime = result.getEndMillis();          

    for(ITestResult resultToCheck : test.getSkippedTests().getAllResults())
    {
        if(failedResultTime == resultToCheck.getEndMillis())
        {
            failsToRemove.add(resultToCheck.getMethod());
            break;
        }
    }

    for(ITestResult resultToCheck : test.getPassedTests().getAllResults())
    {
        if(failedResultTime == resultToCheck.getEndMillis())
        {
            failsToRemove.add(resultToCheck.getMethod());
            break;
        }
    }           
  }

  for(ITestNGMethod method : failsToRemove)
  {
      returnValue.removeResult(method);
  }  

  return returnValue;
}

After all of this is done, you can add the reporter using using .addListener and specify the retryAnalyzer in the @Test annotation.

You should design your test cases to be independent, because modularity decreases complexity: If tests are dependent, you would always need to run all test cases if the last one failed, and needed to check all test cases whether they might have caused the error/failure.

With independent tests, you can run all of them, e.g. during a nightly build, even if some of them fail. Or execute a specific one in isolation. And you need not worry about the ordering of tests. That is why almost all test runners execute all tests and make no guarantee about their order.

This applies especially for unit tests. For a few integration tests and acceptance tests some dependency might be acceptable. In that case, use TestNG's dependency feature.

The assertEquals(Collection<?> actual, Collection<?> expected) documentation says:

Asserts that two collections contain the same elements in the same order. If they do not, an AssertionError is thrown.

Thus the content of the collections will be compared which, in case both the collections are empty, are equal.

Within your running TestNG tagged test, Thread.activeCount() gives you the number of threads just in your current TestNG group. If your threadPoolSize is set to 1, this number will be 1. If your threadPoolSize is greater than 1, this number will be threadPoolSize + 1, because there is also a TestNG manager thread in addition to the pool size. For threads that you create on the fly, you need to construct them passing in your TestNG ThreadGroup. Example:

    Thread aNewWorkerThread = new Thread( Thread.currentThread().getThreadGroup(), yourRunnableThingHere ); 

Those threads, when active, should then be reflected in your Thread.activeCount() return value.

You should use a @Factory.

public class SomeTest {

    @Factory
    public Object[] createInstances() {
        Object[] result = new Object[]{            
            new SomeTest(WebDrivers.getChromeDriver())
            // you can add other drivers here
        };
        return result;
    }

    private final WebDriver driver;

    public SomeTest(WebDriver driver) {
        this.driver = driver
    }

    @DataProvider(name = "URLs")
    public Object[][] createData1() {
     return new Object[][] {
       {"url 1"},
       {"url 2"},
       {"url 3"}};
    }    

    @AfterClass
    public void closeWebDriver(){
        driver.quit();
    }   

    //test methods below

    @Test(dataProvider = "URLs")
    public void test1(String url){
        //test 1 with url
    }

    @Test(dataProvider = "URLs")
    public void test2(String url){
        //test 2 with url
    }

    @Test(dataProvider = "URLs")
    public void test3(String url){
        //test 3 with url
    }

    @Test(dataProvider = "URLs")
    public void test4(String url){
        //test 4 with url
    }

}