How to use beforeConfiguration method of org.testng.reporters.JUnitXMLReporter class

Spying is not needed, mocking is enough. As @David said, also mocking is not needed and not recommended for value object.

Using @Test(expected = Exception.class) has many drawbacks, test can pass when exception is thrown from not expected places. Test is not working but is visible as green.

I prefer BDD style testing with catch-exception.

Reasons for using catch-exceptions

(...) in comparison to the use of try/catch blocks.

• The test is more concise and easier to read.
• The test cannot be corrupted by a missing assertion. Assume you forgot to type fail() behind the method call that is expected to throw an exception.

(...) in comparison to test runner-specific mechanisms that catch and verify exceptions.

• A single test can verify more than one thrown exception.
• The test can verify the properties of the thrown exception after the exception is caught.
• The test can specify by which method call the exception must be thrown.
• The test does not depend on a specific test runner (JUnit4, TestNG).

import static com.googlecode.catchexception.CatchException.caughtException;
import static com.googlecode.catchexception.apis.CatchExceptionAssertJ.*;

public class FooBarTest {

    FooBar sut = new FooBar(); // System Under Test

    @Test
    public void shouldThrowExceptionWhenListHasTooManyElements() {

        when(sut).getMethod(listWithSize(150));

        then(caughtException()).isInstanceOf(Exception.class);
    }

    private List<String> listWithSize(int size) {
        return new ArrayList<String>(Arrays.asList(new String[size]));
    }
}

Full working code for this test: https://gist.github.com/mariuszs/8543918

Not recommended solution with expected and mocking.

@RunWith(MockitoJUnitRunner.class)
public class FooBarTest {

    @Mock
    List<String> codes;

    FooBar fooBarInstance = new FooBar();

    @Test(expected = Exception.class)
    public void shouldThrowExceptionWhenListHasTooManyElements() throws Exception {

        when(codes.size()).thenReturn(150);

        fooBarInstance.getMethod(codes);

    }
}

As mentioned in the warning

Dec 09, 2020 9:16:16 PM io.cucumber.testng.TestNGCucumberOptionsProvider warnWhenJUnitCucumberOptionsAreUsed
WARNING: Ignoring options provided by io.cucumber.junit.CucumberOptions on cucumberOptions.TestRunner. It is recommend to use separate runner classes for JUnit and TestNG.

Currently you are importing junit cucumber options.

import io.cucumber.junit.CucumberOptions;

For TestNg runner class you will need to import testng.CucumberOptions

import io.cucumber.testng.CucumberOptions;

Example from the cucumber-jvm repository

import io.cucumber.testng.AbstractTestNGCucumberTests;
import io.cucumber.testng.CucumberOptions;
import org.testng.annotations.DataProvider;

@CucumberOptions(plugin = { "html:target/results.html", "message:target/results.ndjson" })
public class RunCucumberTest extends AbstractTestNGCucumberTests {

    @DataProvider(parallel = true)
    @Override
    public Object[][] scenarios() {
        return super.scenarios();
    }

}

Okay, I was able to figure it out. And of course it was incredibly dumb and makes you question all of your years being a software engineer.

The problem was this line:

String className = type.toString();

The value type was of class java.lang.reflect.Type.

Can you see the issue? Don't feel bad if you can't because I couldn't either.

If you run this code:

System.out.println(type.toString());
System.out.println(type.getTypeName());

You end up with the following output:

class java.lang.String
java.lang.String

The issue is quite obvious seeing that, I should be using getTypeName() not toString(). Of course it will throw ClassNotFoundException because class<SPACE> precedes the actual class name.

Yet when I looked at the stacktrace, I overlooked it had class<SPACE> before the name of the class.

Sigh. Good times were had learning about class loaders though.

The submit() method returns a Future<?>. If you try to get the result of the Future, the method will throw the Throwable that has been thrown inside the Runnable:

Future<?> future = stpe.submit( .... 

future.get(); // this call will throw the exception that has been thrown in the Runnable.

This way, exceptions/errors in the run method will make the test fail.

My question is, why is @AfterTest method not run after each @Test annotation?

As the Documentation says

@AfterTest: The annotated method will be run after all the test methods belonging to the classes inside the <test> tag have run.

and

@BeforeTest: The annotated method will be run before any test method belonging to the classes inside the <test> tag is run.

Indeed you will need a custom converter for this as it is two nested collection which you do not want to show and those objects might have other child elements which would be serialized.

This is a case not directly covered by XStream, but easily fixed with a custom converter.

You can find all this in a git clone of your project which contains all those modifications. But as everything should be visible here, I subsequently explain the key portions to the question including some code examples.

I implemented such a converter for your project. The code doing the conversion is this:

  @Override
  public void marshal(Object source, HierarchicalStreamWriter writer, MarshallingContext context) {
    TestRow test = (TestRow)source;
    for (TestArguments arg : test.getArguments()) {
      for (ArgObject val : arg.getAllTestArguments()) {
        writer.startNode(val.getKey());
        writer.addAttribute("type", val.getType());
        writer.setValue(val.getVal());
        writer.endNode();

      }
    }
  }

  @Override
  public Object unmarshal(HierarchicalStreamReader reader, UnmarshallingContext context) {
    TestArguments testargs = new TestArguments();
    while (reader.hasMoreChildren()) {
      reader.moveDown();
      String typeval = reader.getAttribute("type");
      if (typeval.isEmpty()) {
        typeval = "null";
      }
      testargs.getAllTestArguments().add(new ArgObject(reader.getNodeName(), typeval, reader.getValue()));
      reader.moveUp();
    }
    TestRow result = new TestRow(testargs);
    return result;
  }

This will serialize all ArgObjects to xml with the flat node structure you defined in your ArgConverter. The deserialization creates the object back from this data.

You had two bugs in your source code which prevented deserialization:

• XStream deserialization does not recognize all alias Annotations. It had problems with your root element SuiteData (aliased to suite). For this I added a aliad to the XStream object like this xStream.alias("suite", SuiteData.class); in your XMLDataHelper.
• If you create a TestArguments instance with a default constructor (as you'll have to through deserialization), you call reset which assigns your argsWrapper member to null which then makes adding arguments quite impossible. I fixed that be newly initializing that member variable in the reset method (argsWrapper = new ArrayList<ArgObject>();).

As far as I understand your code this works as expected for serialization / deserialization. I wrote a small test program to force this process in each way and it seems to produce the same result:

public class XMLDataHelperTest {

  public static void main(String[] args) {
    File file = new File(System.getProperty("user.dir"), "test.xml");
    if (file.isFile()) {
      assertTrue(file.delete());
    }

    // write it once
    XMLDataHelper helper = new XMLDataHelper(file.getAbsolutePath());

    // read it once
    helper = new XMLDataHelper(file.getAbsolutePath());
    System.out.println(file);
  }
}

It is perfectly valid to make sure a unit test runs without encountering an exception.

As per Matt B's suggestion, be sure to document what the test is actually testing to be clear and precise.

There are two steps to accomplished this task:-

Step 1:-

1. Go to localhost:8080/configure (Jenkins configure section)

2. Now go to JDK section and uncheck Install automatically (If you don't uncheck that then it will download latest java every time whenever it is available, and can cause for build failed)

3. put JAVA_HOME in name section and jdk home path in JAVA_HOME section

4. Apply and save

Step 2:-

1. Go to Jenkins and add new Item, also select "Free Style Project" and click on Ok.

2. Click on "Advanced in "Advanced Project Options"

3. Now check option: - "Use custom workspace" and specify your project absolute path in Directory section

4. Apply

5. Now to go "Build" and select "Execute windows batch command"

6. Here in command column give the file name of your batch file
7. Apply and save

Now go to the Jenkins and select your Jenkins project and click on Build :)

The problem lies in your test code. Your webdriver instances are getting shared across test methods when you run tests in parallel and powered by a data provider.

Here's a fixed version of your BaseTest and BrowserDriverFactory to include thread safety.

import java.lang.reflect.Method;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.firefox.FirefoxProfile;
import org.openqa.selenium.support.ui.WebDriverWait;
import org.testng.ITestContext;
import org.testng.ITestResult;
import org.testng.annotations.AfterMethod;
import org.testng.annotations.BeforeMethod;

public class BaseTest {
  private static final ThreadLocal<WebDriver> drivers = new ThreadLocal<>();
  public WebDriverWait wait;
  protected FirefoxProfile profile;
  protected String url = "http://www.qaclickacademy.com/";

  protected String testSuiteName;
  protected String testName;
  protected String testMethodName;

  public WebDriver getDriver() {
    return drivers.get();
  }

  @BeforeMethod()
  public void setUp(Method method, ITestContext ctx) {
    String testName = ctx.getCurrentXmlTest().getName();

    WebDriver driver = BrowserDriverFactory.createDriver("firefox");
    drivers.set(driver);
    driver.get(url);

    driver.manage().window().maximize();
    wait = new WebDriverWait(driver, 5);

    this.testSuiteName = ctx.getSuite().getName();
    this.testName = testName;
    this.testMethodName = method.getName();
  }

  @AfterMethod(alwaysRun = true)
  public void tearDown(ITestResult result) {
    getDriver().quit();
    drivers.remove();
  }
}

BrowserDriverFactory.java

import org.openqa.selenium.WebDriver;
import org.openqa.selenium.chrome.ChromeDriver;
import org.openqa.selenium.firefox.FirefoxDriver;
import org.openqa.selenium.firefox.FirefoxOptions;
import org.openqa.selenium.firefox.FirefoxProfile;

public class BrowserDriverFactory {

  public static WebDriver createDriver(String browser) {
    System.err.println("Create driver: " + browser);

    if (browser.equals("chrome")) {
      System.setProperty("webdriver.chrome.driver", "src/main/resources/chromedriver.exe");
      return new ChromeDriver();
    } else if (browser.equals("firefox")) {
      System.setProperty("webdriver.gecko.driver", "src/main/resources/geckodriver.exe");
      FirefoxOptions options = new FirefoxOptions();
      FirefoxProfile profile = new FirefoxProfile();
      profile.setPreference("browser.helperApps.neverAsk.openFile", "application/octet-stream");
      options.setProfile(profile);
      return new FirefoxDriver(options);
    }
    System.out.println("Do not know how to start: " + browser + ", starting chrome.");
    System.setProperty("webdriver.chrome.driver", "src/main/resources/chromedriver.exe");
    return new ChromeDriver();
  }
}

Now in all the test methods of test classes that extend BaseTest whenever you need access to the driver object, you would get it by invoking getDriver() method.

You can find a full fledge explanation and some more explanation around this in this blog of mine: https://rationaleemotions.com/parallel_webdriver_executions_using_testng/

I also created a library that abstracts out the webdriver lifecycle management and makes it even more simpler for you via a custom annotation.

Take a look at https://github.com/rationaleEmotions/autospawn