How to use omitEmptyStrings method of org.testng.collections.Objects.ToStringHelper class

The collaborator's doSomething() method doesn't declare that it may throw Exception, and you're telling its mock to throw one. It's not possible.

Exception is a checked exception. It can only be thrown if it's declared in the method signature. If the method has no throws clause, all it can do is throwing runtime exceptions (i.e. RuntimeException or any descendant class).

That is because the groups flag only indicates if a method belongs to a group. When that group is enabled, it is executed always, i.e. before all tests methods irrespective of the group the test methods belong to. By default all groups are enabled.

If you want to have the method only executed for the tests of certain groups, you would need to specify onlyForGroups.

public class BeforeTest {

    private static final Logger LOG = Logger.getLogger(BeforeTest.class);

    @BeforeMethod(onlyForGroups = { "g1" })
    public void setUpG1() {
        sleep();
        LOG.info("BeforeMethod G1");
    }

    @Test(groups = { "g1" })
    public void g1Test() {
        sleep();
        LOG.info("g1Test()");
    }

    @BeforeMethod(onlyForGroups = { "g2" }, alwaysRun = false)
    public void setUpG2() {
        sleep();
        LOG.info("BeforeMethod G2");
    }

    @Test(groups = { "g2" })
    public void g2Test() {
        sleep();
        LOG.info("g2Test()");
    }

    private void sleep() {
        try {
            Thread.sleep(500);
        } catch (InterruptedException e) {
            e.printStackTrace();
        }
    }
}

The output is

BeforeMethod G1
g1Test()
BeforeMethod G2
g2Test()

The glue option takes a path value which reflects the package(s) of the glue classes to be included in the classpath.

Find a simplified working example below

Assume following structure

/tmp/cuke-test/features/cukefeature.feature
/tmp/cuke-test/lib
/tmp/cuke-test/project1/src/com/cuke/CukeSteps.java
/tmp/cuke-test/project2/src/UseCukeFromMain.java

cukefeature.feature

Feature: simple test
  Scenario: test programatic call of Cucumber
  Given we have feature file
  When some glue code exists
  Then those steps should not fail

lib

cucumber-core-2.1.0.jar
cucumber-html-0.2.6.jar
cucumber-java-2.1.0.jar
cucumber-jvm-deps-1.0.6.jar
cucumber-testng-2.1.0.jar
gherkin-5.0.0.jar
jcommander-1.64.jar
snakeyaml-1.17.jar
tag-expressions-1.0.1.jar
testng-6.11.jar

CukeSteps.java

package com.cuke;

import cucumber.api.PendingException;
import cucumber.api.Scenario;
import cucumber.api.java.Before;
import cucumber.api.java.en.*;

public class CukeSteps {
    @Given("^we have feature file$")
    public void we_have_feature_file() throws Throwable {
        System.out.println("execute Given step");
    }

    @When("^some glue code exists$")
    public void some_glue_code_exists() throws Throwable {
        System.out.println("execute Then step");
    }

    @Then("^those steps should not fail$")
    public void those_steps_should_not_fail() throws Throwable {
        throw new PendingException();
    }
}

UseCukeFromMain.java

import cucumber.api.cli.Main;

public class UseCukeFromMain {
    public static void main(String[] args) throws Throwable {
        Main.main(new String[]{
            "--glue",
            "com/cuke", // the package which contains the glue classes
            "/tmp/cuke-test/features/cukefeature.feature"}
        );
    }
}

compile the classes

javac -cp "lib/*" -d project1/bin/ project1/src/com/cuke/*.java
javac -cp "lib/*" -d project2/bin/ project2/src/*.java

run the UseCukeFromMain

The root direcotry which contains the glue classes (project1/bin) must be in the classpath.

java -cp "project2/bin:project1/bin:lib/*" UseCukeFromMain

output

execute Given step
execute Then step

1 Scenarios (1 pending)
3 Steps (1 pending, 2 passed)
0m0.104s

cucumber.api.PendingException: TODO: implement me
    at com.cuke.CukeSteps.those_steps_should_not_fail(CukeSteps.java:21)
    at ✽.those steps should not fail(/tmp/cuke-test/features/cukefeature.feature:6)

edit Using Step definitions in default package

Assume following structure

features/cukefeature.feature
lib/
project1/src/CukeSteps.java
project2/src/UseCukeFromMain.java

cukefeature.feature
lib/

the same as in the first example

CukeSteps.java

// note: there is no package statement

import cucumber.api.PendingException;
import cucumber.api.Scenario;
import cucumber.api.java.Before;
import cucumber.api.java.en.*;

public class CukeSteps {
    @Given("^we have feature file$")
    public void we_have_feature_file() throws Throwable {
        System.out.println("execute Given step");
    }

    @When("^some glue code exists$")
    public void some_glue_code_exists() throws Throwable {
        System.out.println("execute Then step");
    }

    @Then("^those steps should not fail$")
    public void those_steps_should_not_fail() throws Throwable {
        throw new PendingException();
    }
}

UseCukeFromMain.java

import cucumber.api.cli.Main;

public class UseCukeFromMain {
    public static void main(String[] args) throws Throwable {
        Main.main(new String[]{
            "--glue",
            "",  // to used Step definitions in default package
            "features/cukefeature.feature"}
        );
    }
}

compile classes

The option -d . creates the class files in the current directory.

javac -cp "lib/*" -d . project1/src/*.java
javac -cp "lib/*" -d project2/bin/ project2/src/*.java

created class files

CukeSteps.class
project2/bin/UseCukeFromMain.class

run the UseCukeFromMain

The current directory is added to the classpath using the ..

java -cp "project2/bin:.:lib/*" UseCukeFromMain

output

execute Given step - default package
execute Then step - default package

1 Scenarios (1 pending)
3 Steps (1 pending, 2 passed)
0m0.096s

cucumber.api.PendingException: TODO: implement me
    at CukeSteps.those_steps_should_not_fail(CukeSteps.java:19)
    at ✽.those steps should not fail(features/cukefeature.feature:5)

edit Using Step definitions from different packages.

Assume following structure

features/cukefeature.feature
lib
project1/src/com/cuke1/CukeStepsB.java
project1/src/com/cuke/CukeStepsA.java
project2/src/UseCukeFromMain.java

cukefeature.feature
lib/

the same as in the first example

The Step definitions are split in two classes, in different packages

CukeStepsA.java

package com.cuke;

import cucumber.api.PendingException;
import cucumber.api.Scenario;
import cucumber.api.java.Before;
import cucumber.api.java.en.*;

public class CukeStepsA {
    @Given("^we have feature file$")
    public void we_have_feature_file() throws Throwable {
        System.out.println("execute Given step - package com.cuke");
    }
}

CukeStepsB.java

package com.cuke1;

import cucumber.api.PendingException;
import cucumber.api.Scenario;
import cucumber.api.java.Before;
import cucumber.api.java.en.*;

public class CukeStepsB {
    @When("^some glue code exists$")
    public void some_glue_code_exists() throws Throwable {
        System.out.println("execute Then step - package com.cuke1");
    }

    @Then("^those steps should not fail$")
    public void those_steps_should_not_fail() throws Throwable {
        throw new PendingException();
    }
}

UseCukeFromMain.java

import cucumber.api.cli.Main;

public class UseCukeFromMain {
    public static void main(String[] args) throws Throwable {
        Main.main(new String[]{
            "--glue",
            "com/cuke",
            "--glue",
            "com/cuke1",
            "features/cukefeature.feature"}
        );
    }
}

compile classes

javac -cp "lib/*" -d project1/bin/ project1/src/com/cuke/*.java project1/src/com/cuke1/*.java
javac -cp "lib/*" -d project2/bin/ project2/src/*.java

created class files

project1/bin/com/cuke1/CukeStepsB.class
project1/bin/com/cuke/CukeStepsA.class
project2/bin/UseCukeFromMain.class

run the UseCukeFromMain

java -cp "project2/bin:project1/bin:lib/*" UseCukeFromMain

output

execute Given step - package com.cuke
execute Then step - package com.cuke1

1 Scenarios (1 pending)
3 Steps (1 pending, 2 passed)
0m0.114s

cucumber.api.PendingException: TODO: implement me
    at com.cuke1.CukeStepsB.those_steps_should_not_fail(CukeStepsB.java:16)
    at ✽.those steps should not fail(features/cukefeature.feature:5)

Yes, that's possible. You can create an annotation to specify the parameters this DataProvider should load from your XML.

@Target(ElementType.METHOD)
@Retention(RetentionPolicy.RUNTIME)
public @interface XmlParameters {
   String[] value();    
}

@Test(dataProvider = "XMLFileLoader")
@XmlParameters({"username", "password"})
public void testSomething(String username, String password) {
    // implementation omitted for brevity
}

@DataProvider(name = "XMLFileLoader")
public static Object[][] getDataFromXmlFile(final Method testMethod) {
    XmlParameters parameters = testMethod.getAnnotation(XmlParameters.class);
    String[] fields = parameters.value();
    //load just the fields you want
    return new Object[][] { { "user1", "pass1" } };
}

This code is not "production ready", you should check if the annotation is not null before reading the values, and is probably better move the interface and the logic to load your xml to another class, so you can reuse on another tests.

Have you looked at $JENKINS_HOME/plugins/email-ext/WEB-INF/classes/hudson/plugins/emailext/templates/html.jelly ? If that isn't sufficient, let me know what additional information you're trying to find and I'll see if I can help.

The only way you can achieve that is to understand how the TestNG framework executes.

Which means:

1. SkipException is not the solution as it will always count all the test cases, even if they are skipped.
2. The number of tests is the number of rows returned by the @DataProvider method.

The solution is then to return the correct number of test cases from the method annotated by @DataProvider:

public class ReadAutoData {
    private static int indexFrom;
    private static int indexTo;

    @DataProvider(name = "autodata")
    public static Object[][] autodata() {
        // you should probably cache this into static variable
        Object[][] arrayObject = getExcelData("C:/dev/AutoDP.xls","NON-COMBO-DP"); 

        return java.util.Arrays.copyOfRange(arrayObject, indexFrom, indexTo);
    }

    public Class<?> autoDataWithinRange(int from, int to) {
        indexFrom = from;
        indexTo   = to;

        return ReadAutoData.class;
    }
}

@Test(dataProvider="autodata", dataProviderClass = ReadAutoData.autoDataWithinRange(0, 10))
public void autoentry(String Quote, String Garage_Zip, etc...) {

This would work if you are not running test concurrently. If you are, you can still use it with small modification using ThreadLocals.

I'm the creator of TestNG and author of the "Next Generation Testing" book, so here are a few (obviously biased) thoughts.

For functional (or end to end / integration / system, whatever term you prefer) testing, TestNG offers a few features that JUnit doesn't have and that users have found very useful:

• Groups. Once you have compiled your tests, you can just ask TestNG to run all the "front-end" tests, or "fast", "slow", "database", etc...

• Support for multithreaded testing. This has two different meanings:

1) You can tell TestNG to run all your tests concurrently in thread pools that you define (one line change in an XML file). This is very configurable so that tests that are not multithreaded can still be run in single threads.

2) You can tell TestNG to invoke your test methods n times from p threads. This gives you a good idea of how thread safe the code you are testing is.

• Test dependencies and deterministic ordering. This is very useful for integration testing where it's very common to see test methods that need prior test methods to succeed before they can even run. TestNG is very popular in the Selenium community for this specific reason ("don't bother testing this page of my web site if the login test failed").

Another immediate advantage of test dependencies is that TestNG can tell you "1 method failed, 99 methods skipped", which is a much more accurate result than "100 methods failed".

TestNG has many, many more features and I'm happy to answer questions here or on our mailing-list (http://groups.google.com/group/testng-users ).

You have to rework your tests:

@BeforeSuite is only called once by suite. You have 2 classes then the first has the dao instanced, the other not (NPE in this case).

One option is to use static for your dao. Another solution could be to put documentDao = DAOFactory.getDocumentDAO(); into @BeforeClass. But you can find many other (better) solutions.

default behaviour for configFailurePolicy is "skip". Because if you get exception in configuration method then there is not need to run your test. But if you want to run your test anyway, change this configuration to "continue" It could be achieved by updating configuration

  <suite name="SuiteName" parallel="tests" thread-count="1" verbose="10" configfailurepolicy="continue">