How to use newSuiteExecutor method of org.testng.thread.Interface IExecutorFactory class

Finally, I figured it out myself. I use javadoc combined with a little bit of annotation processing (just to distinguish the testcase groups). I use a custom doclet, which first builds the list of testcases in the following way:

private MethodDoc[] getTestMethods(RootDoc root) {
    List<MethodDoc> result = new ArrayList<MethodDoc>();
    for (ClassDoc c : root.classes()) {
        for(MethodDoc m : c.methods()) {
            if (isTest(m))
                result.add(m);
        }
    }
    return result.toArray(new MethodDoc[0]);
}

// simplified test detection
private static boolean isTest(MethodDoc m) {
    for(AnnotationDesc a: m.annotations()) {
        if (a.annotationType().name().equals("Test"))
            return true;
    }
    return false;
}

Then, for each test I retrieve the set of groups:

static Set<String> getMethodGroups(MethodDoc m) {
    Set<String> result = getGroups(m);
    result.addAll(getGroups(m.containingClass()));
    return result;
}

static Set<String> getGroups(ProgramElementDoc m) {
    Set<String> result = new HashSet<String>();
    for(AnnotationDesc a: m.annotations()) {
        if (a.annotationType().name().equals("Test")) {
            for(ElementValuePair ev : a.elementValues()) {
                if (ev.element().name().equals("groups")) {
                    String value = ev.value().toString();
                    StringTokenizer tokenizer = new StringTokenizer(value, "{}\", ");
                    while (tokenizer.hasMoreElements()) {
                        result.add(tokenizer.nextToken());
                    }
                }
            }
        }
    }
    return result;
}

The rest is just standard doclet processing. In addition, I found out that I can use custom tags directly in javadoc, e.g.

/**
 * Some description
 * @Step Step 1
 * @Step Step 2
*/
void someMethod() {}

You could add a parameter your @BeforeMethod with the type 'java.lang.reflect.Method'. TestNG will then inject the reflection information for the current test method including the method name, which you could use for switching.

If you add another 'Object' parameter, you will also get the invocation parameters of the test method.

You'all find all on possible parameters for TestNG-annotated methods in chapter 5.18.1 of the TestNG documentation.

I don't know of any matcher that can do what you want, so I'm afraid you'll have to program it.

I would simply sort the target list and then I'd iterate the sublists until a match is found:

List<String> target = new ArrayList<>(anotherList);
target.sort();

boolean result = myList.stream()
    .anyMatch(sublist -> equalsInAnyOrder(sublist, target));

Where method equalsInAnyOrder would be as follows:

public <T> boolean equalsInAnyOrder(List<T> sublist, List<T> target) {

    List<String> copy = new ArrayList<>(sublist);
    copy.sort();

    return copy.equals(target);
}

This sorts each sublist and compares it with the target sorted list, so it's not performance-wise, but at least it's simple and succint code.

EDIT as per OP's need to target Java 6:

The logic is exactly the same as in the Java 8 version. First sort the target list and then compare each sublist until a match is found:

List<String> target = new ArrayList<>(anotherList);
Collections.sort(target);

The stream() with anyMatch has now become a while loop:

boolean match = false;
Iterator<List<String>> it = myList.iterator();
while (it.hasNext() && !match) {
    List<String> sublist = it.next();
    match = equalsInAnyOrder(sublist, target);
}

And now method equalsInAnyOrder looks like this:

public <T> boolean equalsInAnyOrder(List<T> sublist, List<T> target) {

    List<String> copy = new ArrayList<>(sublist);
    Collections.sort(copy);

    return copy.equals(target);
}

There is no such a good idea to have following scenario :

for (WebElement element : webDriver.findElements(locator.getBy())){
  element.click();
}

Why? Because there is no guarantee that the element.click(); will have no effect on other found elements, so the DOM may be changed, so hence the StaleElementReferenceException.

It is better to use the following scenario :

int numberOfElementsFound = getNumberOfElementsFound(locator);
for (int pos = 0; pos < numberOfElementsFound; pos++) {
  getElementWithIndex(locator, pos).click();
}

This is better because you will always take the WebElement refreshed, even the previous click had some effects on it.

EDIT : Example added

  public int getNumberOfElementsFound(By by) {
    return webDriver.findElements(by).size();
  }

  public WebElement getElementWithIndex(By by, int pos) {
    return webDriver.findElements(by).get(pos);
  }

Hope to be enough.

Testng is calling methods in this specific order: serviceData, beforeMethod and m1.

If you want to pass a mock with a data provider, you have to create it before or into the data provider method.

JUnit and TestNG are the main unit test frameworks.

Comparison can be found here - http://www.mkyong.com/unittest/junit-4-vs-testng-comparison

I've used JUnit in most places I've worked, it seems to be the standard, but there are some nice things in TestNG.

I've used the following in combination with JUnit to facilitate testing:

• Mockito for mocking objects - http://code.google.com/p/mockito/
• Hamcrest assertions for more readable assertions - http://code.google.com/p/hamcrest/
• Twip for parameterized testing, the JUnit stuff isn't the greatest, in fact TestNG does this better as well - http://twip.sourceforge.net/

TestNG usually takes the Project Execution environment JRE. If you've changed the JRE from 1.6.x to 1.7 or 1.8 then it should work. Just to make sure the steps, 1) Open the Test file 2) Go to Run configurations 3) open JRE tab 4) click on Alternate JRE and select JDK/JRE > 1.6

In IntelliJ 14 maven profiles are not passed to tests (by default).

You can activate this mechanism via an option in Settings -> Maven -> Running Tests which let you enable passing of systemPropertyVariables to unit tests - this should cover your case.

JUnit has no such runner at the moment. Addressing the underlying issue, the need to get reasonable assurance from a test suite in a limited amount of time, is our highest development priority for the next release. In the meantime, implementing a Filter that works through annotations seems like it wouldn't be a big project, although I'm biased.