How to use Parser class of org.testng.xml.internal package

I feel like your misunderstanding the purposes of WebDriver, Grid and a multi-threaded environment.

Selenium WebDriver is just a test framework that kicks off tests onto a browser. It uses drivers such as geckodriver, and chromedriver to open (and close) browser instances and then hits those instances with actions.

Selenium Grid is a stand-alone server that will run your WebDriver tests remotely on a node and reports back. So if you wanted to use another machines resources to run your test-suite you would be able to do so. It also allows you to combine multiple machines resources by assigning each a node and Selenium Grid will distribute your tests evenly.

As far as your questions about multi-threading,

Is it recommended to use Grid over Selenium WebDriver in a multithreaded environment?

Grid is just a remote-runner of WebDriver. WebDriver is the test-framework. I assume your goal is to reduce your test-suite's run-time by running tests in parallel? Parallel testing can be done on your local machine using WebDriver or can be configured on a Selenium Grid. In my experience, I have gotten more lucky on my local machine than on Grid but both essentially face the same problem. The main danger for parallel testing using WebDriver is thread-safety. JUnit 4.7 onwards and Test-NG RELEASE allow for parallel testing. I highly suggest using Test-NG instead of JUnit for this purpose.

To prevent concurrency issues, your main goal will be to make all your shared resources isolated within each thread. A secondary goal is to limit the scope of all variables to its method. This can be done using a static instance of ThreadLocal It is also really useful to force the initializing of driver instances to be frozen in synchronized methods.

Creating Resources

public class DriverFactory {

    private static String grid = "http://localhost:4444/wd/hub";
    public static ThreadLocal<WebDriver> drivers = new ThreadLocal<WebDriver>();

    public static synchronized void newDriver() {
        ChromeOptions options = new ChromeOptions(); // Assuming Chrome use
            options.addArguments("--start-maximized");

        // USE THIS FOR LOCAL
        // tlDriver.set(new ChromeDriver(options));

        // USE THIS FOR GRID
        try {
            drivers.set(new RemoteWebDriver(new URL(grid),options));
        } catch (MalformedURLException e) {
            e.printStackTrace();
        }
    }

    public static synchronized WebDriver getDriver() {
        return drivers.get();
    }
}

Test Class

public class GoogleTest {
    @BeforeMethod
    public void setup() {
         DriverFactory.newDriver();
    }

    @Test
    public void test1() {
        WebDriver driver = DriverFactory.getDriver();
        driver.navigate().to("https:\\www.google.com");
        Assert.assertEquals(driver.getTitle(), "Google");
    }

    @Test
    public void test2() {
        WebDriver driver = DriverFactory.getDriver();
        driver.navigate().to("https:\\www.google.com");
        Assert.assertEquals(driver.getTitle(), "Google");
    }


    @AfterMethod
    public void tearDown() {
        WebDriver driver = DriverFactory.getDriver();
        driver.quit();
    }
}

TestNG.xml

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd">
<!-- thread-count should be = to maxSessions on your Grid -->
<suite name="Whatever" parallel="methods" thread-count="5">
  <test name="Sample">
    <classes>
      <class name="GoogleTest"/>
    </classes>
  </test>
</suite>

Source: Making your Tests Thread-Safe, Full Guide
Versions Used: TestNG 6.14.3, Selenium 3.11, Selenium Grid 3.14.3

Also, would Selenium Grid take responsibility of cleaning any stale browser instances in its nodes (out of the box)?

Selenium Grid has timeout and a browserTimeout features to free up nodes and close browser instances. Although in my experience I would implore you to control any expected timeouts in your test-cases and have WebDriver close the browser instance, not Selenium Grid. You really want to prevent this from happening and eliminate all causes of un-expected hanging. Test-suites are designed to be quick feedback and are only useful if failures are indicative of an actual failure, not just a dirty test-suite. Also stale nodes/browsers can force you to have to constantly reset your grid + nodes which is a major detriment.

In Run from testNg.xml directly -By default in maven project html reports get created into test-output folder

In case we run from pom.xml using surefire plugin - Reports will get created into target\surefire-reports.

Follow the steps below:

Step# 1:

Create a custom annotation in a separate file (ie: SetTestName.java)

@Retention(RetentionPolicy.RUNTIME)
public @interface SetTestName {
    int idx() default 0;
}

Step# 2:

Create a base class implementing ITest interface of TestNG (TestNameSetter.java).

public class TestNameSetter implements ITest{
    private String newTestName = "";

    private void setTestName(String newTestName){
        this.newTestName = newTestName;
    }

    public String getTestName() {

        return newTestName;
    }


    @BeforeMethod(alwaysRun=true)
    public void getTheNameFromParemeters(Method method, Object [] parameters){
        SetTestName setTestName = method.getAnnotation(SetTestName.class);
        String testCaseName = (String) parameters[setTestName.idx()];
        setTestName(testCaseName);
    }
}

Step# 3:

Use your DataProvider like in the code snippet:

@DataProvider(name="userData")
 public Object[][] sampleDataProvider()
 {
  Object[][] data = {
    {"loginTestUS_Username","loginTestUSPass"}, 
    {"loginTestIN_Username","loginTestINPass"},
    {"loginTestJP_UserName","loginTestJPPass"}
  };

  return data;
 }



 @SetTestName(idx=0)
 @Test(dataProvider="userData")
 public void test1(String userName, String pass)
 {
     System.out.println("Testcase 1");
 }

 @SetTestName(idx=1)
 @Test(dataProvider="userData")
 public void test2(String userName, String pass)
 {
     System.out.println("Testcase 2");
 } 

That's all. Now, you will see your test name changed accordingly in the console.

Follow the link below for your query. I hope, you might get your desired answer here:

http://biggerwrench.blogspot.com/2014/02/testng-dynamically-naming-tests-from.html

You need to have the testng.jar under classpath.

try C:\projectfred> java -cp "path-tojar/testng.jar:path_to_yourtest_classes" org.testng.TestNG testng.xml

Update:

Under linux I ran this command and it would be some thing similar on Windows either

test/bin# java -cp ".:../lib/*" org.testng.TestNG testng.xml

Directory structure:

/bin - All my test packages are under bin including testng.xml
/src - All source files are under src
/lib - All libraries required for the execution of tests are under this.

Once I compile all sources they go under bin directory. So, in the classpath I need to specify contents of bin directory and all the libraries like testng.xml, loggers etc over here. Also copy testng.xml to bin folder if you dont want to specify the full path where the testng.xml is available.

 /bin
    -- testng.xml
    -- testclasses
    -- Properties files if any.
 /lib
    -- testng.jar
    -- log4j.jar

Update:

Go to the folder MyProject and type run the java command like the way shown below:-

java -cp ".: C:\Program Files\jbdevstudio4\studio\plugins\*" org.testng.TestNG testng.xml

I believe the testng.xml file is under C:\Users\me\workspace\MyProject if not please give the full path for testng.xml file

Log4j 2 seems to be pumped on steroids, if you get MDC logging for free already using the rolling file appender.

In any case, your log4j snippet looks strange. We see the closing appender element tag, but not its corresponding opening appender tag.

Your rolling file appender name seems to have a space between the dynamic test name and test start time.

fileName="target/logs/${ctx:suiteTimestamp}/${ctx:testName (${ctx:testStartTime}).log"

Suggestion: How about you divide conquer.

If such type of dynamic configuration is indeed supported. Why don't you try to configure first only the file name with a dynamic pattern?

You seem to be putting your log4j configuration on full steroids before you got the simplest possible configuration to work for your problem.

So put a feet on the break and focus on getting the: fileName="target/logs/dummyTest_dynamicComponent_${ctx:testName}.log"

To work for you.

In log4j 1.x version, you would have the log4j.debug system property to help you out figuring bugy configuration, and the output was very very useful.

Finally, on log4j 1.X version the feature you want to use would require you to program explicitly an MDC appender of your own. Your MDC appender would typically instantiate RollingFileAppenders to log into files and you would tap into the MDC context (keyxValue) pairs put by the user.

But what you're doing looks promising, just reduce the level of complexity of your configuration if it is not working for you.

Finally, I would be very surprised if you would see any log file geting created when you have the following error:

ERROR Unable to invoke factory method in class class org.apache.logging.log4j.core.appender.RollingFileAppender for element RollingFile. Unable to create Appender of type RollingFile.

Log4j is telling you: Hey, that appender you are defining. My factory that is trying to swallog this configuration cannot handle it and I will not instantiate a rolling file appender with this configuration.

So you have to fix that configuration.

Adding to answer.

Here you have a working Log4j 2 configuration for doing what you want:

First snippet if the log4j 2 configuration, where you will see that the root logger is given 3 different appenders to play around. You care mostly about appender 3, but the other two appenders are more of your typical starting point.

<?xml version="1.0" encoding="UTF-8"?>
<Configuration status="WARN">
    <Appenders>
        <!-- APPENDER 1: CONSOLE -->
        <Console name="Console" target="SYSTEM_OUT">
            <PatternLayout pattern="%d{HH:mm:ss.SSS} [%t] %-5level %logger{36} - %msg%n" />
        </Console>

        <!-- APPENDER 2: ROLLING FILE -->
        <RollingFile name="AppenderTwo" fileName="target/logs/test.log" filePattern="logs/$${date:yyyy-MM}/app-%d{MM-dd-yyyy}-%i.log.gz">
            <PatternLayout>
                <Pattern>%d %p %c{1.} [%t] %m%n</Pattern>
            </PatternLayout>
            <Policies>
                <TimeBasedTriggeringPolicy />
                <SizeBasedTriggeringPolicy size="10 MB" />
            </Policies>
        </RollingFile>

        <!-- APPENDER 3: ROUTING APPENDER -->
        <Routing name="AppenderThree">
            <Routes pattern="${ctx:stackOverFlow}">
                <!-- Route Nr.1 -->
                <Route>
                    <!-- Rolling file appender for route Nr.1 -->
                    <RollingFile name="NestedAppender-${ctx:stackOverFlow}" fileName="target/logs/test_threadContext_${ctx:stackOverFlow}.log"
                        filePattern="logs/$${date:yyyy-MM}/app-%d{MM-dd-yyyy}-%i.log.gz">
                        <PatternLayout>
                            <Pattern>%d %p %c{1.} [%t] %m%n</Pattern>
                        </PatternLayout>
                        <Policies>
                            <TimeBasedTriggeringPolicy />
                            <SizeBasedTriggeringPolicy size="10 MB" />
                        </Policies>
                    </RollingFile>
                </Route>

                <!-- Route Nr.2 fallback -->
                <!-- By having this set to ${ctx:filename} it will match when filename is not set in the context -->
                <Route ref="Console" key="${ctx:stackOverFlow}" />
            </Routes>
        </Routing>


    </Appenders>
    <Loggers>
        <Root level="all">
            <AppenderRef ref="Console" />
            <AppenderRef ref="AppenderTwo" />
            <AppenderRef ref="AppenderThree" />
        </Root>

    </Loggers>
</Configuration>

This last appender is configured based on the following thread: https://issues.apache.org/jira/browse/LOG4J2-129

The second snippet is a dummy junit test that you get out of eclipse when you create a new maven project out of a basic archetype. You will see in the test snippet that the stack over flow context context is being set in to the thread context, like you do in your snippets.

package stackoverflow.test.tutorial;

import org.apache.logging.log4j.ThreadContext;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

import junit.framework.Test;
import junit.framework.TestCase;
import junit.framework.TestSuite;

/**
 * Unit test for simple App.
 */
public class AppTest extends TestCase {
    private static final Logger LOGGER = LoggerFactory.getLogger(TestCase.class);

    /**
     * Create the test case
     *
     * @param testName
     *            name of the test case
     */
    public AppTest(String testName) {
        super(testName);
    }

    /**
     * @return the suite of tests being tested
     */
    public static Test suite() {
        return new TestSuite(AppTest.class);
    }

    /**
     * Rigourous Test :-)
     */
    public void testApp() {
        ThreadContext.put("stackOverFlow", "dummyContextValue");
        LOGGER.info("LALAL LLA");
        assertTrue(true);
    }
}

The last snippet are the maven dependencies:

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
  <modelVersion>4.0.0</modelVersion>

  <groupId>stackoverflow.test</groupId>
  <artifactId>tutorial</artifactId>
  <version>0.0.1-SNAPSHOT</version>
  <packaging>jar</packaging>

  <name>tutorial</name>
  <url>http://maven.apache.org</url>

  <properties>
    <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
  </properties>

  <dependencies>
   <dependency>
    <groupId>org.slf4j</groupId>
    <artifactId>slf4j-api</artifactId>
    <version>1.7.12</version>    
  </dependency>  
    <dependency>
    <groupId>org.apache.logging.log4j</groupId>
    <artifactId>log4j-api</artifactId>
    <version>2.5</version>
    <scope>runtime</scope>
  </dependency>
  <dependency>
    <groupId>org.apache.logging.log4j</groupId>
    <artifactId>log4j-core</artifactId>
    <version>2.5</version>
    <scope>runtime</scope>
  </dependency>
  <dependency>
    <groupId>org.apache.logging.log4j</groupId>
    <artifactId>log4j-slf4j-impl</artifactId>
    <version>2.5</version>
</dependency>

    <dependency>
      <groupId>junit</groupId>
      <artifactId>junit</artifactId>
      <version>3.8.1</version>
      <scope>test</scope>
    </dependency>
  </dependencies>
</project>

I find funy that log4j brings in this new Routing Appender. If you could imagine how many people have had to implement their own rolling file appenders with MDC context support to do this type of stuff. It is very useful in web apps.

Cheers.

Please upgrade the TestNG with latest version.

Then, change the version of the TestNG in project's xml file:

 <dependency>
        <groupId>org.testng</groupId>
        <artifactId>testng</artifactId>
        <version>6.1.1</version>
        <scope>compile</scope>
    </dependency>

to

<dependency>
        <groupId>org.testng</groupId>
        <artifactId>testng</artifactId>
        <version><!-- Updated Version Name --></version>
        <scope>compile</scope>
    </dependency>

Additionally, until this issue is fixed, if things still aren't working for you after following the previous advice OR you do not wish your code to be executed before every single method, then add the following code to your test class:

@Override
@BeforeSuite
protected void springTestContextPrepareTestInstance() throws Exception {
    super.springTestContextPrepareTestInstance();
}

This ensures that the Spring Context will be prepared before your @BeforeClass methods are executed.

*note, I posted this answer since in the title you're asking about @BeforeClass, even though there is no usage of @BeforeClass in your sample code.

Solved. Main problem was to override TestNG lifecycle + save maven test management. So I started to dig TestNG source code and came to next solution:

1. Extend IExecutionListener which will store needed data (queue) as a public static final member.
2. Same interface IExecutionListener participates in TestNG lifecycle: it has lifecycle callbacks onExecutionStart and onExecutionFinish. First one were used to populate queue with credentials from file, and second one is to ease GC work - clean this queue.
3. Populated queue was used inside of init method annotated with @BeforeMethod. Note: do not store value from queue as a plain member as you'll face overwriting of this member from different threads that performs tests from same class but different tests. Use ThreadLocal to provide thread isolation of this variable.
4. Special data structure that serves good for my case is BlockingQueue (in LinkedBlockingQueue implementation) with take and put as getter/setter.
5. Also do not forget to add your custom listener to your xml

If you're sure you really want to do this: There may be a better way, but this is all I could come up with...

JUnit4 has an annotation: @RunWith which lets you override the default Runner for your tests.

In your case you would want to create a special subclass of BlockJunit4ClassRunner, and override computeTestMethods() to return tests in the order you want them executed. For example, let's say I want to execute my tests in reverse alphabetical order:

public class OrderedRunner extends BlockJUnit4ClassRunner {

    public OrderedRunner(Class klass) throws InitializationError {
        super(klass);
    }

    @Override
    protected List computeTestMethods() {
        List list = super.computeTestMethods();
        List copy = new ArrayList(list);
        Collections.sort(copy, new Comparator() {
            public int compare(FrameworkMethod o1, FrameworkMethod o2) {
                return o2.getName().compareTo(o1.getName());
            }
        });
        return copy;
    }
}

@RunWith(OrderedRunner.class)
public class OrderOfTest {
    @Test public void testA() { System.out.println("A"); }
    @Test public void testC() { System.out.println("C"); }
    @Test public void testB() { System.out.println("B"); }
}

Running this test produces:

C
B
A

For your specific case, you would want a comparator that would sort the tests by name in the order you want them executed. (I would suggest defining the comparator using something like Google Guava's class Ordering.explicit("methodName1","methodName2").onResultOf(...); where onResultOf is provided a function that converts FrameworkMethod to its name... though obviously you are free to implement that any way you want.