How to use getAfterTestConfigurationMethods method of org.testng.Interface ITestMethodFinder class

There are many open points on this question which are not clear due to lack of details.

According to your description the project structure is hence as following:

project
  |
  |--- test-module
  |--- core-module
  |--- driver-module (depends on `core` and `test`)
  |
   \ pom.xml

You actually execute tests only via the driver module and via the exec-maven-plugin, however the allure documentation specifies the classic approach, to execute the test phase, that is, the Maven phase dedicated to tests execution (via the maven-surefire-plugin, automatically invoked by Maven during this phase via default bindings).

You actually configure the maven-surefire-plugin as specified by its documentation: specifying it into the parent pom.xml file would be fine in this case, in its pluginManagement section > Maven will pick up its global configuration during the default-test execution of the default maven-surefire-plugin binding.

However, the whole mechanism is linked to the test phase. Do you execute it?

You did not provide details about the exec-maven-plugin and what it is supposed to do concerning the driver module and why you would use the jetty-maven-plugin to see the report. Normally, test reports are available and can be seen direct html files, no need to host them in an embedded jetty server, unless required by the overall process (CI, deployment to a company server, etc.). These reports should also be made available to the project documentation site, which can be generated via the site lifecycle and its maven-site-plugin.

From your details, you share the jetty-maven-plugin configuration pointing at a site folder: is this folder generated during the test phase (if you invoke it)? Or during your exec invocation?

Concerning the way you use dependencies in the parent pom, you actually did not share it in your question, so not easy to help. Normally, you would put dependencies in the dependencies section of the parent pom which are common to all modules (e.g. classic example is log4j, something every module would use). Otherwise, you would use the dependencyManagement section of the parent pom to manage versions of certain dependencies which could be use by one or more modules (but modules would need to re-declare them to effectively use them, omitting their versions though, specified by the parent). That is, the parent pom is a central place of governance and harmonisation.

Update

Concerning the /target/allure-results folder generated, you need also to check its content to be a valid site directory (that is, e.g. it should contain an index.html).

Creating a new Maven webapp project and adding the following to its pom.xml file:

<build>
    <plugins>
        <plugin>
            <groupId>org.eclipse.jetty</groupId>
            <artifactId>jetty-maven-plugin</artifactId>
            <version>9.2.10.v20150310</version>
            <configuration>
                <webAppSourceDirectory>${project.build.directory}/site</webAppSourceDirectory>
                <stopKey>stop</stopKey>
                <stopPort>1234</stopPort>
            </configuration>
        </plugin>
    </plugins>
</build>

(Note: exactly as per your question, but pointing simply to the site directory)

And executing:

mvn clean install site
mvn jetty:run

the Maven site (generated during the site phase as invoked above) would be available at localhost:8080 (default jetty URL). That's because an index.html was generated.

However, if I manually delete the index.html, the jetty will show an Directory: / page, listing the available files.

Hence, most probably, the allure report didn't generate an index.html file, because most probably it is not meant to generate it, but simply an HTML report generared during the test phase.

Based on your last paragraph, what you're trying to do is inject some 'extra methods' into unit testing since you're already testing a specific class. I do not know of a testing harness that allows you to attach tests based on the hierarchy of a class.

However, with your own suggestion of using TestNG for building something similar, I think you might be very close. You could very well incorporate some base code that adds your class to a list of 'default test classes', which are in turn tested if they implement a specific interface.

Still, regarding the general case, I think you're out of luck, since the Java type system is one-way, you can only (easily) find out what interfaces a class implements, not the other way around. Furthermore, the problem is 'where to stop looking': if you have a test that checks all your comparable implementers, do you want it to check the validity of String's one too, since that is in your Java environment?

I would log your results to the db of your choice and then create a mobile friendly webpage that queries the db and summarizes the run.

If that's too much work, can you not just post your HTML file to a webserver that you have access to from your mobile device?

1. should i use maven-surefire-plugin and do a heavy cleanup in the test package?
2. is it any way to tell hudson (not in pom.xml of the project being built) to run specific unit tests and ignore others?

If you just want to run unit tests in a single-module project, you can do

mvn test

That will run all maven lifecycle phases up to test, including compile and test-compile (as you can see in built-in lifecycle bindings, the surefire:test goal is executed in phase test). Now if you want to restrict the unit tests that are executed, you can configure the surefire:test plugin execution with the test parameter:

mvn test -Dtest=Funky*Test

(This will execute all test classes that start with Funky and end with Test)

Unfortunately, the command line configuration is limited, if you want multiple includes and excludes you will have to do some XML configuration. I'd suggest to use a dedicated profile for hudson:

<profiles>
    <profile>
        <id>hudson</id>
        <build>
            <plugins>
                <plugin>
                    <groupId>org.apache.maven.plugins</groupId>
                    <artifactId>maven-surefire-plugin</artifactId>
                    <version>2.6</version>
                    <configuration>
                        <includes>
                            <include>**/TestA.java</include>
                            <include>**/Funky*Test.java</include>
                            <include>**/Test**.java</include>
                        </includes>
                        <excludes>
                            <!-- overrides above include -->
                            <exclude>**/TestButNotMe.java</exclude>
                        </excludes>
                    </configuration>
                </plugin>
            </plugins>
        </build>
    </profile>
</profiles>

Now you can let your hudson call maven like this:

mvn -Phudson test

No JUnit does not support test ordering, except in the manner that you say, through Suite. This only defines the order in which the test classes are executed. This has been there for a long time, including JUnit 3 Suite classes.

For a more complete explanation, there are three things we need to talk about here:

1. the ordering of tests classes within a test suite
2. the ordering of test classes when they are found by reflection by Eclipse or Maven
3. the ordering of test methods (annotated with @Test) within a test class.

The ordering of tests classes within a test suite

When you specify the list of classes to be executed in a test suite, you're defining an array, and these test classes will be executed in order, except when you're doing parallel execution. Unfortunately, this allows the introduction of dependencies between your test classes.

The ordering of test classes when they are found by reflection

When searching the classpath for classes, the order in which they are found is not guaranteed, so cannot be depended upon. It isn't actually JUnit which is doing the searching, but the Eclipse Junit plugin, or maven surefire or failsafe.

The ordering of test methods within a test class

JUnit does not guarantee the order of execution of tests within a class. Most of the time, on most JVMs pre version 7, the order in which they are found using reflection is in declaration order, ie the order they are in the file. This is the order in which they are executed. However, with JVM 7, this is no longer guaranteed, so there won't be a consistent order. There is a github issue #293 Sort test methods for predictability open with suggested solutions, and there is a thread on the junit mailing list: Alphabetizing test method run order?. So you cannot depend upon the order that tests will be executed using JUnit, but this is currently under discussion.

I just had this problem and I'm pretty confident that it's a bug in the TestNG Plugin for eclipse, since I was able to run my tests from the command line just fine.

The fix that seems to be working for me and others is just to clean your project (Project → Clean). One person had to re-install the whole plugin.

Put the method in a group and use dependsOnGroups.

class c1 {
  @Test(groups={"c1.verifyConfig"})
  public void verifyConfig() {
    //verify some test config parameters
  }
}

class c2 {
  @Test(dependsOnGroups={"c1.verifyConfig"})
  public void dotest() {
    //Actual test
  }
}

It is recommended to verify configuration in a @Before* and throw if something goes wrong there so the tests won't run. This way the tests can focus on just testing.

class c2 {
  @BeforeClass
  public static void verifyConfig() {
    //verify some test config parameters
    //Usually just throw exceptions
    //Assert statements will work
  }

  @Test
  public void dotest() {
    //Actual test
  }
}

public class TestCase{
   public WebDriver driver;

   @BeforeTest
   public void setUp(){
       driver = new ChromeDriver();
   }

Here you have declared a driver and initialized.

public class WikiMain extends WebPage{

private WebDriver driver;

public WikiMain(WebDriver driver){
    super(driver);
}

Another driver without any initialaztion. So when you ran here

 WikiMainEn wikiMainEn = (WikiMainEn) wikiMain.getWikiEn();

You were using a uninitialized driver. Because you've never initilize the subclass's(WikiMain) driver. You just initilized WebPage.class 's driver.

In my opinion you can set the WebPage's driver protected and remove the subclass one.

 protected WebDriver driver;

And remove

public class WikiMain extends WebPage{

 private WebDriver driver; //remove it

There is no method on FileReader/FileWriter to tell you what file it points to (because theoretically you should never need to know - that's why you're working with the reader and not the File).

You could use reflection to read private fields of the underlying FileInputStream/FileOutputStream.

But I think the correct approach to this is don't test things that cannot break - namely, don't test the Java core libraries. Unless the code posted is a simplified version, testInitiateFileWriter() is effectively testing new FileWriter (getFile(path),false), and you already know that getFile(path) works (as per your passing test), so the only thing you're testing here is new FileWriter(...).

Footnote: Of course Java libraries can break / contain bugs, but the chance is negligible in most cases and for such widely-used part as FileReader/FileWriter.