How to use getLanguage method of org.testng.xml.XmlScript class

You seem to be almost exactly in the point where the place I worked at was when I started there 1,5 years ago, only difference being that you've started toying with branches which is actually something we still don't do at my work but more about that later on in this answer.

Anyway, you're listing a very good set of tools which can help a small company and those work really nicely as subtopics so without further ado,

Version control systems

Most commonly small companies currently use CVS or SVN and there's nothing bad in that, in fact I'd be really worried if no version control was really used at all. However you have to use version control right, just having one won't make your life easier. We currently use CVS and are looking into Mercurial, but we've found that the following works as a good set of conventions when working with CVS (and I'd suspect SVN too):

• Have separate users for all commiters. It's beyond valuable to know who commited what.
• Don't allow empty commit messages. In fact if possible, configure the repository to reject any commits without comments and/or default comment. Initial commit for FooBarizer is better than Empty log message
• Use tags to mark milestones, prototypes, alphas, betas, release candidates and final versions. Don't use tags for experimental work or as footnotes/Post-It notes.
• Don't use branches since they really don't work if you're continuing on developing the application. This is mainly because in CVS and SVN branching just doesn't work as expected and it becomes an exercise in futility to maintain any more than two living branches ( head and any secondary branch ) over time.

Always remember that for the software company the source code is your source of income and contains all your business value, so treat it that way. Also if you have extra 70 minutes, I really recommend that you watch through this talk Linus Thorvalds gave at Google about git and (d)VCS in general, it's really insightful.

Automated builds and Continuous Integration environments

These are about the same actually. Daily builds is a PR joke and has little no resemblance to the state of the actual software beyond some very rudimentary "Does it compile?" issues. You can compile a lot of awful code noise that doesn't do anything, keeping the software quality up has nothing to do with getting the code to compile.

On the other hand unit tests is a great way to maintain software quality and I can with a bit of personal pride say that rigorous unit testing helps even the worst of the programmers to improve a lot and catch stupid errors. In fact there has so far only been a total of three bugs that code I have written has reached production environments and I'd say that in 18 months that's a pretty damn good achievement. In our new production code we usually have a instruction code coverage of +80%, mostly +90% and in one special case reaching all the way to 98%. This part is very lively field and you're better of Googling for the following: TDD, BDD, unit tests, integration tests, acceptance tests, xUnit, mock objects.

That's a bit of a lengthy preface, I know. The actual meat for all the above is this: If you want to have automated builds, have them occur every time someone commits and make sure there's a constantly increasing and improving amount of unit tests for production code. Have the continuous integration system of your choice (we use Hudson CI) run all the unit tests related to project and only accept builds if all the tests pass. Do not make any compromises! If unit tests show that the software is broken, fix the software.

Additionally, Continuous Integration systems aren't just for compiling code but instead they should be used for tracking the state of the software project's metrics. For Hudson CI I can recommend all these plugins:

• Checkstyle - Checks if the actual source code is written in a way you define. Big part of writing maintainable code is to use common conventions.
• Cobertura - Code coverage metrics, very useful to see how the coverage develops over time. Also keeping in line with the "source is God" mentality, allows you to discard builds if coverage falls below a certain level.
• Task Scanner - Simple but sweet: Scans for specific tags such as BUG, TODO, NOTE etc. in your code and creates a list from them for everyone to read. Simple way to track short notes or known bugs which needs fixing or whatever you can come up with.

Project structure and Dependency Management

This is a controversial one. Basically everyone agrees that having an unified structure is great but since there's several camps with different requirements, habits and views to issue they tend to disagree. For example Maven people really believe that there's only one way - the Maven way - to do things and that's it while Ivy supporters believe that the project structure shouldn't be hammered down your throat by external parties, only the dependencies need to be managed properly and in an unified manner. Just that it's not left unclear, our company simply loves Ivy.

So since we don't use project structure imposed by external parties, I'm going to tell you a bit about how we got into what we got into our current project structure.

In the beginning we used individual projects for actual software and related tests (usually named Product and Product_TEST). This is very close to what you have, one huge directory for everything with the dependencies as JARs directly included in the directory. What we did was that we checked out both projects from CVS and then linked the actual project to the test software project in Eclipse as runtime dependency. A bit clunky but it worked.

We soon came to realize that these extra steps are completely useless since by using Ant - by the way, you can invoke Ant tasks directly in Hudson - we could tell the JAR/WAR building step to ignore everything by either file name (say, everything that ends with Test or TestCase) or by source folder. Pretty soon we converted our software project to use a simple structure two root folders, src and test. We haven't looked back ever since. The only debate we currently have is if we should allow for a third folder called spikes to exist in our standard project structure and that's not a very heated debate at all.

This has worked tremendously well and doesn't require any additional support or plugins from any of IDEs out there which is a great plus - number two reason we didn't choose Maven was seeing how M2Eclipse basically took over Eclipse. And since you must be wondering, number one reason for rejecting Maven was the clunkiness of Maven itself, endless amount of lengthy XML declarations for configuration and the related learning curve was considered a too big cost as to what we would get from using it.

Rather interestingly later on commiting to Ivy instead of Maven has allowed us to a smooth shift to do some Grails development which uses folder and class names as conventions for just about everything when structuring the web application.

Also a final note about Maven, while it claims to promote convention over configuration, if you don't want to do things exactly the way the Maven's structure says you should do things, you're in a world of pain for the aforementioned reasons. Certainly that's an expected side effect of having conventions but no convention shouldn't be final, there always has to be at least some room for changes, bending the rules or choosing the appropriate from a certain set.

In short, my opinion is that Maven is a bazooka, you work in a house and you ultimate goal is to have it bug free. Each of these are good on it's own and work even if you pick any two of them, but the three together just doesn't work.

Final words

As long as you have less than 10 code-centric people, you have all the flexibility needed to do the important decisions. When you go beyond that, you have to live with whatever choices you've made, no matter how good or bad they are. Don't just believe things you hear on the Internet, sit down and test everything rigorously - heck, our senior tech guy even wrote his bachelor's thesis about Java web frameworks just to figure out which one we should use - and really figure out what you really need. Don't commit to anything just because you may need some of the functionality it provides in distant future, pick those things that has the lowest possible negative impact to the whole company. Being the 10th person hired to the company I work at I can undersign everything in this paragraph with my own blood, we currently have 16+ people working and changing certain conventions would actually be a bit scary at this point.

The only difference is related to programming style.

Commenting source code is commonly recognized as violation. Commented code should be removed from source code.

There is violation Avoid commented-out lines of code in Sonarqube with nice description

Here are the main reasons why commented code is a code smell :

• It always raises more questions than it gives answers
• Everybody will forget very quickly how relevant the commented code is
• This is distraction when going down the code as it stops the flow of eyes
• It is a bad SCM engine : Subversion, CVS and Git are really more trustworthy !
• The simple fact of understanding why code was commented out in the first place can take a lot of time

Example real case: Code cleanup: remove commented out code

Is driver2 is initialized? I don't think so.

For example if your testing against Safari browser, then you need to do something like this:

WebDriver driver = new SafariDriver()

And then use this object for your automation.

did you explicitly specify in build.gradle that you use testng?

test {
    useTestNG()
}

You're missing the annotation @PrepareForTest and the use of mockStatic() which are required when you wish to mock a final static class with PowerMockito. I think where you might be getting confused is that you're dealing with a final static class and not only a final class, so there are a few additional mock calls you need to be aware of.

Given these implementation classes

/*
The final static class that's giving you all the mocking grief
*/
public final class FinalStaticClass {

    // this object instance "will not" be returned when we mock() the class
    private static MyObject obj = new MyObject();

    public static MyObject getMyObject() {
        return obj; 
    }
}

/*
A simple value object used by MyClass
*/
public class MyObject {

}

/*
The class you wish to test
*/
public class MyClass {

    private MyObject obj;

    public void methodToTest() {
        obj = FinalStaticClass.getMyObject();
        // do something
    }

    public MyObject getMyObject() {
        return obj;
    }
}

Make sure you include powermock-mockito-1.4.10-full.jar in your project, then you can use this test class

import static org.junit.Assert.assertEquals;
import static org.powermock.api.mockito.PowerMockito.mockStatic;
import static org.powermock.api.mockito.PowerMockito.when;

import ...    

@RunWith(PowerMockRunner.class)
@PrepareForTest(FinalStaticClass.class)
public class MyClassTest{

    @Mock
    MyObject expectedObject;

    @InjectMocks
    MyClass myClass = new MyClass();

    @Test
    public void test1(){

        // mock all static methods
        mockStatic(FinalStaticClass.class);

        when(FinalStaticClass.getMyObject()).thenReturn(expectedObject);

        // execute the method under test
        myClass.methodToTest();

        assertEquals(expectedObject, myClass.getMyObject());

    }
}

You can see that expectedObject is a mock that you have created in the test and not the implementation returned from FinalStaticClass

Try using this code:

ListenerApadter:

public class MyTestListenerAdapter extends TestListenerAdapter {
    @Override
    public void onTestFailure(ITestResult result) {
        if (result.getMethod().getRetryAnalyzer() != null) {
            MyRetryAnalyzer retryAnalyzer = (MyRetryAnalyzer)result.getMethod().getRetryAnalyzer();

            if(retryAnalyzer.isRetryAvailable()) {
                result.setStatus(ITestResult.SKIP);
            } else {
                result.setStatus(ITestResult.FAILURE);
            }
            Reporter.setCurrentTestResult(result);
        }
    }

   @Overrride
   public void onFinish(ITestContext context) {
     Iterator<ITestResult> failedTestCases =context.getFailedTests().getAllResults().iterator();
    while (failedTestCases.hasNext()) {
        System.out.println("failedTestCases");
        ITestResult failedTestCase = failedTestCases.next();
        ITestNGMethod method = failedTestCase.getMethod();
        if (context.getFailedTests().getResults(method).size() > 1) {
            System.out.println("failed test case remove as dup:" + failedTestCase.getTestClass().toString());
            failedTestCases.remove();
        } else {

            if (context.getPassedTests().getResults(method).size() > 0) {
                System.out.println("failed test case remove as pass retry:" + failedTestCase.getTestClass().toString());
                failedTestCases.remove();
            }
        }
    }
   }
}

RetryAnalizer:

public class MyRetryAnalyzer implements IRetryAnalyzer {
    private static int MAX_RETRY_COUNT = 3;

    AtomicInteger count = new AtomicInteger(MAX_RETRY_COUNT);

    public boolean isRetryAvailable() {
        return (count.intValue() > 0);
    }

    @Override
    public boolean retry(ITestResult result) {
        boolean retry = false;
        if (isRetryAvailable()) {
            System.out.println("Going to retry test case: " + result.getMethod() + ", " + (MAX_RETRY_COUNT - count.intValue() + 1) + " out of " + MAX_RETRY_COUNT);
            retry = true;
            count.decrementAndGet();
        }
        return retry;
    }
}

POM.xml -> Surefire Configuration:

This is where you should configure "overwrite" surefire listener wich has their own counters.

<plugin>
  <groupId>org.apache.maven.plugins</groupId>
  <artifactId>maven-surefire-plugin</artifactId>
  <version>2.18.1</version>
  <configuration>
    <suiteXmlFiles><suiteXmlFile>${basedir}/testng.xml</suiteXmlFile></suiteXmlFiles>
 <properties> 
   <property>
    <name>listener</name>
    <value>Utils.MyTestListenerAdapter,Utils.MyRetryAnalizer</value>
   </property>
 </properties>

This is actually by design in TestNG. I see two potential options:

First, look at using a DataProvider with your tests so they can be executed repeatedly with different parameters.

Or, alter the XML layout so that each method is part of its own tag. In this configuration, the XML will be longer, but you'll be able to invoke a test method as many times as you like as part of a suite.

EDIT: More details now.

In response to your comment, here's a sample XML layout that worked for me when I was up against this same problem. Try this:

<suite name="WebServiceTestSuite">
<test name="webServiceTest">
    <classes>
        <class name="com.abcco.webservice.DivisionTest">
            <methods>
                <include name="registrationCreateTest"></include>
            </methods>
        </class>
    </classes>
</test>
<test name="webServiceTest">
    <classes>
        <class name="com.abcco.webservice.DivisionTest">
            <methods>
                <include name="registrationActivateTest"></include>
            </methods>
        </class>
    </classes>
</test>
<test name="webServiceTest">
    <classes>
        <class name="com.abcco.webservice.DivisionTest">
            <methods>
                <include name="appUserLoginTest"></include>
            </methods>
        </class>
    </classes>
</test>
<test name="webServiceTest">
    <classes>
        <class name="com.abcco.webservice.DivisionTest">
            <methods>
                <include name="appUserLogoutTest"></include>
            </methods>
        </class>
    </classes>
</test>
<test name="webServiceTest">
    <classes>
        <class name="com.abcco.webservice.DivisionTest">
            <methods>
                <include name="appUserLoginTest"></include>
            </methods>
        </class>
    </classes>
</test>
<test name="webServiceTest">
    <classes>
        <class name="com.abcco.webservice.DivisionTest">
            <methods>
                <include name="sessionValidateTest"></include>
            </methods>
        </class>
    </classes>
</test>
<test name="webServiceTest">
    <classes>
        <class name="com.abcco.webservice.DivisionTest">
            <methods>
                <include name="sessionExtendTest"></include>
            </methods>
        </class>
    </classes>
</test>
</suite>

I removed the verbose="10" parallel="none" from the <test> tags, as the layout that worked for me didn't have this... You may be able to add the verbose="10" to the <suite> tag to get any specific functionality there that you need. Have a go with this and post your results and I'll be happy to help further. Without seeing your runner configuration or knowing too much about the rest of your setup, there may be some other things to consider.

You'll find that this is a much more verbose XML layout, but it solved this same problem for me at the time. You could also consider re-arranging your code a bit and making an @Test method that calls all the methods on this XML as a single test, allowing you to reduce the length of your XML file.

TestNG doesn't provide this OOB. By default it generates a report in html.

If you want to write a report on the console, you need write a customs TestListenerAdapter.

There's a small example of how to do something similar in the TestNG logging documentation. You can use the same example, and override onTestFailure(), read the result of the test, and get the exception message from ITestResult.getThrowable().

From the Mockito documentation :

Resetting mocks (Since 1.8.0)

Smart Mockito users hardly use this feature because they know it could be a sign of poor tests. Normally, you don't need to reset your mocks, just create new mocks for each test method. Instead of reset() please consider writing simple, small and focused test methods over lengthy, over-specified tests. First potential code smell is reset() in the middle of the test method. This probably means you're testing too much. Follow the whisper of your test methods: "Please keep us small & focused on single behavior". There are several threads about it on mockito mailing list.

The only reason we added reset() method is to make it possible to work with container-injected mocks.

You should probably just recreate the spy in a @BeforeMethod :

public class Program {

    private TestObject testObject = new TestObject();
    private TestObject spyTestObject;

    @BeforeMethod
    public void buildSpy() {
        spyTestObject = spy(testObject);
    }

    @Test
    public void test1() {
       spyTestObject.makeSth(); 
       verify(spyTestObject , only()).someMethodNeedToBeChecked(); 
    }

    @Test
    public void test2() {
        // Some different scenario
       spyTestObject.makeSth(); 
       verify(spyTestObject , only()).someMethodNeedToBeChecked(); 
        ...
    }
}