How to use testSuiteFinished method of net.serenitybdd.junit.runners.FailureDetectingStepListener class

I figured out how it actually works. All you have to do is add Surefire plugin with Failsafe plugin when you use both of them. It works like a charm.

<build>
    <plugins>
        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-surefire-plugin</artifactId>
            <version>2.22.1</version>
            <configuration>
                <skip>false</skip>
            </configuration>
        </plugin>
        <plugin>
            <artifactId>maven-failsafe-plugin</artifactId>
            <version>2.22.1</version>
            <configuration>
                <includes>
                    <!-- Run every java class in the 'tests' package -->
                    <include>${test.directory}/*.java</include>
                </includes>

            </configuration>
            <executions>
                <execution>
                    <goals>
                        <goal>integration-test</goal>
                        <goal>verify</goal>
                    </goals>
                </execution>
            </executions>
        </plugin>
        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-compiler-plugin</artifactId>
            <version>3.7.0</version>
            <configuration>
                <source>1.8</source>
                <target>1.8</target>
            </configuration>
        </plugin>
        <plugin>
            <groupId>net.serenity-bdd.maven.plugins</groupId>
            <artifactId>serenity-maven-plugin</artifactId>
            <version>${serenity.maven.version}</version>
            <configuration>
                <tags>${tags}</tags>
            </configuration>
            <executions>
                <execution>
                    <id>serenity-reports</id>enter code here
                    <phase>post-integration-test</phase>
                    <goals>
                        <goal>aggregate</goal>
                    </goals>
                </execution>
            </executions>
        </plugin>
    </plugins>
</build>

Looks like you are indirectly running Serenity through the junit-vintage-engine. When executed this way, parallel execution is simply not supported.

To be exact you are running your tests through Surefire -> JUnit 5 -> JUnit Vintage -> JUnit 4 -> Serenity -> Cucumber.

You'd have to wait for JUnit Vintage to support parallel execution or get rid of JUnit 5 from your depencies so Surefire can run JUnit 4 directly.

However given that you are also using Spring you may not be able to get rid of JUnit 5.

You could ask Serenity to support JUnit 5 but I suspect they will be unable too. They have never bothered to contribute the extension points they'd need upstream. Instead they have hacked their stuff into Cucumber.

You're using the wrong value for parallel. You have to set it to methods or both. Otherwise Surefire will run all tests of your runner class serially.

https://github.com/cucumber/cucumber-jvm/tree/main/cucumber-junit

Cucumber JUnit supports parallel execution of feature files across multiple threads. To enable this with maven set the parallel property to either methods or both.

<build>
    <plugins>
        <plugin>
            <artifactId>maven-surefire-plugin</artifactId>
            <!-- Use 2.22.1 or higher -->
            <version>${maven-surefire-plugin.version}</version>  
            <configuration>
                <parallel>both</parallel>
                <threadCount>4</threadCount>
            </configuration>
        </plugin>
    </plugins>
</build>

You can have your own runner in serenity and instantiate driver with your desired capabilities for it. By doing so you can configure and redirect logs.

@RunWith(CucumberWithSerenity.class)
@CucumberOptions(
        plugin = {"pretty"},
        features = "<feature file path>",
        glue = {"<steps path as comma separated values"}
)
public class MyRunner {

       }

Just confirm first that your using supported surefire and junit version. For more details refer https://maven.apache.org/surefire/maven-surefire-plugin/examples/single-test.html

In case your using maven failsafe plugin then the syntax will vary little bit. Something like this

mvn -Dit.test=ITCircle#test* verify

Refer https://maven.apache.org/surefire/maven-failsafe-plugin/examples/single-test.html for more details.

Looks like you are indirectly running Serenity through the junit-vintage-engine. When executed this way, parallel execution is simply not supported.

To be exact you are running your tests through Surefire -> JUnit 5 -> JUnit Vintage -> JUnit 4 -> Serenity -> Cucumber.

You'd have to wait for JUnit Vintage to support parallel execution or get rid of JUnit 5 from your depencies so Surefire can run JUnit 4 directly.

However given that you are also using Spring you may not be able to get rid of JUnit 5.

You could ask Serenity to support JUnit 5 but I suspect they will be unable too. They have never bothered to contribute the extension points they'd need upstream. Instead they have hacked their stuff into Cucumber.

I believe that this should be possible using the custom environments option when scheduling a run in Device Farm. Admittedly, I don't have experience with serenity but here is blog which uses cucumber tests.

https://aws.amazon.com/blogs/mobile/testing-mobile-apps-with-cucumber-and-appium-through-testng-on-aws-device-farm/

It looks like serenity is similar or extended from cucumber so in theory this might work.

[Edit]

Here are a few short gradle build tasks which should help to create a test package for Device Farm.

Note: this assumes that the dependencies are structured in the build.gradle file like this:

dependencies {

    testCompile(
            'net.serenity-bdd:serenity-junit:2.0.18',
            'net.serenity-bdd:serenity-cucumber:1.9.20',
            'org.assertj:assertj-core:3.11.1',
            'ch.qos.logback:logback-classic:1.2.3',
            'io.github.bonigarcia:webdrivermanager:3.0.0'
    )
}

sample build.gradle code

//source: https://stackoverflow.com/a/27455099/8016330
task getDeps(type: Copy) {
    from sourceSets.test.runtimeClasspath
    // if you need this from the dependencies in the build.gradle then it should be : 
    // from sourceSets.main.runtimeClasspath
    into 'build/libs/dependency-jars'
}

//packaging tests task which is generated from sample using gradle init
task packageTests(type: Jar) {
    dependsOn getDeps
    from sourceSets.test.output
    classifier = 'tests'
}

//create zip archive
//source: https://stackoverflow.com/a/36883221/8016330
task zip(type: Zip) {
    dependsOn packageTests
    from 'build/libs/'
    include '*'
    include '*/*' //to include contents of a folder present inside dependency-jars directory
    archiveName 'zip-with-dependencies.zip'
    destinationDir(file('build/libs/'))
}

Then you should be able to create the zip file using this command:

./gradlew clean zip

Note: You'll need the clean command otherwise it will package the previous zip archive in the new zip archive.

Hth

-James

Find the solution for parallel execution with cucumber 7 and serenity junit 4. Below pom.xml is tested and working to execute parallel. According to this pom.xml to execute two ways:

1. locally - add to your runner in tags tag you want to run and then execute this command mvn clean verify

2. mvn clean verify -Dtags="@yourtagHere"

Working pom.xml:

<?xml version="1.0" encoding="UTF-8"?>

4.0.0

<groupId>com.d</groupId>
<artifactId>Automation</artifactId>
<version>1.0-SNAPSHOT</version>

<properties>
    <serenity.version>3.2.0</serenity.version>
    <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
    <encoding>UTF-8</encoding>
    <tags></tags>
    <parallel.tests>5</parallel.tests>
    <webdriver.base.url></webdriver.base.url>
    <maven.compiler.source>11</maven.compiler.source>
    <maven.compiler.target>11</maven.compiler.target>
    <java.version>11</java.version>
</properties>
<dependencies>
    <dependency>
        <groupId>net.serenity-bdd</groupId>
        <artifactId>serenity-rest-assured</artifactId>
        <version>${serenity.version}</version>
    </dependency>
    <dependency>
        <groupId>net.serenity-bdd</groupId>
        <artifactId>serenity-cucumber</artifactId>
        <version>${serenity.version}</version>
    </dependency>
    <dependency>
        <groupId>net.serenity-bdd</groupId>
        <artifactId>serenity-junit</artifactId>
        <version>${serenity.version}</version>
    </dependency>
    <dependency>
        <groupId>com.googlecode.lambdaj</groupId>
        <artifactId>lambdaj</artifactId>
        <version>2.3.3</version>
    </dependency>
    <dependency>
        <groupId>com.fasterxml.jackson.core</groupId>
        <artifactId>jackson-databind</artifactId>
        <version>2.12.4</version>
    </dependency>
    <dependency>
        <groupId>javax.ws.rs</groupId>
        <artifactId>javax.ws.rs-api</artifactId>
        <version>2.1.1</version>
    </dependency>
    <dependency>
        <groupId>org.codehaus.jettison</groupId>
        <artifactId>jettison</artifactId>
        <version>1.4.1</version>
    </dependency>
    <dependency>
        <groupId>commons-dbutils</groupId>
        <artifactId>commons-dbutils</artifactId>
        <version>1.7</version>
    </dependency>
    <dependency>
        <groupId>com.microsoft.sqlserver</groupId>
        <artifactId>mssql-jdbc</artifactId>
        <version>9.4.0.jre11</version>
    </dependency>
    <dependency>
        <groupId>com.jayway.jsonpath</groupId>
        <artifactId>json-path</artifactId>
        <version>2.6.0</version>
    </dependency>
    <dependency>
        <groupId>com.github.javafaker</groupId>
        <artifactId>javafaker</artifactId>
        <version>1.0.2</version>
    </dependency>
    <dependency>
        <groupId>org.awaitility</groupId>
        <artifactId>awaitility</artifactId>
        <version>4.1.1</version>
    </dependency>
</dependencies>

<build>
    <plugins>
        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-surefire-plugin</artifactId>
            <version>3.0.0-M5</version>
            <configuration>
                <skip>true</skip>
            </configuration>
        </plugin>
        <plugin>
            <artifactId>maven-failsafe-plugin</artifactId>
            <version>3.0.0-M5</version>
            <configuration>
                <includes>
                    <include>**/*Runner.java</include>
                </includes>
                <systemPropertyVariables>
                    <webdriver.base.url>${webdriver.base.url}</webdriver.base.url>
                </systemPropertyVariables>
                <parallel>methods</parallel>
                <threadCount>${parallel.tests}</threadCount>
                <!-- <forkCount>${parallel.tests}</forkCount>-->
                <!--<useUnlimitedThreads>true</useUnlimitedThreads>-->
            </configuration>
            <executions>
                <execution>
                    <goals>
                        <goal>integration-test</goal>
                        <goal>verify</goal>
                    </goals>
                </execution>
            </executions>
        </plugin>
        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-compiler-plugin</artifactId>
            <version>3.10.0</version>
            <configuration>
                <source>11</source>
                <target>11</target>
                <compilerArgs>
                    <arg>-parameters</arg>
                </compilerArgs>
            </configuration>
        </plugin>
        <plugin>
            <groupId>net.serenity-bdd.maven.plugins</groupId>
            <artifactId>serenity-maven-plugin</artifactId>
            <version>${serenity.version}</version>
            <configuration>
                <tags>${tags}</tags>
                <reports>single-page-html</reports>
            </configuration>
            <executions>
                <execution>
                    <id>serenity-reports</id>
                    <phase>post-integration-test</phase>
                    <goals>
                        <goal>aggregate</goal>
                    </goals>
                </execution>
            </executions>
            <dependencies>
                <dependency>
                    <groupId>net.serenity-bdd</groupId>
                    <artifactId>serenity-single-page-report</artifactId>
                    <version>${serenity.version}</version>
                </dependency>
            </dependencies>
        </plugin>
    </plugins>
</build>

Enter the following property in serenity.properties file

webdriver.base.url = http://en.wiktionary.org

In your page object define the DefaultUrl like this

@DefaultUrl("/wiki/Wiktionary")
public class DictionaryPage extends PageObject {


}

Serenity will construct the complete URL while calling DefaultUrl.

For further reference on all the properties available in serenity.properties please refer link below https://github.com/serenity-bdd/serenity-documentation/blob/master/src/asciidoc/system-props.adoc