10

I have a Cucumber-JVM, JUnit, Selenium setup. I initiate the run by running RunSmokeTests.java using JUnit within Eclipse. I have also set up a maven profile to run the tests from command line, and possibly Jenkins in the future.

When the tests are run then some of them may fail sometimes, mainly due to the application taking longer than expected. I would then have to re-run these scenarios. At the moment I run them by manually attaching @rerun tag to the ones that failed and then running RunReruns.java, which is similar to RunSmokeTest.java but with @rerun tag.

With the increasing number of automated tests it is time consuming to tag the tests and start the run and clear the tags. Is there a automated way with Cucumber-JVM to re-run failed tests?

RunSmokeTests.java

package testGlueClasses;
import cucumber.api.junit.Cucumber;
import org.junit.runner.RunWith;

@RunWith(Cucumber.class)
@Cucumber.Options(features = "src/test/java", strict = true, format = {
        "html:target/CucumberReport", "json:target/JSON/Cucumber.json",
        "FrameworkCore.CustomTestReporter" }, tags = { "@SmokeTest" }, glue = {
        "FrameworkCore", "MyApp.Utils", "MyApp.StepDefinitions" })
public class RunSmokeTests {

}

Maven snippet: 

    <profile>
        <id>smoke</id>
        <properties>
            <include.tests>
                **/RunSmokeTests.java
            </include.tests>
        </properties>
    </profile>
4

6 回答 6

6

我想出了另一个解决方案，使用 maven & cucumber 重新运行刚刚失败的测试。

1) 使用记录测试失败RunNotifier

public class RerunningCucumber extends Cucumber {

    private final String className;

    @SuppressWarnings("rawtypes")
    public RerunningCucumber(Class clazz) throws InitializationError, IOException {
        super(clazz);
        className = clazz.getSimpleName();
    }


    @Override
    public void run(RunNotifier notifier) {
        notifier.addListener(new RunListener(){

            public void testFailure(Failure failure) throws Exception {

                Throwable error = failure.getException();
                if (error instanceof AssertionError){
                    //Nothing. This is a normal failure. Continue
                    return;
                }

                //No! A wild exception has appeared!
                //Let's run this test again.
                RerunningCucumber.addFile(className);
            }

        });
        super.run(notifier);
    }


    private static final String filename = "target/rerun.properties";
    private static final Set<String> addedClasses = new HashSet<String>();
    public static synchronized void addFile(String className) throws IOException{
        //First find the file

        if (addedClasses.contains(className)){
            return;
        }

        File file = new File(filename);
        if (!file.exists()){
            //Need to create the file
            PrintWriter writer = new PrintWriter(file, "UTF-8");
            writer.print("retryclasses=**/"+className+".class");
            writer.close();
        }
        else {
            PrintWriter out = new PrintWriter(new BufferedWriter(new FileWriter(file, true)));
            out.print(",**/"+className+".class");
            out.close();
        }

        addedClasses.add(className);
    }
}

2）使用自定义类作为黄瓜测试的跑步者。

这将运行测试，并且每当出现故障时，将失败的类输出到文件中。诀窍是保持功能简短并创建大量测试类以避免重复测试。

@RunWith(RerunningCucumber.class)
@CucumberOptions(features = {"classpath:features/testFeature.feature}, format = {
        "html:target/cucumber-html-report/testFeature.html",
        "json:target/cucumber-json-report/testFeature.json"},
        tags = {"@testFeature"})

public class RunTestFeature {
}

3）将Rerun配置文件添加到maven。

这做了三件事：1）它将失败的类加载到内存中，2）只清理失败的类属性文件，3）只重新运行从属性文件加载的失败的测试：

    <profile>
        <id>retry</id>
        <build>
            <plugins>
                <plugin>
                    <groupId>org.codehaus.mojo</groupId>
                    <artifactId>properties-maven-plugin</artifactId>
                    <version>1.0-alpha-2</version>
                    <executions>
                        <!-- Associate the read-project-properties goal with the initialize 
                            phase, to read the properties file. -->
                        <execution>
                            <phase>pre-clean</phase>
                            <goals>
                                <goal>read-project-properties</goal>
                            </goals>
                            <configuration>
                                <files>
                                    <file>target/rerun.properties</file>
                                </files>
                            </configuration>
                        </execution>
                    </executions>
                </plugin>
                <plugin>
                    <groupId>org.apache.maven.plugins</groupId>
                    <artifactId>maven-clean-plugin</artifactId>
                    <version>2.6.1</version>
                    <configuration>
                        <filesets>
                            <fileset>
                                <directory>target</directory>
                                <includes>
                                    <include>rerun.properties</include>
                                </includes>
                            </fileset>
                        </filesets>
                    </configuration>
                </plugin>
                <plugin>
                    <groupId>org.apache.maven.plugins</groupId>
                    <artifactId>maven-antrun-plugin</artifactId>
                    <version>1.6</version>
                    <executions>
                        <execution>
                            <phase>compile</phase>
                            <goals>
                                <goal>run</goal>
                            </goals>
                            <configuration>
                                <target>
                                    <echo>Retrying the following classes: "${retryclasses}"</echo>
                                </target>
                            </configuration>
                        </execution>
                    </executions>
                </plugin>
                <plugin>
                    <groupId>org.apache.maven.plugins</groupId>
                    <artifactId>maven-surefire-plugin</artifactId>
                    <version>2.17</version>
                    <configuration>
                        <includes>
                            <include>${retryclasses}</include>
                        </includes>
                        <testFailureIgnore>true</testFailureIgnore>
                    </configuration>
                    <executions>
                        <execution>
                            <phase>test</phase>
                            <goals>
                                <goal>test</goal>
                            </goals>
                        </execution>
                    </executions>
                </plugin>
            </plugins>
        </build>
    </profile>

4) 用法

第一次试运行：

mvn clean test

下一个测试运行：

mvn clean test -Pretry
mvn clean test -Pretry
mvn clean test -Pretry
...

您可以根据需要重复多次，直到没有错误为止。

于 2014-12-16T17:27:28.637 回答
4

I don't have an executable example at hand, but you can do this also on the jvm. There is a RerunFormatter that writes a text file listing the file and line numbers of failed scenarios:

@CucumberOptions(format = {"rerun:target/rerun.txt"})

You should be able to specify this file as input for another test class by prefixing it with @:

@CucumberOptions(features = {"@target/rerun.txt"})
于 2014-12-10T13:18:54.410 回答
1

您可以将黄瓜选项传递给 mvn，如下所示

 mvn clean verify  -Dcucumber.options="@rerun.txt"

请注意，这里有一个棘手的部分。如果您在首次运行和重新运行时使用相同的测试运行程序（我相信这就是您想要的），那么测试运行程序将包含类似

@CucumberOptions(plugin = { "rerun:target/rerun.txt"})

如果您使用以下相同的重新运行文件名使用 maven 重新运行

 mvn clean verify  -Dcucumber.options="@target/rerun.txt"

然后黄瓜会抱怨它找不到重新运行的文件。为什么？因为插件“rerun:target/rerun.txt”会先用这个测试运行器删除文件。

解决方法是先复制/重命名文件，然后像这样启动 mvn 运行

mv target/rerun.txt rerun.txt &&  mvn clean verify  -Dcucumber.options="@rerun.txt"

这实际上就是你想要的。因为假设文件 target/rerun.txt 中有 5 个失败的场景。在一些修复后重新运行，其中 2 个通过了。现在 target/rerun.txt 将只包含其余 3 个失败的场景，这将是您在调试过程中的新起点。

于 2019-03-06T21:35:19.363 回答
0

1） 使用junit4（黄瓜-junit引擎）可以通过rerun插件和features黄瓜选项轻松完成。为失败的场景添加另一个 Maven 配置文件，例如 RerunCucumber.class。

使用主测试运行器运行您的初始构建并通过rerun插件：

@RunWith(Cucumber.class)
@CucumberOptions(tags = "@wip",
        monochrome = true,
        plugin = {"html:target/cucumber", "json:target/wip.json", "rerun:target/rerun_wip.txt"})
public class RunCucumber {
}

构建完成后，失败的场景将被写入target/rerun_wip.txt.

然后可以通过 rerunner 执行失败的场景：

@RunWith(Cucumber.class)
@CucumberOptions(features = {"@features = {"@target/rerun_wip.txt"}"},
        monochrome = true,
        plugin = {"html:target/rerun/failed_tests", "json:target/rerun/failed_tests.json"})
public class RerunCucumber {
}

将从target/rerun_wip.txt.

2） 使用junit5（黄瓜-junit-platform-engine）没有这样的方法（没有features黄瓜选项）。有关使用 junit5 重新运行失败场景的更多信息：https://github.com/cucumber/cucumber-jvm/tree/main/junit-platform-engine，在“重新运行失败的场景”部分

于 2022-02-02T17:10:24.590 回答
0

对于 maven 上的 cucumber + java，我发现了这个命令：

mvn clean test -Dsurefire.rerunFailingTestsCount=2

你必须有一个surefire插件的实际版本，我的是3.0.0-M5。没有什么特别的你甚至需要。在这里找到解决方案Surefire 重新运行失败的测试不起作用

于 2022-02-01T13:08:43.753 回答
-1

您可以使用cucumber-jvm-parallel-plugin贡献的代码作为解决方法，直到它上线。点击命令，如下图所示。

  1. git clone -b tagwiseOutlinewiseIssueRerun https://github.com/sugatmankar/cucumber-jvm-parallel-plugin.git
  2. mvn 全新安装。
  3. 现在编辑您的项目 pom 文件并按照此处所述使用。
  4. 使用这个插件的例子在这里。
于 2016-12-12T11:55:34.113 回答