24

我正在开发一个需要以编程方式调用 TestNG(使用数据提供程序)的项目。一切都很好,只是在报告中,我们得到了 @Test 方法的名称,它是处理许多情况的通用方法。我们想要的是在报告中获得一个有意义的名称。

我正在研究这个并找到了 3 种方法,但不幸的是,对我来说都失败了。

1) 实施 ITest

我在这里这里找到了这个

我在输入@Test 方法后立即设置我想要的名称(对于我尝试的所有 3 种方法,这就是我设置名称的方式)。这个名称是从 getTestName() 返回的。我观察到的是 getTestName() 在我的@Test 之前和之后被调用。最初,它返回 null(为了处理 NullPointerException,我返回 "" 而不是 null),后来它返回正确的值。但我没有看到这反映在报告中

编辑:还尝试按照 artdanil 的建议从@BeforeMethod 设置名称

2 和 3

两者都基于上面第二个链接中给出的解决方案

通过在 XmlSuite 中覆盖 setName,我得到了

Exception in thread "main" java.lang.AssertionError: l should not be null
        at org.testng.ClassMethodMap.removeAndCheckIfLast(ClassMethodMap.java:58)
        at org.testng.internal.TestMethodWorker.invokeAfterClassMethods(TestMethodWorker.java:208)
        at org.testng.internal.TestMethodWorker.run(TestMethodWorker.java:114)
        at org.testng.TestRunner.privateRun(TestRunner.java:767)
        ...

通过覆盖 toString(),我在日志中看到了这些(带有我的评论),但在报告中没有更新

[2013-03-05 14:53:22,174] (Main.java:30) - calling execute 
    [2013-03-05 14:53:22,346] GenericFunctionTest.<init>(GenericFunctionTest.java:52) - inside constructor
    [2013-03-05 14:53:22,372] GenericFunctionTest.toString(GenericFunctionTest.java:276) - returning **//this followed by 3 invocations before arriving at @Test method**
    [2013-03-05 14:53:22,410] GenericFunctionTest.toString(GenericFunctionTest.java:276) - returning 
    [2013-03-05 14:53:22,416] GenericFunctionTest.toString(GenericFunctionTest.java:276) - returning 
    [2013-03-05 14:53:22,455] GenericFunctionTest.toString(GenericFunctionTest.java:276) - returning 
    [2013-03-05 14:53:22,892] GenericFunctionTest.<init>(GenericFunctionTest.java:52) - inside constructor 
    [2013-03-05 14:53:23,178] GenericFunctionTest.toString(GenericFunctionTest.java:276) - returning **//again blank as i havent set it yet**
    [2013-03-05 14:53:23,182] GenericFunctionTest.getResult(GenericFunctionTest.java:69) - inside with test case:TestCase{signature=Signature{...}}**//I am setting it immedietely after this**
    [2013-03-05 14:53:23,293] GenericFunctionTest.toString(GenericFunctionTest.java:276) - returning MyMethodName **//What i want**
    [2013-03-05 14:53:23,299] GenericFunctionTest.toString(GenericFunctionTest.java:276) - returning MyMethodName **// again**

编辑:通过硬编码一个值而不是在我的测试方法条目中设置它,再次尝试所有 3 个。但同样的结果

4

7 回答 7

15

我遇到了同样的问题,发现在使用 注释的方法中设置存储测试用例名称的字段@BeforeMethod,使用TestNG的本机注入来提供方法名称和测试参数是有帮助的。测试名称取自DataProvider. 如果您的测试方法没有参数,只需报告方法名称即可。

//oversimplified for demontration purposes
public class TestParameters {
    private String testName = null;
    private String testDescription = null;

    public TestParameters(String name,
                          String description) {
        this.testName = name;
        this.testDescription = description;
    }

    public String getTestName() {
        return testName;
    }
    public String getTestDescription() {
        return testDescription;
    }
}

public class SampleTest implements ITest {
    // Has to be set to prevent NullPointerException from reporters
    protected String mTestCaseName = "";

    @DataProvider(name="BasicDataProvider")
    public Object[][] getTestData() {
        Object[][] data = new Object[][] {
                { new TestParameters("TestCase1", "Sample test 1")},
                { new TestParameters("TestCase2", "Sample test 2")},
                { new TestParameters("TestCase3", "Sample test 3")},
                { new TestParameters("TestCase4", "Sample test 4")},
                { new TestParameters("TestCase5", "Sample test 5") }
        };
        return data;
    }

    @BeforeMethod(alwaysRun = true)
    public void testData(Method method, Object[] testData) {
        String testCase = "";
        if (testData != null && testData.length > 0) {
            TestParameters testParams = null;
            //Check if test method has actually received required parameters
            for (Object testParameter : testData) {
                if (testParameter instanceof TestParameters) {
                    testParams = (TestParameters)testParameter;
                    break;
                }
            }
            if (testParams != null) {
                testCase = testParams.getTestName();
            }
        }
        this.mTestCaseName = String.format("%s(%s)", method.getName(), testCase);
    }

    @Override
    public String getTestName() {
        return this.mTestCaseName;
    }

    @Test(dataProvider="BasicDataProvider")
    public void testSample1(TestParameters testParams){
        //test code here
    }

    @Test(dataProvider="BasicDataProvider")
    public void testSample2(TestParameters testParams){
        //test code here
    }

    @Test
    public void testSample3(){
        //test code here
    }
}

编辑:根据下面的评论,我意识到报告中的样本会很有用。

从上面运行代码的报告中提取:

<testng-results skipped="0" failed="0" total="5" passed="5">
  <suite name="SampleTests" duration-ms="2818" started-at="<some-time>" finished-at="<some-time>">
    <test name="Test1" duration-ms="2818" started-at="<some-time>" finished-at="<some-time>">
        <test-method 
            status="PASS" 
            signature="testSample1(org.example.test.TestParameters)[pri:0, instance:org.example.test.TimeTest@c9d92c]"
            test-instance-name="testSample1(TestCase5)"
            name="testSample1" 
            duration-ms="1014"
            started-at="<some-time-before>" 
            data-provider="BasicDataProvider" 
            finished-at="<some-time-later>" >
            <!-- excluded for demonstration purposes -->
        </test-method>
        <!-- the rest of test results excluded for brevity -->
    </test>
  </suite>
</testng-result>

请注意,getTestName()方法返回的值在test-instance-name属性中,而不是在name属性中。

于 2013-03-11T21:37:42.540 回答
4

我遇到了类似的问题。首先,我实施了已经提到的 ITest 策略。这是解决方案的一部分,但不是完全的。

出于某种原因,TestNG 在构建不同的报告时,会在构建报告时在测试上调用 getName()。如果您不使用数据提供程序来生成不同的运行并使用 ITest 策略为每次运行设置唯一名称,这很好。如果您使用数据提供程序来生成同一测试的多次运行并希望每次运行都有一个唯一的名称,那么就会出现问题。由于 ITest 策略将测试名称保留为上次运行设置的名称。

所以我必须实现一个非常自定义的 getName()。一些假设(在我的特定情况下):

  1. 只运行三个报告:TestHTMLReporter、EmailableReporter、XMLSuiteResultWriter。
  2. 如果由于其中一位假定的记者没有调用 get name ,那么返回当前设置的名称就可以了。
  3. 当记者运行时,它会按顺序调用 getName(),每次运行只调用 1 次。
public String getTestName()
{
    String name = testName;
    StackTraceElement[] stackTrace = Thread.currentThread().getStackTrace();//.toString();
    if(calledFrom(stackTrace, "XMLSuiteResultWriter"))
    {
        name = testNames.size()>0?testNames.get(xmlNameIndex<testNames.size()?xmlNameIndex:0):"undefined";
        xmlNameIndex++;
        if(xmlNameIndex>=testNames.size())
            xmlNameIndex = 0;
    }
    else if(calledFrom(stackTrace, "EmailableReporter"))
    {
        name = testNames.size()>0?testNames.get(emailNameIndex<testNames.size()?emailNameIndex:0):"undefined";
        emailNameIndex++;
        if(emailNameIndex>=testNames.size())
            emailNameIndex = 0;
    }
    if(calledFrom(stackTrace, "TestHTMLReporter"))
    {
        if(testNames.size()<0)
        {
            name = "undefined";
        }
        else
        {
            if(htmlNameIndex < testNamesFailed.size())
            {
                name = testNamesFailed.get(htmlNameIndex);
            }
            else
            {
                int htmlPassedIndex = htmlNameIndex - testNamesFailed.size();
                if(htmlPassedIndex < testNamesPassed.size())
                {
                    name = testNamesPassed.get(htmlPassedIndex);
                }
                else
                {
                    name = "undefined";
                }
            }
        }
        htmlNameIndex++;
        if(htmlNameIndex>=testNames.size())
            htmlNameIndex = 0;
    }
    return name;
}

private boolean calledFrom(StackTraceElement[] stackTrace, String checkForMethod)
{
    boolean calledFrom = false;
    for(StackTraceElement element : stackTrace)
    {
        String stack = element.toString();
        if(stack.contains(checkForMethod))
            calledFrom = true;
    }
    return calledFrom;
}

在为运行设置名称时(我在遵循 ITest 策略定义的 @BeforeMethod(alwaysRun=true) 方法中执行此操作)我将名称添加到 ArrayList testNames。但随后的 html 报告不正确。大多数其他报告按顺序提取信息,例如 XMLSuiteResultWriter,但 TestHTMLReporter 通过首先获取失败测试的所有名称,然后获取通过测试的名称来获取名称。所以我必须实现额外的 ArrayLists: testNamesFailed 和 testNamesPassed 并在测试完成时根据它们是否通过来添加测试名称。

我会坦率地承认,这在很大程度上是一种黑客行为,而且非常脆弱。理想情况下,TestNG 在运行时将测试添加到集合中,并从该集合而不是原始测试中获取名称。如果您有 TestNG 来运行其他报告,您将必须弄清楚他们请求名称的顺序以及在线程堆栈跟踪中搜索的足够独特的字符串是什么。

--编辑 1

或者,使用 ITest 策略和工厂模式(@factory 注释)。

TestNG 使用 @Factory 和 @DataProvider

http://beust.com/weblog/2004/09/27/testngs-factory/

它确实需要一些小的改动。这包括创建一个与原始测试方法具有相同参数的构造函数。测试方法现在没有参数。您可以在新构造函数中设置名称,然后在 getTestName 方法中简单地返回它。确保从测试方法中删除数据提供者规范。

于 2015-03-11T18:49:48.293 回答
3

请在 TestNG 报告中找到以下代码以设置测试用例的自定义名称。

此代码中提供了以下功能。

  • 多次在同一测试用例上动态执行
  • 为报告设置自定义测试用例名称
  • 设置多个测试用例执行的并行执行

    import java.lang.reflect.Field;
    import org.testng.ITest;
    import org.testng.ITestResult;
    import org.testng.Reporter;
    import org.testng.annotations.AfterMethod;
    import org.testng.annotations.DataProvider;
    import org.testng.annotations.Factory;
    import org.testng.annotations.Test;
    import org.testng.internal.BaseTestMethod;
    import com.test.data.ServiceProcessData;
    
    public class ServiceTest implements ITest {
    
    protected ServiceProcessData serviceProcessData;
    protected String testCaseName = "";
    
    @Test
    public void executeServiceTest() {
        System.out.println(this.serviceProcessData.toString());
    }
    
    @Factory(dataProvider = "processDataList")
    public RiskServiceTest(ServiceProcessData serviceProcessData) {
        this.serviceProcessData = serviceProcessData;
    }
    
    @DataProvider(name = "processDataList", parallel = true)
    public static Object[] getProcessDataList() {
    
        Object[] serviceProcessDataList = new Object[0];
        //Set data in serviceProcessDataList
        return serviceProcessDataList;
    }
    
    @Override
    public String getTestName() {
    
        this.testCaseName = "User custom testcase name";
    
        return this.testCaseName;
    }
    
    @AfterMethod(alwaysRun = true)
    public void setResultTestName(ITestResult result) {
        try {
            BaseTestMethod baseTestMethod = (BaseTestMethod) result.getMethod();
            Field f = baseTestMethod.getClass().getSuperclass().getDeclaredField("m_methodName");
            f.setAccessible(true);
            f.set(baseTestMethod, this.testCaseName);
        } catch (Exception e) {
            ErrorMessageHelper.getInstance().setErrorMessage(e);
            Reporter.log("Exception : " + e.getMessage());
        }
    }}
    

    谢谢

于 2018-03-27T21:09:01.960 回答
3

如果您想更改 HTML 报告中的名称,那将是一个彻头彻尾的黑客攻击。我是这样做的:

public class NinjaTest {
...
...
@AfterMethod (alwaysRun = true)
public void afterMethod(ITestResult result, Method method) {
    try {
        //I have XML test suites organized in directories. 
        String xmlFile = result.getTestContext().getCurrentXmlTest().getSuite().getFileName();
        String suiteName = xmlFile.substring(xmlFile.lastIndexOf("\\") + 1, xmlFile.lastIndexOf(".xml"));
        String pathToFile = xmlFile.substring(0, xmlFile.lastIndexOf("\\") );
        String directory = pathToFile.substring(pathToFile.lastIndexOf("\\") + 1);
        String testMethodName = String.format("%s/%s - %s", directory, suiteName, method.getName());

        //Total hack to change display name in HTML report  \(^o^)/ 
        Field methodName = org.testng.internal.BaseTestMethod.class.getDeclaredField("m_methodName");
        methodName.setAccessible(true);
        methodName.set(result.getMethod(), testMethodName);
    } catch (Exception e) {
        // Eh....  ¯\_(ツ)_/¯
        e.printStackTrace();
    }
}
...
...
于 2017-05-01T23:37:29.290 回答
2

artdanil 的回答并没有完全解决我的问题,测试名称没有在电子邮件报告中更新。

@jersey-city-ninja 发布的答案确实更新了 Emailable 报告中的名称,但它为所有 Dataprovider 值重复了上次更新的测试名称,因为 Pilotg2 发布的内容对于使用 Dataprovider 的测试是正确的,即 getTestName 方法不断返回最后一组方法的名称和数据提供者的所有测试名称都相同。

所以这里的答案是@pilotg2 和@jersey-city-ninja 发布的答案的组合,以及克服重复方法名称的额外步骤。

请注意,这会更新 Emailable 报告、XML 报告、HTML 报告、Junit 报告中的测试名称。我没有看到它在更新 Eclipse - TestNg 执行视图 - 如果我发现了什么,将会更新

import org.testng.Assert;
import org.testng.ITest;
import org.testng.ITestContext;
import org.testng.ITestResult;
import org.testng.annotations.AfterMethod;
import org.testng.annotations.BeforeClass;
import org.testng.annotations.BeforeMethod;
import org.testng.annotations.DataProvider;
import org.testng.annotations.Test;

import java.lang.reflect.Method;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.Iterator;
import java.util.List;


public class NewDataProviderTest implements ITest {
    //The Java ThreadLocal class enables you to create variables that can only be read and written by the same thread
    private ThreadLocal<String> testName = new ThreadLocal<>();

    /*
    * TestNG, for some reason, when building different reports, calls getName() on the test while building the report.
    * This is fine if you are not using a data provider to generate different runs and set a unique name for each run by using the ITest strategy.
    * If you are using a data provider to generate multiple runs of the same test and want each run to have a unique name then there is a problem.
    * As the ITest strategy returns the name for the test as the name set by the last run.
    * */
    private int emailNameIndex = 0;
    private int htmlNameIndex = 0;
    private int xmlNameIndex = 0;
    private ArrayList<String> allTests = new ArrayList<String>();
    /*
    * TestHTMLReporter gets the name by first getting all the names for failed tests and then the names for passing tests
    * Hence keeping them in 2 separate lists
    * */
    private ArrayList<String> passedTests = new ArrayList<String>();
    private ArrayList<String> failedTests = new ArrayList<String>();

    @BeforeClass(alwaysRun = true)
    public void initialize() {
        this.testName.set("");
    }

    @BeforeMethod(alwaysRun = true)
    public void setCustomTestcaseName(Method method, Object[] testData) {
        //Set the default name
        this.testName.set(method.getName());
        //Change the test name only if Dataprovider is used
        //Check if data provider is used in the test
        if (testData != null && testData.length > 0) {
            System.out.println("\n\nParameters "+testData[0]+" are passed to the test - "+method.getName());
            //Taking 1st field in the Dataprovider for updating testname - can be changed as desired maybe using a variable
            //I'm changing the name only if the Dataprovider field is String
            if (testData[0] instanceof String) {
                //Taking 1st field in the Dataprovider for updating testname - can be changed as desired
                System.out.println("I'm setting custom name to the test as "+method.getName() + "_" + testData[0]);
                this.testName.set(method.getName() + "_" + testData[0]);
            }

        }
        //Add the name to the collection that stores all list names
        allTests.add(testName.get());

    }

     @AfterMethod (alwaysRun = true)
     public void setTheTestcaseNameInResult(ITestResult result, Method method) {
        //Fill the Passed and Failed tests collections
         try {
             if(result.getStatus() == ITestResult.SUCCESS) {
                 System.out.println("Adding "+ result.getTestName() + " to passed tests collection");
                 passedTests.add(result.getTestName());
             }
             if(result.getStatus() == ITestResult.FAILURE) {
                 System.out.println("Adding " + result.getTestName() + " to FAILURE tests collection");
                 failedTests.add(result.getTestName());
             }
         } catch (Exception e) {
             e.printStackTrace();
         }
         // To change display name in HTML report
         //Only changing the name if the parameter is instance of String
         if(iTestResult.getParameters().length > 0) {
             if (iTestResult.getParameters()[0] instanceof String) {
                 System.out.println("Updating the name as Parameters are passed to the test-"+method.getName());
                 try {
                     /* This helps in setting unique name to method for each test instance for a data provider*/
                     Field resultMethod = TestResult.class.getDeclaredField("m_method");
                     resultMethod.setAccessible(true);
                     resultMethod.set(iTestResult, iTestResult.getMethod().clone());

                     Field methodName = org.testng.internal.BaseTestMethod.class.getDeclaredField("m_methodName");
                     methodName.setAccessible(true);
                     methodName.set(iTestResult.getMethod(), this.getTestName());
                 } catch (Exception e) {
                     e.printStackTrace();
                 }
                 System.out.println("New Name is - " + iTestResult.getMethod().getMethodName());

             }
         }
     }

    @Override
    public String getTestName() {
        String name = testName.get();
        StackTraceElement[] stackTrace = Thread.currentThread().getStackTrace();// .toString();
        //This is called
        if (isCalledFromMethod(stackTrace, "XMLSuiteResultWriter")) {
            //System.out.println("Got called from XMLSuiteResultWriter");
            if (allTestNames.size() > 0) {
                if (xmlNameIndex < allTestNames.size()) {
                    name = allTestNames.get(xmlNameIndex);
                } else {
                    name = allTestNames.get(0);
                }
            } else {
                name = "undefined";
            }
            xmlNameIndex++;
            if (xmlNameIndex >= allTestNames.size()) {
                xmlNameIndex = 0;
            }
            // System.out.println("Got called from XMLSuiteResultWriter returning name - "+name);
        } else if (isCalledFromMethod(stackTrace, "EmailableReporter")) {
            if (allTestNames.size() > 0) {
                if (emailNameIndex < allTestNames.size()) {
                    name = allTestNames.get(emailNameIndex);
                } else {
                    name = allTestNames.get(0);
                }
            } else {
                name = "undefined";
            }
            emailNameIndex++;
            if (emailNameIndex >= allTestNames.size()) {
                emailNameIndex = 0;
            }
            System.out.println("Got called from EmailableReporter returning name -"+name);
        }
        if (isCalledFromMethod(stackTrace, "TestHTMLReporter")) {
            if (allTestNames.size() <= 0) {
                name = "undefined";
            } else {
                if (htmlNameIndex < failedTestNames.size()) {
                    name = failedTestNames.get(htmlNameIndex);
                } else {
                    int htmlPassedIndex = htmlNameIndex - failedTestNames.size();
                    if (htmlPassedIndex < passedTestNames.size()) {
                        name = passedTestNames.get(htmlPassedIndex);
                    } else {
                        name = "undefined";
                    }
                }
            }
            htmlNameIndex++;
            if (htmlNameIndex >= allTestNames.size()) {
                htmlNameIndex = 0;
            }
            System.out.println("Got called from TestHTMLReporter returning name - "+name);
        }
        System.out.println("Returning testname as-"+name);
        return name;
    }

    private boolean isCalledFromMethod(StackTraceElement[] stackTrace, String checkForMethod) {
        boolean calledFrom = false;
        for (StackTraceElement element : stackTrace) {
            String stack = element.toString();
            // System.out.println("Rohit the called from value is:"+stack);
            if (stack.contains(checkForMethod))
                calledFrom = true;
        }
        return calledFrom;
    }


  @Test(groups= {"L1", "L2", "L3"}, dataProvider = "dp1")
  public void dataProviderTest(String username) {
      System.out.println("\n\nI'm in dataProviderTest with data-"+username);
      /* Fail the test if value is L2 - deliberately so that we have failed test in report */ 
      if(username.contains("L2")) {
          Assert.fail();
      }

  }

  @Test(dependsOnMethods = "dataProviderTest", groups= {"L1", "L2", "L3"}, dataProvider = "dp1")
  public void dataProviderDependentTest(String username) {
      System.out.println("\n\nI'm in dataProvider DEPENDENT Test with data-"+username);

  }

  //This test consumes data of type list so the name will not be updated in report
  @Test(groups= {"L1", "L2", "L3"}, dataProvider = "dp2")
  public void dataListProviderTest(List<String[]> list) {
      Object[] arr = list.get(0);
        List<Object> arrList = Arrays.asList(arr);
        Iterator<Object> iterator = arrList.iterator();
        while (iterator.hasNext()) {

        String[] data = (String[]) iterator.next();
        System.out.println("In list test - "+data[0]);
        }    

  }

  @DataProvider(name="dp1")
  public Object[][] getDataForTest(ITestContext iTestContext){
      Object[][] L1 = new Object[][] {
          {"L1"}, {"L2"}, {"L3"}
      };


      return L1;
  }


  @DataProvider(name="dp2")
  public Object[][] getDataListForTest(ITestContext iTestContext){
      List<Object[][]> list = new ArrayList<Object[][]>();
      Object[][] L1 = new Object[][] {
          new String [] {"L1", "l1"}, 
          new String [] {"L1", "l1"}
      };

     list.add(L1);
      return new Object[][] { { list } };
  }

}
于 2020-06-08T13:39:05.320 回答
1

Try implementing the org.testng.ITest interface that requires a getTestName() method. This way reporting handles the returned value properly.

于 2015-01-05T00:03:28.710 回答
1

遇到同样的问题,我们就是这样解决的:

问题是所有 ITestResult 结果对象共享一个 ITestNGMethod 对象实例,因此当方法名称更改时,它会影响所有结果。

来自:org.testng.reporters.XMLSuiteResultWriter#getTestResultAttributes

    attributes.setProperty(XMLReporterConfig.ATTR_NAME, testResult.getMethod().getMethodName());

name 属性正在从 testResult.getMethod().getMethodName() 中读取

基本上我们为每个结果克隆了方法对象,因此每个结果都可以有一个独立的 getMethod()。

我们添加了这段代码:

  @AfterMethod(alwaysRun = true)
public void setResultTestName(ITestResult result, ITestContext context) {
    try {
        BaseTestMethod baseTestMethod = (BaseTestMethod) result.getMethod().clone();

        String featureName = StringUtils.substringBetween(result.getParameters()[1].toString(), "[", "]");
        String scenarioName = result.getParameters()[0].toString().replace("\"", "");
        Field f = baseTestMethod.getClass().getSuperclass().getDeclaredField("m_methodName");
        f.setAccessible(true);
        f.set(baseTestMethod, featureName + " - " + scenarioName);

        f = result.getClass().getDeclaredField("m_method");
        f.setAccessible(true);
        f.set(result, baseTestMethod);

    } catch (Exception e) {
        e.printStackTrace();
    }
}
于 2021-11-16T20:33:56.423 回答