3

我读过在单元测试中生成随机数据通常是一个坏主意（我确实理解为什么），但是对随机数据进行测试，然后从发现错误的随机测试中构建一个固定的单元测试用例似乎不错。但是我不明白如何很好地组织它。我的问题实际上与特定的编程语言或特定的单元测试框架无关，所以我将使用 python 和一些伪单元测试框架。这是我如何看待它的编码：

def random_test_cases():
   datasets = [
       dataset1,
       dataset2,
       ...
       datasetn
   ]
   for dataset in datasets:
       assertTrue(...)
       assertEquals(...)
       assertRaises(...)
       # and so on

问题是：当这个测试用例失败时，我无法确定是哪个数据集导致失败。我看到了两种解决方法：

  1. 每个数据集创建一个测试用例——问题是测试用例的负载和代码重复。
  2. 通常测试框架让我们将消息传递给断言函数（在我的示例中，我可以执行类似的操作assertTrue(..., message = str(dataset))）。问题是我应该将这样的消息传递给每个断言，这看起来也不优雅。

有没有更简单的方法呢？

4

5 回答 5

6

我仍然认为这是一个坏主意。

单元测试需要简单明了。给定相同的代码和相同的单元测试，您应该能够无限运行它并且永远不会得到不同的响应，除非有外部因素在起作用。与此相反的目标将增加自动化的维护成本，这与目标背道而驰。

在维护方面之外，对我来说这似乎很懒惰。如果您考虑到您的功能并了解正面和负面的测试用例，那么开发单元测试就很简单了。

我也不同意显示如何在同一个测试用例中执行多个测试用例的用户。当测试失败时，您应该能够立即判断哪个测试失败并知道失败的原因。测试应该尽可能简单，并尽可能简洁/与被测代码相关。

于 2012-07-19T05:21:49.680 回答
2

I also think it's a bad idea.

Mind you, not throwing random data at your code, but having unit tests doing that. It all boils down to why you unit test in the first place. The answer is "to drive the design of the code". Random data doesn't drive the design of the code, because it depends on a very rigid public interface. Mind you, you can find bugs with it, but that's not what unit tests are about. And let me note that I'm talking about unit tests, and not tests in general.

That being said, I strongly suggest taking a look at QuickCheck. It's Haskell, so it's a bit dodgy on presentation and a bit PhD-ish on documentation, but you should be able to figure it out. I'm going to summarize how it works, though.

After you pick the code you want to test (let's say the sort() function), you establish invariants which should hold. In this examples, you can have the following invariants if result = sort(input):.

  • Every element in result should be smaller than or equal to the next one.
  • Every element in input should be present in result the same number of times.
  • result and input should have the same length (this is repeats the previous, but let's have it for illustration).

You encode each variant in a simple function that takes the result and the output and checks whether those invariants code.

Then, you tell QuickCheck how to generate input. Since this is Haskell and the type system kicks ass, it can see that the function takes a list of integers and it knows how to generate those. It basically generates random lists of random integers and random length. Of course, it can be more fine-grained if you have a more complex data type (for example, only positive integers, only squares, etc.).

Finally, when you have those two, you just run QuickCheck. It generates all that stuff randomly and checks the invariants. If some fail, it will show you exactly which ones. It would also tell you the random seed, so you can rerun this exact failure if you need to. And as an extra bonus, whenever it gets a failed invariant, it will try to reduce the input to the smallest possible subset that fails the invariant (if you think of a tree structure, it will reduce it to the smallest subtree that fails the invariant).

And there you have it. In my opinion, this is how you should go about testing stuff with random data. It's definitely not unit tests and I even think you should run it differently (say, have CI run it every now and then, as opposed to running it on every change (since it will quickly get slow)). And let me repeat, it's a different benefit from unit testing - QuickCheck finds bugs, while unit testing drives design.

于 2012-07-19T13:05:49.283 回答
2

您可以通过扩展而不是枚举来定义测试，或者您可以从一个案例中调用多个测试案例。

从单个测试用例调用多个测试用例：

MyTest()
{
    MyTest(1, "A")
    MyTest(1, "B")
    MyTest(2, "A")
    MyTest(2, "B")
    MyTest(3, "A")
    MyTest(3, "B")
}

有时有一些优雅的方法可以通过一些测试框架来实现这一点。以下是在 NUnit 中的操作方法：

[Test, Combinatorial]
public void MyTest(
    [Values(1,2,3)] int x,
    [Values("A","B")] string s)
{
    ...
}
于 2012-07-19T03:28:42.523 回答
0

通常，只要您选择正确的断言方法，单元测试框架就支持“信息失败”。

但是，如果其他一切都不起作用，您可以轻松地将数据集跟踪到控制台/输出文件。技术含量低，但应该工作。

[TestCaseSource("GetDatasets")]
public Test.. (Dataset d)
{
   Console.WriteLine(PrettyPrintDataset(d));
   // proceed with checks
   Console.WriteLine("Worked!");
}
于 2012-07-19T06:02:39.397 回答
0

在快速检查 R 我们试图解决这个问题如下

  • 测试实际上是伪随机的（种子是固定的），所以你总是可以重现你的测试结果（当然，除非外部因素）
  • 该test函数返回足够的数据来重现错误，包括失败的断言和使其失败的数据。repro在 的返回值上调用的便利函数test将使您在失败的断言开始时进入调试器，并将参数设置为失败的见证人。如果测试以批处理模式执行，则等效信息存储在文件中，检索它的命令打印在 stderr 中。然后你可以repro像以前一样打电话。无论您是否使用 R 编程，我都想知道这是否开始满足您的要求。此解决方案的某些方面可能难以在动态性较低或没有一流功能的语言中实现。
于 2015-02-12T20:52:54.413 回答