6

我正在运行ScalaCheck测试sbt,如果我的测试由于被测代码引发异常而失败,则测试报告会显示失败的测试、引发的异常和消息,但不会显示整个堆栈跟踪(请注意Exception: java.lang.NullPointerException: exception下面的仅异常消息) .

[info] Loading global plugins from /Users/jacek/.sbt/0.13/plugins
[info] Set current project to scalacheck (in build file:/Users/jacek/sandbox/scalacheck/)
[info] Updating {file:/Users/jacek/sandbox/scalacheck/}scalacheck...
[info] Resolving jline#jline;2.11 ...
[info] Done updating.
[info] Compiling 1 Scala source to /Users/jacek/sandbox/scalacheck/target/scala-2.11/test-classes...
[info] ! String.throw exception: Exception raised on property evaluation.
[info] > ARG_0: ""
[info] > Exception: java.lang.NullPointerException: exception
[error] Error: Total 1, Failed 0, Errors 1, Passed 0
[error] Error during tests:
[error]     StringSpecification
[error] (test:test) sbt.TestsFailedException: Tests unsuccessful
[error] Total time: 5 s, completed Jun 25, 2014 3:25:47 AM

我从 2008 年发现https://groups.google.com/forum/#!msg/scalacheck/AGBgE_JlqpI/B2eSG84_QzYJ似乎报告了同样的问题,并表明它应该在下一个版本中修复。我目前正在使用最新版本 1.11.4。

我还发现http://www.scala-sbt.org/release/docs/Testing.html表明sbt有一个testOptions我认为使用起来很合理的密钥,并且我知道 ScalaTest 有一个完整堆栈跟踪的设置"-F",但是不适用于 ScalaCheck。即使是上面页面中的示例,testOptions in Test += Tests.Argument(TestFrameworks.ScalaCheck, "-d", "-g")也给了我一个错误:

[error] Could not run test org.example.myproject.MyTestClass: java.lang.Exception: [1.1] failure: option name expected
[error] 
[error] -d -g
[error] ^

我如何使用这些测试参数,是否在任何地方都有这些参数的列表,最后,是否有可能从中获取堆栈跟踪,还是我在追逐红鲱鱼?

4

2 回答 2

4

tl;drverbosity ScalaCheck 的发布版本不支持under sbt。您必须自己从源代码构建一个版本才能拥有该功能。

ScalaCheck 的可用选项在测试执行中描述:

Available options:
  -workers, -w: Number of threads to execute in parallel for testing
  -minSize, -n: Minimum data generation size
  -verbosity, -v: Verbosity level
  -minSuccessfulTests, -s: Number of tests that must succeed in order to pass a property
  -maxDiscardRatio, -r: The maximum ratio between discarded and succeeded tests allowed before ScalaCheck stops testing a property. At least minSuccessfulTests will always be tested, though.
  -maxSize, -x: Maximum data generation size

org.scalacheck.util.Pretty的源代码告诉我们更多关于不同级别的 vebosity:

implicit def prettyThrowable(e: Throwable) = Pretty { prms =>
  val strs = e.getStackTrace.map { st =>
    import st._
    getClassName+"."+getMethodName + "("+getFileName+":"+getLineNumber+")"
  }

  val strs2 =
    if(prms.verbosity <= 0) Array[String]()
    else if(prms.verbosity <= 1) strs.take(5)
    else strs

  e.getClass.getName + ": " + e.getMessage / strs2.mkString("\n")
}

因此,0什么都不提供,1从堆栈跟踪中取出 5 行,而大于的数字1会为您提供整个堆栈跟踪,如下所示:

➜  scalacheck  scala -cp .:/Users/jacek/.ivy2/cache/org.scalacheck/scalacheck_2.11/jars/scalacheck_2.11-1.11.4.jar StringSpecification -verbosity 3
+ String.startsWith: OK, passed 100 tests.
Elapsed time: 0.242 sec
! String.concatenate: Falsified after 0 passed tests.
> ARG_0: ""
> ARG_1: ""
Elapsed time: 0.003 sec
+ String.substring: OK, passed 100 tests.
Elapsed time: 0.126 sec
! String.throw exception: Exception raised on property evaluation.
> ARG_0: ""
> Exception: java.lang.NullPointerException: exception
StringSpecification$$anonfun$14.apply(StringSpecification.scala:19)
StringSpecification$$anonfun$14.apply(StringSpecification.scala:18)
scala.Function1$$anonfun$andThen$1.apply(Function1.scala:55)
org.scalacheck.Prop$$anonfun$forAllShrink$1$$anonfun$3.apply(Prop.scala:622
  )
org.scalacheck.Prop$$anonfun$forAllShrink$1$$anonfun$3.apply(Prop.scala:622
  )
org.scalacheck.Prop$.secure(Prop.scala:473)
org.scalacheck.Prop$$anonfun$forAllShrink$1.org$scalacheck$Prop$$anonfun$$r
  esult$1(Prop.scala:622)
org.scalacheck.Prop$$anonfun$forAllShrink$1.apply(Prop.scala:659)
org.scalacheck.Prop$$anonfun$forAllShrink$1.apply(Prop.scala:616)
org.scalacheck.Prop$$anon$1.apply(Prop.scala:309)
org.scalacheck.Test$.org$scalacheck$Test$$workerFun$1(Test.scala:335)
org.scalacheck.Test$$anonfun$org$scalacheck$Test$$worker$1$1.apply(Test.sca
  la:316)
org.scalacheck.Test$$anonfun$org$scalacheck$Test$$worker$1$1.apply(Test.sca
  la:316)
org.scalacheck.Test$.check(Test.scala:385)
org.scalacheck.Test$$anonfun$checkProperties$1.apply(Test.scala:402)
org.scalacheck.Test$$anonfun$checkProperties$1.apply(Test.scala:395)
scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala
  :245)
scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala
  :245)
scala.collection.immutable.List.foreach(List.scala:383)
scala.collection.generic.TraversableForwarder$class.foreach(TraversableForw
  arder.scala:35)
scala.collection.mutable.ListBuffer.foreach(ListBuffer.scala:45)
scala.collection.TraversableLike$class.map(TraversableLike.scala:245)
scala.collection.AbstractTraversable.map(Traversable.scala:104)
org.scalacheck.Test$.checkProperties(Test.scala:395)
org.scalacheck.Properties.mainRunner(Properties.scala:62)
org.scalacheck.Prop$class.main(Prop.scala:106)
org.scalacheck.Properties.main(Properties.scala:27)
StringSpecification.main(StringSpecification.scala:-1)
sun.reflect.NativeMethodAccessorImpl.invoke0(NativeMethodAccessorImpl.java:
  -2)
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:5
  7)
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImp
  l.java:43)
java.lang.reflect.Method.invoke(Method.java:606)
scala.reflect.internal.util.ScalaClassLoader$$anonfun$run$1.apply(ScalaClas
  sLoader.scala:68)
scala.reflect.internal.util.ScalaClassLoader$class.asContext(ScalaClassLoad
  er.scala:31)
scala.reflect.internal.util.ScalaClassLoader$URLClassLoader.asContext(Scala
  ClassLoader.scala:99)
scala.reflect.internal.util.ScalaClassLoader$class.run(ScalaClassLoader.sca
  la:68)
scala.reflect.internal.util.ScalaClassLoader$URLClassLoader.run(ScalaClassL
  oader.scala:99)
scala.tools.nsc.CommonRunner$class.run(ObjectRunner.scala:22)
scala.tools.nsc.ObjectRunner$.run(ObjectRunner.scala:39)
scala.tools.nsc.CommonRunner$class.runAndCatch(ObjectRunner.scala:29)
scala.tools.nsc.ObjectRunner$.runAndCatch(ObjectRunner.scala:39)
scala.tools.nsc.MainGenericRunner.runTarget$1(MainGenericRunner.scala:72)
scala.tools.nsc.MainGenericRunner.process(MainGenericRunner.scala:94)
scala.tools.nsc.MainGenericRunner$.main(MainGenericRunner.scala:103)
scala.tools.nsc.MainGenericRunner.main(MainGenericRunner.scala:-1)
Elapsed time: 0.000 sec

你说得对,sbt 的文档是不正确的。没有"-d", "-g"选项,我相信它们只是文档中的复制粘贴错误。它已经在即将被接受的拉取请求中修复。

verbosity在最新版本的 ScalaCheck 1.11.4 中,sbt 下不支持该选项。以下是示例项目的完整构建定义。

构建.sbt

scalaVersion := "2.11.1"

libraryDependencies += "org.scalacheck" %% "scalacheck" % "1.11.4" % "test"

testOptions in Test += Tests.Argument(TestFrameworks.ScalaCheck, "-verbosity", "3")

即使在verbosity中正确指定了参数build.sbt,测试执行也不会打印多于一行。

➜  scalacheck  xsbt test
[info] Loading global plugins from /Users/jacek/.sbt/0.13/plugins
[info] Set current project to scalacheck (in build file:/Users/jacek/sandbox/scalacheck/)
[info] + String.startsWith: OK, passed 100 tests.
[info] ! String.concatenate: Falsified after 0 passed tests.
[info] > ARG_0: ""
[info] > ARG_1: ""
[info] + String.substring: OK, passed 100 tests.
[info] ! String.throw exception: Exception raised on property evaluation.
[info] > ARG_0: ""
[info] > Exception: java.lang.NullPointerException: exception
[error] Error: Total 4, Failed 1, Errors 1, Passed 2
[error] Error during tests:
[error]     StringSpecification
[error] (test:test) sbt.TestsFailedException: Tests unsuccessful
[error] Total time: 1 s, completed Jun 25, 2014 2:57:08 AM

原因是org.scalacheck.ScalaCheckFramework具有以下实现并假设verbosity始终为0

override def onTestResult(n: String, r: Test.Result) = {
  for (l <- loggers) {
    import Pretty._
    l.info(
      (if (r.passed) "+ " else "! ") + n + ": " + pretty(r, Params(0))
    )
  }
  handler.handle(asEvent((n,r)))
}

在 TestFrameworks.ScalaCheck 的 sbt下的 Support verbosity 下,已经接受了修复它的请求请求。您必须sbt publishLocal在克隆存储库的目录中自己构建 ScalaCheck。不要忘记使用 1.12.0-SNAPSHOT 版本build.sbt来获取更改。

于 2014-06-25T01:21:58.183 回答
3

由于 Jacek Laskowski 指出的问题已经解决,您现在可以通过在build.sbt文件中添加一行来设置 ScalaCheck 的详细级别,从 sbt 中的 ScalaCheck 获取堆栈跟踪。

testOptions in Test += Tests.Argument(TestFrameworks.ScalaCheck, "-verbosity", "2")

正如 Jacek Laskowski 所指出的,详细程度为 0 只会给您一个例外。

[info] ! RandomAccessFile.readWriteInt: Exception raised on property evaluation.
[info] > ARG_0: 0
[info] > ARG_0_ORIGINAL: -2073744736
[info] > Exception: java.lang.UnsupportedOperationException: null

详细程度为 1 将为您提供堆栈跟踪的前五行。

[info] ! RandomAccessFile.readWriteInt: Exception raised on property evaluation.
[info] > ARG_0: 0
[info] > ARG_0_ORIGINAL: 2147483647
[info] > Exception: java.lang.UnsupportedOperationException: null
[info] java.nio.IntBuffer.array(IntBuffer.java:994)
[info] csc365a01.RandomAccessFile.readInt(RandomAccessFile.scala:22)
[info] RandomAccessFileProps$$anonfun$1$$anonfun$apply$1.apply$mcZI$sp(RandomAccessFileSpec.scala:14)
[info] RandomAccessFileProps$$anonfun$1$$anonfun$apply$1.apply(RandomAccessFileSpec.scala:11)
[info] RandomAccessFileProps$$anonfun$1$$anonfun$apply$1.apply(RandomAccessFileSpec.scala:11)

大于 1 的详细程度将为您提供完整的堆栈跟踪。

[info] ! RandomAccessFile.readWriteInt: Exception raised on property evaluation.
[info] > ARG_0: 0
[info] > ARG_0_ORIGINAL: 1693735989
[info] > Exception: java.lang.UnsupportedOperationException: null
[info] java.nio.IntBuffer.array(IntBuffer.java:994)
[info] csc365a01.RandomAccessFile.readInt(RandomAccessFile.scala:22)
[info] RandomAccessFileProps$$anonfun$1$$anonfun$apply$1.apply$mcZI$sp(RandomAccessFileSpec.scala:14)
[info] RandomAccessFileProps$$anonfun$1$$anonfun$apply$1.apply(RandomAccessFileSpec.scala:11)
[info] RandomAccessFileProps$$anonfun$1$$anonfun$apply$1.apply(RandomAccessFileSpec.scala:11)
[info] scala.Function1$$anonfun$andThen$1.apply(Function1.scala:52)
[info] org.scalacheck.Prop$$anonfun$forAllShrink$1$$anonfun$3.apply(Prop.scala:712)
[info] org.scalacheck.Prop$$anonfun$forAllShrink$1$$anonfun$3.apply(Prop.scala:712)
[info] org.scalacheck.Prop$.secure(Prop.scala:456)
[info] org.scalacheck.Prop$$anonfun$forAllShrink$1.org$scalacheck$Prop$$anonfun$$result$1(Prop.scala:712)
[info] org.scalacheck.Prop$$anonfun$forAllShrink$1$$anonfun$4.apply(Prop.scala:719)
[info] org.scalacheck.Prop$$anonfun$forAllShrink$1$$anonfun$4.apply(Prop.scala:719)
[info] scala.collection.immutable.Stream.map(Stream.scala:418)
[info] org.scalacheck.Prop$$anonfun$forAllShrink$1.getFirstFailure$1(Prop.scala:719)
[info] org.scalacheck.Prop$$anonfun$forAllShrink$1.shrinker$1(Prop.scala:729)
[info] org.scalacheck.Prop$$anonfun$forAllShrink$1.apply(Prop.scala:751)
[info] org.scalacheck.Prop$$anonfun$forAllShrink$1.apply(Prop.scala:706)
[info] org.scalacheck.Prop$$anonfun$apply$5.apply(Prop.scala:292)
[info] org.scalacheck.Prop$$anonfun$apply$5.apply(Prop.scala:291)
[info] org.scalacheck.PropFromFun.apply(Prop.scala:22)
[info] org.scalacheck.Prop$$anonfun$delay$1.apply(Prop.scala:461)
[info] org.scalacheck.Prop$$anonfun$delay$1.apply(Prop.scala:461)
[info] org.scalacheck.Prop$$anonfun$apply$5.apply(Prop.scala:292)
[info] org.scalacheck.Prop$$anonfun$apply$5.apply(Prop.scala:291)
[info] org.scalacheck.PropFromFun.apply(Prop.scala:22)
[info] org.scalacheck.Test$.org$scalacheck$Test$$workerFun$1(Test.scala:294)
[info] org.scalacheck.Test$$anonfun$3.apply(Test.scala:323)
[info] org.scalacheck.Test$$anonfun$3.apply(Test.scala:323)
[info] org.scalacheck.Platform$.runWorkers(Platform.scala:40)
[info] org.scalacheck.Test$.check(Test.scala:323)
[info] org.scalacheck.ScalaCheckRunner$$anon$2$$anonfun$execute$3$$anonfun$apply$2.apply(ScalaCheckFramework.scala:102)
[info] org.scalacheck.ScalaCheckRunner$$anon$2$$anonfun$execute$3$$anonfun$apply$2.apply(ScalaCheckFramework.scala:100)
[info] scala.collection.TraversableLike$WithFilter$$anonfun$foreach$1.apply(TraversableLike.scala:733)
[info] scala.collection.immutable.List.foreach(List.scala:381)
[info] scala.collection.generic.TraversableForwarder$class.foreach(TraversableForwarder.scala:35)
[info] scala.collection.mutable.ListBuffer.foreach(ListBuffer.scala:45)
[info] scala.collection.TraversableLike$WithFilter.foreach(TraversableLike.scala:732)
[info] org.scalacheck.ScalaCheckRunner$$anon$2$$anonfun$execute$3.apply(ScalaCheckFramework.scala:100)
[info] org.scalacheck.ScalaCheckRunner$$anon$2$$anonfun$execute$3.apply(ScalaCheckFramework.scala:97)
[info] scala.collection.TraversableLike$$anonfun$flatMap$1.apply(TraversableLike.scala:241)
[info] scala.collection.TraversableLike$$anonfun$flatMap$1.apply(TraversableLike.scala:241)
[info] scala.collection.IndexedSeqOptimized$class.foreach(IndexedSeqOptimized.scala:33)
[info] scala.collection.mutable.ArrayOps$ofRef.foreach(ArrayOps.scala:186)
[info] scala.collection.TraversableLike$class.flatMap(TraversableLike.scala:241)
[info] scala.collection.mutable.ArrayOps$ofRef.flatMap(ArrayOps.scala:186)
[info] org.scalacheck.ScalaCheckRunner$$anon$2.execute(ScalaCheckFramework.scala:97)
[info] sbt.TestRunner.runTest$1(TestFramework.scala:76)
[info] sbt.TestRunner.run(TestFramework.scala:85)
[info] sbt.TestFramework$$anon$2$$anonfun$$init$$1$$anonfun$apply$8.apply(TestFramework.scala:202)
[info] sbt.TestFramework$$anon$2$$anonfun$$init$$1$$anonfun$apply$8.apply(TestFramework.scala:202)
[info] sbt.TestFramework$.sbt$TestFramework$$withContextLoader(TestFramework.scala:185)
[info] sbt.TestFramework$$anon$2$$anonfun$$init$$1.apply(TestFramework.scala:202)
[info] sbt.TestFramework$$anon$2$$anonfun$$init$$1.apply(TestFramework.scala:202)
[info] sbt.TestFunction.apply(TestFramework.scala:207)
[info] sbt.Tests$$anonfun$9.apply(Tests.scala:216)
[info] sbt.Tests$$anonfun$9.apply(Tests.scala:216)
[info] sbt.std.Transform$$anon$3$$anonfun$apply$2.apply(System.scala:44)
[info] sbt.std.Transform$$anon$3$$anonfun$apply$2.apply(System.scala:44)
[info] sbt.std.Transform$$anon$4.work(System.scala:63)
[info] sbt.Execute$$anonfun$submit$1$$anonfun$apply$1.apply(Execute.scala:226)
[info] sbt.Execute$$anonfun$submit$1$$anonfun$apply$1.apply(Execute.scala:226)
[info] sbt.ErrorHandling$.wideConvert(ErrorHandling.scala:17)
[info] sbt.Execute.work(Execute.scala:235)
[info] sbt.Execute$$anonfun$submit$1.apply(Execute.scala:226)
[info] sbt.Execute$$anonfun$submit$1.apply(Execute.scala:226)
[info] sbt.ConcurrentRestrictions$$anon$4$$anonfun$1.apply(ConcurrentRestrictions.scala:159)
[info] sbt.CompletionService$$anon$2.call(CompletionService.scala:28)
[info] java.util.concurrent.FutureTask.run(FutureTask.java:266)
[info] java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
[info] java.util.concurrent.FutureTask.run(FutureTask.java:266)
[info] java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
[info] java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[info] java.lang.Thread.run(Thread.java:745)
[info] Elapsed time: 0.047 sec
于 2017-03-02T18:15:20.513 回答