java - 如何从 JUnit 测试中运行 JMH？

Question

如何使用 JUnit 测试在现有项目中运行 JMH 基准测试？官方文档建议制作一个单独的项目，使用 Maven shade 插件，并在main方法中启动 JMH。这是必要的吗？为什么推荐它？

Answer 1

我一直在使用 JUnit 在我现有的 Maven 项目中运行 JMH，没有明显的不良影响。我无法回答为什么作者建议以不同的方式做事。我没有观察到结果的差异。JMH 启动一个单独的 JVM 来运行基准测试以隔离它们。这是我所做的：

• 将 JMH 依赖项添加到您的 POM：

  <dependency>
    <groupId>org.openjdk.jmh</groupId>
    <artifactId>jmh-core</artifactId>
    <version>1.21</version>
    <scope>test</scope>
  </dependency>
  <dependency>
    <groupId>org.openjdk.jmh</groupId>
    <artifactId>jmh-generator-annprocess</artifactId>
    <version>1.21</version>
    <scope>test</scope>
  </dependency>
  

  请注意，我已将它们放在范围内test。

  在 Eclipse 中，您可能需要手动配置注释处理器。NetBeans 会自动处理这个问题。

• 创建您的 JUnit 和 JMH 类。我选择将两者合并到一个类中，但这取决于你。请注意，OptionsBuilder.include这实际上决定了您的 JUnit 测试将运行哪些基准测试！

  import java.util.ArrayList;
  import java.util.List;
  import java.util.Random;
  import java.util.concurrent.TimeUnit;
  import org.junit.Test;
  import org.openjdk.jmh.annotations.*;
  import org.openjdk.jmh.infra.Blackhole;
  import org.openjdk.jmh.runner.Runner;
  import org.openjdk.jmh.runner.options.*;
  
  
  public class TestBenchmark 
  {
  
        @Test public void 
      launchBenchmark() throws Exception {
  
              Options opt = new OptionsBuilder()
                  // Specify which benchmarks to run. 
                  // You can be more specific if you'd like to run only one benchmark per test.
                  .include(this.getClass().getName() + ".*")
                  // Set the following options as needed
                  .mode (Mode.AverageTime)
                  .timeUnit(TimeUnit.MICROSECONDS)
                  .warmupTime(TimeValue.seconds(1))
                  .warmupIterations(2)
                  .measurementTime(TimeValue.seconds(1))
                  .measurementIterations(2)
                  .threads(2)
                  .forks(1)
                  .shouldFailOnError(true)
                  .shouldDoGC(true)
                  //.jvmArgs("-XX:+UnlockDiagnosticVMOptions", "-XX:+PrintInlining")
                  //.addProfiler(WinPerfAsmProfiler.class)
                  .build();
  
              new Runner(opt).run();
          }
  
      // The JMH samples are the best documentation for how to use it
      // http://hg.openjdk.java.net/code-tools/jmh/file/tip/jmh-samples/src/main/java/org/openjdk/jmh/samples/
      @State (Scope.Thread)
      public static class BenchmarkState
      {
          List<Integer> list;
  
            @Setup (Level.Trial) public void
          initialize() {
  
                  Random rand = new Random();
  
                  list = new ArrayList<>();
                  for (int i = 0; i < 1000; i++)
                      list.add (rand.nextInt());
              }
      }
  
        @Benchmark public void 
      benchmark1 (BenchmarkState state, Blackhole bh) {
  
              List<Integer> list = state.list;
  
              for (int i = 0; i < 1000; i++)
                  bh.consume (list.get (i));
          }
  }
  

• JMH 的注释处理器似乎不适用于 NetBeans 中的保存时编译。Clean and Build每当您修改基准时，您可能需要做一个完整的。（任何建议表示赞赏！）

• 运行您的launchBenchmark测试并观察结果！

  -------------------------------------------------------
   T E S T S
  -------------------------------------------------------
  Running com.Foo
  # JMH version: 1.21
  # VM version: JDK 1.8.0_172, Java HotSpot(TM) 64-Bit Server VM, 25.172-b11
  # VM invoker: /usr/lib/jvm/java-8-jdk/jre/bin/java
  # VM options: <none>
  # Warmup: 2 iterations, 1 s each
  # Measurement: 2 iterations, 1 s each
  # Timeout: 10 min per iteration
  # Threads: 2 threads, will synchronize iterations
  # Benchmark mode: Average time, time/op
  # Benchmark: com.Foo.benchmark1
  
  # Run progress: 0.00% complete, ETA 00:00:04
  # Fork: 1 of 1
  # Warmup Iteration   1: 4.258 us/op
  # Warmup Iteration   2: 4.359 us/op
  Iteration   1: 4.121 us/op
  Iteration   2: 4.029 us/op
  
  
  Result "benchmark1":
    4.075 us/op
  
  
  # Run complete. Total time: 00:00:06
  
  REMEMBER: The numbers below are just data. To gain reusable insights, you need to follow up on
  why the numbers are the way they are. Use profilers (see -prof, -lprof), design factorial
  experiments, perform baseline and negative tests that provide experimental control, make sure
  the benchmarking environment is safe on JVM/OS/HW level, ask for reviews from the domain experts.
  Do not assume the numbers tell you what you want them to tell.
  
  Benchmark                                Mode  Cnt  Score   Error  Units
  Foo.benchmark1                           avgt    2  4.075          us/op
  Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 7.013 sec
  

• Runner.run甚至返回RunResult可以进行断言等的对象。

Answer 2

使用注释的声明性方法：

@State(Scope.Benchmark)
@Threads(1)
public class TestBenchmark {
       
    @Param({"10","100","1000"})
    public int iterations;


    @Setup(Level.Invocation)
    public void setupInvokation() throws Exception {
        // executed before each invocation of the benchmark
    }

    @Setup(Level.Iteration)
    public void setupIteration() throws Exception {
        // executed before each invocation of the iteration
    }

    @Benchmark
    @BenchmarkMode(Mode.AverageTime)
    @Fork(warmups = 1, value = 1)
    @Warmup(batchSize = -1, iterations = 3, time = 10, timeUnit = TimeUnit.MILLISECONDS)
    @Measurement(batchSize = -1, iterations = 10, time = 10, timeUnit = TimeUnit.MILLISECONDS)
    @OutputTimeUnit(TimeUnit.MILLISECONDS)
    public void test() throws Exception {
       Thread.sleep(ThreadLocalRandom.current().nextInt(0, iterations));
    }


    @Test
    public void benchmark() throws Exception {
        String[] argv = {};
        org.openjdk.jmh.Main.main(argv);
    }

}