39

如何使用 JUnit 测试在现有项目中运行 JMH 基准测试?官方文档建议制作一个单独的项目,使用 Maven shade 插件,并在main方法中启动 JMH。这是必要的吗?为什么推荐它?

4

2 回答 2

70

我一直在使用 JUnit 在我现有的 Maven 项目中运行 JMH,没有明显的不良影响。我无法回答为什么作者建议以不同的方式做事。我没有观察到结果的差异。JMH 启动一个单独的 JVM 来运行基准测试以隔离它们。这是我所做的:

  • 将 JMH 依赖项添加到您的 POM:

    <dependency>
      <groupId>org.openjdk.jmh</groupId>
      <artifactId>jmh-core</artifactId>
      <version>1.21</version>
      <scope>test</scope>
    </dependency>
    <dependency>
      <groupId>org.openjdk.jmh</groupId>
      <artifactId>jmh-generator-annprocess</artifactId>
      <version>1.21</version>
      <scope>test</scope>
    </dependency>
    

    请注意,我已将它们放在范围内test

    在 Eclipse 中,您可能需要手动配置注释处理器。NetBeans 会自动处理这个问题。

  • 创建您的 JUnit 和 JMH 类。我选择将两者合并到一个类中,但这取决于你。请注意,OptionsBuilder.include这实际上决定了您的 JUnit 测试将运行哪些基准测试!

    import java.util.ArrayList;
    import java.util.List;
    import java.util.Random;
    import java.util.concurrent.TimeUnit;
    import org.junit.Test;
    import org.openjdk.jmh.annotations.*;
    import org.openjdk.jmh.infra.Blackhole;
    import org.openjdk.jmh.runner.Runner;
    import org.openjdk.jmh.runner.options.*;
    
    
    public class TestBenchmark 
    {
    
          @Test public void 
        launchBenchmark() throws Exception {
    
                Options opt = new OptionsBuilder()
                    // Specify which benchmarks to run. 
                    // You can be more specific if you'd like to run only one benchmark per test.
                    .include(this.getClass().getName() + ".*")
                    // Set the following options as needed
                    .mode (Mode.AverageTime)
                    .timeUnit(TimeUnit.MICROSECONDS)
                    .warmupTime(TimeValue.seconds(1))
                    .warmupIterations(2)
                    .measurementTime(TimeValue.seconds(1))
                    .measurementIterations(2)
                    .threads(2)
                    .forks(1)
                    .shouldFailOnError(true)
                    .shouldDoGC(true)
                    //.jvmArgs("-XX:+UnlockDiagnosticVMOptions", "-XX:+PrintInlining")
                    //.addProfiler(WinPerfAsmProfiler.class)
                    .build();
    
                new Runner(opt).run();
            }
    
        // The JMH samples are the best documentation for how to use it
        // http://hg.openjdk.java.net/code-tools/jmh/file/tip/jmh-samples/src/main/java/org/openjdk/jmh/samples/
        @State (Scope.Thread)
        public static class BenchmarkState
        {
            List<Integer> list;
    
              @Setup (Level.Trial) public void
            initialize() {
    
                    Random rand = new Random();
    
                    list = new ArrayList<>();
                    for (int i = 0; i < 1000; i++)
                        list.add (rand.nextInt());
                }
        }
    
          @Benchmark public void 
        benchmark1 (BenchmarkState state, Blackhole bh) {
    
                List<Integer> list = state.list;
    
                for (int i = 0; i < 1000; i++)
                    bh.consume (list.get (i));
            }
    }
    
  • JMH 的注释处理器似乎不适用于 NetBeans 中的保存时编译。Clean and Build每当您修改基准时,您可能需要做一个完整的。(任何建议表示赞赏!)

  • 运行您的launchBenchmark测试并观察结果!

    -------------------------------------------------------
     T E S T S
    -------------------------------------------------------
    Running com.Foo
    # JMH version: 1.21
    # VM version: JDK 1.8.0_172, Java HotSpot(TM) 64-Bit Server VM, 25.172-b11
    # VM invoker: /usr/lib/jvm/java-8-jdk/jre/bin/java
    # VM options: <none>
    # Warmup: 2 iterations, 1 s each
    # Measurement: 2 iterations, 1 s each
    # Timeout: 10 min per iteration
    # Threads: 2 threads, will synchronize iterations
    # Benchmark mode: Average time, time/op
    # Benchmark: com.Foo.benchmark1
    
    # Run progress: 0.00% complete, ETA 00:00:04
    # Fork: 1 of 1
    # Warmup Iteration   1: 4.258 us/op
    # Warmup Iteration   2: 4.359 us/op
    Iteration   1: 4.121 us/op
    Iteration   2: 4.029 us/op
    
    
    Result "benchmark1":
      4.075 us/op
    
    
    # Run complete. Total time: 00:00:06
    
    REMEMBER: The numbers below are just data. To gain reusable insights, you need to follow up on
    why the numbers are the way they are. Use profilers (see -prof, -lprof), design factorial
    experiments, perform baseline and negative tests that provide experimental control, make sure
    the benchmarking environment is safe on JVM/OS/HW level, ask for reviews from the domain experts.
    Do not assume the numbers tell you what you want them to tell.
    
    Benchmark                                Mode  Cnt  Score   Error  Units
    Foo.benchmark1                           avgt    2  4.075          us/op
    Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 7.013 sec
    
  • Runner.run甚至返回RunResult可以进行断言等的对象。

于 2015-05-27T14:59:11.883 回答
3

使用注释的声明性方法:

@State(Scope.Benchmark)
@Threads(1)
public class TestBenchmark {
       
    @Param({"10","100","1000"})
    public int iterations;


    @Setup(Level.Invocation)
    public void setupInvokation() throws Exception {
        // executed before each invocation of the benchmark
    }

    @Setup(Level.Iteration)
    public void setupIteration() throws Exception {
        // executed before each invocation of the iteration
    }

    @Benchmark
    @BenchmarkMode(Mode.AverageTime)
    @Fork(warmups = 1, value = 1)
    @Warmup(batchSize = -1, iterations = 3, time = 10, timeUnit = TimeUnit.MILLISECONDS)
    @Measurement(batchSize = -1, iterations = 10, time = 10, timeUnit = TimeUnit.MILLISECONDS)
    @OutputTimeUnit(TimeUnit.MILLISECONDS)
    public void test() throws Exception {
       Thread.sleep(ThreadLocalRandom.current().nextInt(0, iterations));
    }


    @Test
    public void benchmark() throws Exception {
        String[] argv = {};
        org.openjdk.jmh.Main.main(argv);
    }

}
于 2019-12-21T15:03:01.953 回答