5

rspec --bisect在 circleci 上运行时，我看到了一些意想不到的行为。平分常常无限期地运行，直到它在 5 小时后超时。二分法最初似乎是有效的，但是一旦达到预期的终点，它就会开始缓慢地检查相反方向的子集，直到超时。

我的环境：
Ruby 2.4.2p198 (2017-09-14 revision 59899) [x86_64-darwin16]
RSpec 3.7
- rspec-core 3.7.1
- rspec-expectations 3.7.0
- rspec-mocks 3.7.0
- rspec-rails 3.7.2
- rspec-support 3.7.1

命令：
DISABLE_SPRING=true RAILS_ENV=test bundle exec rspec #{all_specs} --order rand:21237 --bisect

Running suite to find failures... (1 minute 5.94 seconds)
Starting bisect with 1 failing example and 424 non-failing examples.
Checking that failure(s) are order-dependent... failure appears to be order-dependent

Round 1: bisecting over non-failing examples 1-424 .. multiple culprits detected - splitting candidates (1 minute 26.35 seconds)
Round 2: bisecting over non-failing examples 1-212 .. multiple culprits detected - splitting candidates (1 minute 29.08 seconds)
Round 3: bisecting over non-failing examples 1-106 .. multiple culprits detected - splitting candidates (1 minute 26.57 seconds)
Round 4: bisecting over non-failing examples 1-53 .. multiple culprits detected - splitting candidates (1 minute 23.85 seconds)
Round 5: bisecting over non-failing examples 1-27 .. multiple culprits detected - splitting candidates (1 minute 29.03 seconds)
Round 6: bisecting over non-failing examples 1-14 .. multiple culprits detected - splitting candidates (1 minute 21.54 seconds)
Round 7: bisecting over non-failing examples 1-7 .. multiple culprits detected - splitting candidates (1 minute 28 seconds)
Round 8: bisecting over non-failing examples 1-4 .. multiple culprits detected - splitting candidates (1 minute 30.43 seconds)
Round 9: bisecting over non-failing examples 1-2 .. multiple culprits detected - splitting candidates (1 minute 28.99 seconds)
Round 10: bisecting over non-failing examples 3-4 .. multiple culprits detected - splitting candidates (1 minute 26.5 seconds)
Round 11: bisecting over non-failing examples 5-7 .. multiple culprits detected - splitting candidates (1 minute 54.99 seconds)
Round 12: bisecting over non-failing examples 5-6 .. multiple culprits detected - splitting candidates (1 minute 42.05 seconds)
Round 13: bisecting over non-failing examples 8-14 .. multiple culprits detected - splitting candidates (1 minute 46.97 seconds)
Round 14: bisecting over non-failing examples 8-11 .. multiple culprits detected - splitting candidates (1 minute 38.02 seconds)
Round 15: bisecting over non-failing examples 8-9 .. multiple culprits detected - splitting candidates (1 minute 35.84 seconds)
Round 16: bisecting over non-failing examples 10-11 .. multiple culprits detected - splitting candidates (1 minute 23.13 seconds)
Round 17: bisecting over non-failing examples 12-14 .. multiple culprits detected - splitting candidates (1 minute 23.11 seconds)
Round 18: bisecting over non-failing examples 12-13 .. multiple culprits detected - splitting candidates (1 minute 36.75 seconds)
Round 19: bisecting over non-failing examples 15-27 .. multiple culprits detected - splitting candidates (1 minute 29.76 seconds)
Round 20: bisecting over non-failing examples 15-21 .. multiple culprits detected - splitting candidates (1 minute 39.97 seconds)
Round 21: bisecting over non-failing examples 15-18 .. multiple culprits detected - splitting candidates (1 minute 35.21 seconds)
Round 23: bisecting over non-failing examples 17-18 .. multiple culprits detected - splitting candidates (1 minute 29.7 seconds)
Round 24: bisecting over non-failing examples 19-21 .. multiple culprits detected - splitting candidates (1 minute 29.41 seconds)
Round 25: bisecting over non-failing examples 19-20 .. multiple culprits detected - splitting candidates (1 minute 31.41 seconds)
Round 26: bisecting over non-failing examples 22-27 .. multiple culprits detected - splitting candidates (1 minute 25.67 seconds)
...
Round 183: bisecting over non-failing examples 181-186 .. multiple culprits detected - splitting candidates (1 minute 58.92 seconds)
Round 184: bisecting over non-failing examples 181-183 .. multiple culprits detected - splitting candidates (1 minute 44.64 seconds)
Build timed out after 5h0m0s

知道是什么导致 bisect 以这种方式表现吗？谢谢！

4

3 回答 3

5

这是一个错误，最近已修复并与rspec-core3.9.1一起发布。

于 2020-01-07T00:19:10.170 回答
5

我可以为您提供一些关于如何继续的指示：

首先 - 在任何时候你都可以Ctr-C处理并rspec输出迄今为止最小的命令来重现（希望它会比 {all_specs}.

您应该在超时之前尝试此操作。有可能要检查的可能组合的数量如此之大以至于超时（例如，如果它有更多时间，它将成功）。然后使用这个最小命令--bisect再次运行。

您还可以尝试--bisect=verbose更好地了解幕后发生的事情。

您的整个套件似乎非常快（约 2 分钟），所以超时很奇怪。仅在以特定顺序运行整个套件时出现故障的可能性很小。如果是这样的话 - 你基本上注定要失败。

于 2018-07-31T15:36:47.573 回答
3

对我来说，问题是--bisect如果你使用 spring 似乎什么都不做。即使是详细的，它也没有提供有用的输出。我的测试套件应该在 10 秒内运行。

于 2018-12-21T19:09:14.543 回答