43

似乎有两种完全不同的测试方法，我想引用它们。

问题是，这些观点是 5 年前（2007 年）提出的，我很感兴趣，从那时起发生了什么变化以及我应该走哪条路。

布兰登守护者：

理论上，测试应该与实现无关。这导致不那么脆弱的测试并实际测试结果（或行为）。

使用 RSpec，我觉得完全模拟模型来测试控制器的常用方法最终会迫使你过多地关注控制器的实现。

这本身并不算太糟糕，但问题是它过多地关注控制器来决定如何使用模型。为什么我的控制器调用 Thing.new 很重要？如果我的控制器决定采用 Thing.create！和救援路线？如果我的模型有一个特殊的初始化方法，比如 Thing.build_with_foo？如果我更改实现，我的行为规范不应该失败。

当您有嵌套资源并且为每个控制器创建多个模型时，这个问题会变得更糟。我的一些设置方法最终会长达 15 行或更多行并且非常脆弱。

RSpec 的目的是将您的控制器逻辑与您的模型完全隔离开来，这在理论上听起来不错，但对于像 Rails 这样的集成堆栈来说，这几乎是违反规定的。特别是如果你练习瘦控制器/胖模型纪律，控制器中的逻辑量会变得非常小，而设置会变得巨大。

那么 BDD 想要做什么呢？退后一步，我真正想要测试的行为不是我的控制器调用 Thing.new，而是给定参数 X，它会创建一个新事物并重定向到它。

大卫切利姆斯基：

这都是关于权衡的。

The fact that AR chooses inheritance rather than delegation puts us in a testing bind – we have to be coupled to the database OR we have to be more intimate with the implementation. We accept this design choice because we reap benefits in expressiveness and DRY-ness.

In grappling with the dilemma, I chose faster tests at the cost of slightly more brittle. You’re choosing less brittle tests at the cost of them running slightly slower. It’s a trade-off either way.

In practice, I run the tests hundreds, if not thousands, of times a day (I use autotest and take very granular steps) and I change whether I use “new” or “create” almost never. Also due to granular steps, new models that appear are quite volatile at first. The valid_thing_attrs approach minimizes the pain from this a bit, but it still means that every new required field means that I have to change valid_thing_attrs.

But if your approach is working for you in practice, then its good! In fact, I’d strongly recommend that you publish a plugin with generators that produce the examples the way you like them. I’m sure that a lot of people would benefit from that.

Ryan Bates:

Out of curiosity, how often do you use mocks in your tests/specs? Perhaps I'm doing something wrong, but I'm finding it severely limiting. Since switching to rSpec over a month ago, I've been doing what they recommend in the docs where the controller and view layers do not hit the database at all and the models are completely mocked out. This gives you a nice speed boost and makes some things easier, but I'm finding the cons of doing this far outweigh the pros. Since using mocks, my specs have turned into a maintenance nightmare. Specs are meant to test the behavior, not the implementation. I don't care if a method was called I just want to make sure the resulting output is correct. Because mocking makes specs picky about the implementation, it makes simple refactorings (that don't change the behavior) impossible to do without having to constantly go back and "fix" the specs. I'm very opinionated about what a spec/tests should cover. A test should only break when the app breaks. This is one reason why I hardly test the view layer because I find it too rigid. It often leads to tests breaking without the app breaking when changing little things in the view. I'm finding the same problem with mocks. On top of all this, I just realized today that mocking/stubbing a class method (sometimes) sticks around between specs. Specs should be self contained and not influenced by other specs. This breaks that rule and leads to tricky bugs. What have I learned from all this? Be careful where you use mocking. Stubbing is not as bad, but still has some of the same issues.

I took the past few hours and removed nearly all mocks from my specs. I also merged the controller and view specs into one using "integrate_views" in the controller spec. I am also loading all fixtures for each controller spec so there's some test data to fill the views. The end result? My specs are shorter, simpler, more consistent, less rigid, and they test the entire stack together (model, view, controller) so no bugs can slip through the cracks. I'm not saying this is the "right" way for everyone. If your project requires a very strict spec case then it may not be for you, but in my case this is worlds better than what I had before using mocks. I still think stubbing is a good solution in a few spots so I'm still doing that.

4

3 回答 3

16

I think all three opinions are still completely valid. Ryan and I were struggling with the maintainability of mocking, while David felt the maintenance tradeoff was worth it for the increase in speed.

But these tradeoffs are symptoms of a deeper problem, which David alluded to in 2007: ActiveRecord. The design of ActiveRecord encourages you to create god objects that do too much, know too much about the rest of the system, and have too much surface area. This leads to tests that have too much to test, know too much about the rest of the system, and are either too slow or brittle.

So what's the solution? Separate as much of your application from the framework as possible. Write lots of small classes that model your domain and don't inherit from anything. Each object should have limited surface area (no more than a few methods) and explicit dependencies passed in through the constructor.

With this approach, I've only been writing two types of tests: isolated unit tests, and full-stack system tests. In the isolation tests, I mock or stub everything that is not the object under test. These tests are insanely fast and often don't even require loading the whole Rails environment. The full stack tests exercise the whole system. They are painfully slow and give useless feedback when they fail. I write as few as necessary, but enough to give me confidence that all my well-tested objects integrate well.

Unfortunately, I can't point you to an example project that does this well (yet). I talk a little about it in my presentation on Why Our Code Smells, watch Corey Haines' presentation on Fast Rails Tests, and I highly recommend reading Growing Object Oriented Software Guided by Tests.

于 2012-06-13T12:47:53.093 回答
9

Thanks for compiling the quotes from 2007. It is fun to look back.

My current testing approach is covered in this RailsCasts episode which I have been quite happy with. In summary I have two levels of tests.

  • High level: I use request specs in RSpec, Capybara, and VCR. Tests can be flagged to execute JavaScript as necessary. Mocking is avoided here because the goal is to test the entire stack. Each controller action is tested at least once, maybe a few times.

  • Low level: This is where all complex logic is tested - primarily models and helpers. I avoid mocking here as well. The tests hit the database or surrounding objects when necessary.

Notice there are no controller or view specs. I feel these are adequately covered in request specs.

Since there is little mocking, how do I keep the tests fast? Here are some tips.

  • Avoid excessive branching logic in the high level tests. Any complex logic should be moved to the lower level.

  • When generating records (such as with Factory Girl), use build first and only switch to create when necessary.

  • Use Guard with Spork to skip the Rails startup time. The relevant tests are often done within a few seconds after saving the file. Use a :focus tag in RSpec to limit which tests run when working on a specific area. If it's a large test suite, set all_after_pass: false, all_on_start: false in the Guardfile to only run them all when needed.

  • I use multiple assertions per test. Executing the same setup code for each assertion will greatly increase the test time. RSpec will print out the line that failed so it is easy to locate it.

I find mocking adds brittleness to the tests which is why I avoid it. True, it can be great as an aid for OO design, but in the structure of a Rails app this doesn't feel as effective. Instead I rely heavily on refactoring and let the code itself tell me how the design should go.

This approach works best on small-medium size Rails applications without extensive, complex domain logic.

于 2012-06-13T18:37:53.047 回答
8

Great questions and great discussion. @ryanb and @bkeepers mention that they only write two types of tests. I take a similar approach, but have a third type of test:

  • Unit tests: isolated tests, typically, but not always, against plain ruby objects. My unit tests don't involve the DB, 3rd party API calls, or any other external stuff.
  • Integration tests: these are still focused on testing one class; the differences is that they integrate that class with the external stuff I avoid in my unit tests. My models will often have both unit tests and integration tests, where the unit tests focus in the pure logic that can be tested w/o involving the DB, and the integration tests will involve the DB. In addition, I tend to test 3rd party API wrappers with integration tests, using VCR to keep the tests fast and deterministic, but letting my CI builds make the HTTP requests for real (to catch any API changes).
  • Acceptance tests: end-to-end tests, for an entire feature. This isn't just about UI testing via capybara; I do the same in my gems, which may not have an HTML UI at all. In those cases, this exercises whatever the gem does end-to-end. I also tend to use VCR in these tests (if they make external HTTP requests), and like in my integration tests, my CI build is setup to make the HTTP requests for real.

As far as mocking goes, I don't have a "one size fits all" approach. I've definitely overmocked in the past, but I still find it to be a very useful technique, especially when using something like rspec-fire. In general, I mock collaborators playing roles freely (particularly if I own them, and they are service objects) and try to avoid it in most other cases.

Probably the biggest change to my testing over the last year or so has been inspired by DAS: whereas I used to have a spec_helper.rb that loads the entire environment, now I explicitly load just the class-under test (and any dependencies). Besides the improved test speed (which does make a huge difference!) it helps me identify when my class-under-test is pulling in too many dependencies.

于 2012-06-13T21:35:04.937 回答