0

We have the following setup:

  • Visual Studio 2012 / SQL Server 2012
  • TFS 2010, Build Server runs unit tests automatically on build.
  • A solution with lots of code and 2 Test Projects (so 2 separate assemblies at build time), the tests are done with Visual Studio's unit test framework.
  • The first assembly contains all actual unit tests, and they all run ok. Within most tests, we write some data in the db, run some code, then clean the data.
  • We use the second assembly for database seeding. Since the tests and assemblies are loaded/executed in alphabetical order, we have a test in the second assembly whose name starts with "Z" (the assembly name also starts with "Z"). The build server logs also confirm that the Z test is the last to be executed. So this test only writes some data in the db. We preferred this for seeding versus using a DB project due to an existing infrastructure which reads data from Excel files and writes to SQL Server.

The problem is that when we run all tests on our dev machines, the Z test is executed last, and the seed data remains in the database. When the tests are executed on the build server, the Z test is also executed last, but the data doesn't remain in the db.

According to the SQL Server profiler, the inserts/create tables are executed, so the data gets in the db both on our machines and on the build server, but on the build server some delete/drop queries are executed right after the inserts. We couldn't tell where they are coming from.

We tried running only the Z test assembly and the data remains in the database. So the drops must be caused by the other assembly somehow. But how? And why is there a difference between the ways visual studio and the build server run the tests?

Has anyone encountered something like this before?

4

2 回答 2

2

Depending on the order of your unit tests is a dangerous thing. First it violates the whole idea of a unit test (they should be order independent) but it's also something you just can't guarantee.

It's a coincidence that your unit tests always run in the same order on your development machines.

If you need some code to run before your other tests run to seed your database, you can use the Assembly Initialize attribute. This way you can make sure this method always runs before your other tests.

One other thing to note, unit tests that use a database are actually integration tests. I would be very cautious using such a construction. I wrote a blog post about this some time ago that shows the kind of problems you can encounter but also points to some solutions: Unit Testing, hell or heaven?

于 2013-02-11T08:16:23.870 回答
0

We managed to fix it in the end. It turns out that the first assembly had some cleanup code that ran in a [ClassCleanup] method. The first assembly's ClassCleanup code ran after the Z test (from the second assembly). So the quick fix was just to replace the Z seeding test with an [AssemblyCleanup] method which did the seeding.

于 2013-02-11T09:49:26.747 回答