Ok I have found the answer myself after several dead ends. In the end I fixed this unintentionally while fixing another issue. Basically we had just turned on the automatic execution of unit tests for our builds. The test would run sucessfully but then immediately the build would bomb out with a message saying it was unable to report to the build drop folder.
What was happening was that while the Build service runs under one account and has a set of rights; some of the functionality is actually driven through the TFSService account. fter wading a heap of permissions I had my tests being reported. Then I noticed that builds had started to trigger on check-ins; I can't tell you exactly which permission fixed this but hopefully this answer will at least set people down the right path.
One other note a few of the builds started failing due to conflicting workspace mappings - this was a separate issue that I resolved by deleting some obsolete workspaces using the Attrice Sidekicks for Team Foundation tool.
Hope this helps somebody else.