It 'can' be done.. but you would need to write your own custom attribute that basically emits for client side and is validated server side. For me I just extract the validation code into a method and check on the server.
Something similar came up recently as well:
Prevent form from submitting when using unobtrusive validation in ASP.NET MVC 3
I wonder if one couldnt inherit from the remote attribute and add their own server side code them. hmm.. maybe I'll have to try this.
I would be happy though if someone here said they already did this : )