In the original Presentation Model pattern described by Martin Fowler, the view "asks" the view-model how to display itself. This seems to favor putting color and size properties on your view-model rather than triggers in your view. The application of this pattern within WPF, however, is somewhat different. In WPF, you typically define how the view looks by using Styles and DataTemplates in the view. Returning specific colors directly from the view-model would be contrary to this approach. So the short answer is: no, don't put color properties on your view-model.
Also within the original Presentation Model pattern, the view-model is an abstraction of the view. So instead of it returning the exact color, it would be preferable to return a "key" that the view can then use to look up the actual color. For example, instead of PersonViewModel.FaceColor returning Red, you'd have PersonViewModel.Mood returning Angry. The view could then use a Style or DataTemplate trigger that translates this to the actual Red color.
So that's my answer and I'm sticking by it, but it's also interesting to consider arguments in the other direction. For one, putting color properties on your view-model is still unit-testable, which seems has become the primary critera for what's okay in the view-model.
Remaining "agnostic" to the view technology isn't a huge factor in either direction. The goal to maintain binary compatibility of view-models with other view technologies is realistic only within the XAML family. Moving all your view-models to their own project which lacks a direct dependency on WPF is a nice idea. But you would have to exclude anything that uses ICommand, or make an exception for a WindowsBase.dll reference. Practically speaking, though, it won't buy you much. We're pretty much glued to Micrsoft technologies! If you decide to port to another GUI framework, then my guess is you're looking at source code conversion. I'm waiting to see if Microsoft goes under before I try it ;) Porting could include changing your color types, if you decided to put them in your view-model. Although not a reason against color properties on your view-model, it's not necessary a reason for it either.