There are a ton of drivers & famous applications that are not available in 64-bit. Adobe for instance does not provider a 64-bit Flash player plugin for Internet Explorer. And because of that, even though I am running 64-bit Vista, I have to run 32-bit IE. Microsoft Office, Visual Studio also don't ship in 64-bit AFAIK.
Now personally, I haven't had much problems building my applications in 64-bit. I just have to remember a few rules of thumb, e.g. always use SIZE_T instead of UINT32 for string lengths etc.
So my question is, what is preventing people from building for 64-bit?