I have strange behavior using these stuff in Windows.
At first I've installed MinGW with MSYS. I added all necessary parts into %PATH%, having where ssh output as in MSYS istall directory. All compilers and unix-commands as ssh, ssh-keygen work just fine.
Then I donwloaded git for Windows (without MSYS), installed it using second option (to run git from command line). where git works fine, targeting at git install dir. Empty repos can be created, files can be added, local branches can be made.
Then I configured git as it said in official tutorial, setting keys in my home directory through windows command line, so on so forth.
And now - the juice. I can connect via ssh to github using windows command line. I can connect to github via ssh to github using git bash. In both cases it greets me telling about successful login. I cannot perform any push via command line neither via git bash - it rejects my public key in both cases.
What am I doing wrong?
UPD. Somehow it all started working. I have no idea, what happened. I just pressed random buttons and input random commands (just kidding). I am trying to reproduce this problem on virgin system to understand what gone wrong. But, that a pity for me, I suppose that problem was in misspell on one of git-configuration steps. =\