Set up a ssh client on windows can be awful, specialy when you want to push/pull or to clone a git repo.
On my machine i wasn't able to clone a git repo via ssh because i had to use an non-standard ssh port. Why should this become a problem? When i exec the following line on one of my linux machines it worked:
git clone ssh://git@hostname.com:5632/Compilenix/repo.git
On my Windows it doesn't work, saying something about a refused ssh-key...
The solution, which worked in my case, was to write a ssh client config on my Windows for "hostname.com" into "/C/Users/USERNAME/.ssh/config".
Host hostname.com
Port 5632
User git