Windows SSH keygen

Just created my key without any problem (Seven Ultimate 64bits, msysgit 1.6.5.1.1367.gcd48)

$ ssh-keygen -C "vonc@xxxx" -t rsa
Generating public/private rsa key pair.
Enter file in which to save the key (/c/Users/VonC/.ssh/id_rsa):# just press enter
                                                                # to accept the
                                                                # default location
Enter passphrase (empty for no passphrase):                     
Enter same passphrase again:
Your identification has been saved in /c/Users/VonC/.ssh/id_rsa.
Your public key has been saved in /c/Users/VonC/.ssh/id_rsa.pub.
The key fingerprint is:
xx:yy:zz:aa:bb:cc:... vonc@xxxx

With the result:

VonC@P ~/.ssh
$ ls -alrt
total 10
-rw-r--r--    1 VonC Administ      642 May 23 21:47 known_hosts
drwxr-xr-x   43 VonC Administ    16384 Jun 15 17:01 ..
-rw-r--r--    1 VonC Administ      398 Jun 19 16:14 id_rsa.pub
-rw-r--r--    1 VonC Administ     1675 Jun 19 16:14 id_rsa
drwxr-xr-x    2 VonC Administ        0 Jun 19 16:14 .

Could you check in your bash session what value your $HOME environment variable is set?

VonC@P ~/.ssh
$ env|grep HOME
HOMEPATH=\Users\VonC
HOME=/c/Users/VonC     # <=== this must be correctly set
HOMEDRIVE=C: