Disclaimer - I found this solution thanks to my friend Piotr who needed it for different use case, described below.

Recently I had to use multiple repositories on one server with different deploy keys. Solution suggested by github docs was not viable, because I could not alter hostname easilly, as it suggest. It turns out, that actually you can use different keys on per directory basis, when using git. It’s doable thanks to git conditional includes:

deploy@localhost:~$ cat .gitconfig
[includeIf "gitdir:~/investment/"]
        path = .gitconfig-ip

deploy@localhost:~$ cat .gitconfig-ip
[core]
        sshCommand = ssh -i ~/.ssh/id_rsa_ip -F /dev/null

In sshCommand -F /dev/null causes ~/.ssh/config to be ignored.

Another use case is to have different ssh keys for personal and proffessional stuff