In this post I will share a few techniques I use when working with ssh and scp. ssh is a fantastically versatile program that accepts many different arguments and options. Memorizing all these can be a burden, though. The permutations can get out of hand quickly when combined with various usernames, hosts, and keys. Until adopting these techniques, it always seemed that my memory would fail during peak flow and I would find myself Googling and man diving — my least favorite pastime.
This post is geared towards people working on macOS and Unix servers.
The Hostname Problem
I work on dozens of projects across dozens of servers. Each server has a different hostname. Sometimes the hostnames are explicitly named like, -<{dev|uat|prd}..com. Other times they’re named with a date like vh07182014..com. And sometimes they’re totally random because the server is a decade old. This gets even worse when different ports are used. Essentially I could never remember the hostname, nor did I particularly want to.
In order to simply ssh into a server, I would need to consult emails, notes, and system logs just to find out what the server was called. Then I’d type something like this:
ssh myusername@system-dev.someserver.com -p 3080
This is too much to type and too much to remember.
My first thought was to start adding these to my .bash_profile as aliases. This works OK, but there is a better option (especially if you work with scp; more on that later). It turns out we can save all these ssh configurations in the ~/.ssh/config file:
Host pidev Hostname sl20170111.someserver.com Port 3080Host piprd Hostname sl20170113.someserver.com Port 3082Host pistg Hostname sl20170112.someserver.com Port 3081
You can set the Host alias to be whatever you want, but now we have another problem. To combat this, I have adopted a pretty strict convention of <{dev|stg|uat|prd}>. This personal convention both aids in mnemonics and gives me the added benefit of forcing myself to explicitly state what environment I’m about to enter. It is yet another subtle layer of safety to make sure I don’t do something I may regret on a production system. If I can manage to remember what the project is called (I hope so) and whether I should be working on production or not (I really hope so) I can easily ssh without consulting notes or trial and error. I let the config file do all the hard work.
Now instead of typing:
ssh myusername@sl20171111.someserver.com -p 3080
I can type:
ssh myusername@pidev
The User Problem
Most of the servers I work on are managed by my company, so I have the luxury of a consistent username everywhere. But exceptions do come up and the ~/.ssh/config file can help here, too.
At the bottom of the ~/.ssh/config file I’ve added my usual User and some other defaults I want applied to every ssh session in a wildcard Host. Note that individual User entries will override the * entry for instances where you may have a different username:
Host mysterydev Hostname ml20060920.someotherserver.com Port 3080 User anotherusernameHost pidev Hostname sl20170111.someserver.com Port 3080Host piprd Hostname sl20170113.someserver.com Port 3082Host pistg Hostname sl20170112.someserver.com Port 3081Host * User myusername ServerAliveInterval 300 ServerAliveCountMax 36
If I’m working on mysterydev, then anotherusername will be used. But everywhere else myusername will be used. ssh knows the Hostname, Port, User, ServerAliveInterval, and ServerAliveCountMax from just the Host. So if I want to work on pidev, I can type:
ssh pidev
The Multiple SSH Key Problem
I have multiple ssh keys. To my knowledge, ssh isn’t smart enough to add any additional keys beyond id_rsa by default. Embarrassingly, I did something like this almost every day:
$ ssh mysteryprodPermission denied (publickey).$ ssh-add ~/.ssh/key_i_forgot_to_addCould not open a connection to your authentication agent.$ eval ssh-agentAgent pid 15419$ ssh-add ~/.ssh/key_i_forgot_to_addIdentity added: .ssh/key_i_forgot_to_add (.ssh/key_i_forgot_to_add$ ssh mysteryprod
Try to login. Fail. Try to add my other key. Fail. Fire up the agent. Add my other key. Try to login again. The shame was too much to bear and one day it occurred to me to add ssh-add -K ~/.ssh/idrsa ~/.ssh/keyiforgotto_add 2> /dev/null to my .bash_profile. It worked, but was a little hacky for my tastes.
Unsurprisingly, the config file can help here too. Simply add an IdentifyFile to a Host entry:
Host mysteryprod Hostname ml20060925.someotherserver.com Port 3080 User anotherusername IdentityFile ~/.ssh/key_i_forgot_to_add
Now ssh knows to use this particular key for connections to this server.
Bonus: The SCP Problem
I use scp to transfer .tar.gz archives around regularly. Say I want to transfer a tarball somewhere else:
scp foo.tar.gz myusername@sl20170111.someserver.com:/home/myusername -p 3080
Certainly there must be a better way. scp is built on top of ssh and, as it turns out, is pretty smart and can read the ssh settings. This means all the work we put into our config file also works with scp:
scp foo.tar.gz pidev:
The trick here is adding the colon after the Host. Without this, scp will think you’re trying to copy the tarball to your local system. It will oblige and you’ll have a second tarball sitting next to the first, just named pidev. If you don’t want the tarball to land in your home directory on the remote server, you can add an absolute path after the colon:
scp foo.tar.gz pidev:/var/www/dev
Conclusion
Ever since I configured my ~/.ssh/config this way, working with ssh and scp is actually quite pleasant. I periodically weed out my ~/.ssh/config file by removing old Hosts I no longer need. I also try to keep it in alphabetical order so I can quickly maintain it. By prefixing all my projects the same way, (in the above examples with pi) all the project environments naturally clump together.
