ssh-agent and 'ssh -A' is also useful if you have to login to one machine to access another, without having to copy your private key to the first machine.

For example if you login remotely to a machine, and want to access a git repository on another:

  $ eval `ssh-agent -s`
  $ ssh-add ~/.ssh/id_<yourkey>
  $ ssh -A <firstserver>
  you@firstserver$ git clone git+ssh://<secondserver>/path/to/repository

I use agent forwarding often, but you still need to be careful, especially if you forward your agent to a machine not under your control. From the ssh man page:

Agent forwarding should be enabled with caution. Users with the ability to bypass file permissions on the remote host (for the agent's UNIX-domain socket) can access the local agent through the forwarded connection. An attacker cannot obtain key material from the agent, however they can perform operations on the keys that enable them to authenticate using the identities loaded into the agent.

Consider using a dedicated key for each of those circumstances, set sane defaults in your ~/.ssh/config on all machines, and be very careful about what ends up in any of your ~/.ssh/known_hosts files, as they provide a road map to other destinations.

The ssh_config HashKnownHosts option hashes the contents of the known_hosts file, making it intractable to get a list of hosts. But of course your shell history will still provide it.