Speeding up interations with the remote git repository

Why?

I interact with git a lot.  Even my prompt interacts with the remote git repository. In my case the git repo is now located 1000km away which means I noticed the slow down compared to when it was on the LAN.

Before measurements

Using a simplistic

time git pull

took between 1.5s and 2.2s

After measurements

The same command typically takes < 1s.

The magic

These steps worked on Centos 6.  Edit your ~/.ssh/config and include the following

ControlMaster auto
ControlPath /tmp/ssh_mux_%h_%p_%r

Then start up a persistent ssh connection to the machine housing the git repo.  In my case the machine itself is called git.

nohup ssh -N git 2>&1 >/dev/null &

If we had a more modern version of ssh we could put

ControlPersist yes

into the ssh config and then dispense with the explicit opening of a persistent connection.

A little deeper

To see the effect of the persistent connection I wrote a little timestamp script.

#!/bin/sh
while read line
do
DATE=$(date +%Y%m%dT%H%M%S.%N)
echo $DATE $line
done

Then I can run

ssh -v git echo hello world 2>&1 |timestamp

to see what is going on.

Prior verbose output

20130903T125138.904422582 OpenSSH_5.3p1, OpenSSL 1.0.0-fips 29 Mar 2010
20130903T125138.905558991 debug1: Reading configuration data /data2/home/geoff/.ssh/config
20130903T125138.906317488 debug1: Reading configuration data /etc/ssh/ssh_config
20130903T125138.907019567 debug1: Applying options for *
20130903T125138.908487991 debug1: auto-mux: Trying existing master
20130903T125138.909272700 debug1: Control socket "/tmp/ssh_mux_git_22_geoff" does not exist
20130903T125138.912443346 debug1: Connecting to git [172.16.0.5] port 22.
20130903T125138.949749521 debug1: Connection established.
20130903T125138.950558206 debug1: identity file /data2/home/geoff/.ssh/identity type -1
20130903T125138.951323692 debug1: identity file /data2/home/geoff/.ssh/id_rsa type 1
20130903T125138.952002463 debug1: identity file /data2/home/geoff/.ssh/id_dsa type 2
20130903T125139.000878506 debug1: Remote protocol version 1.99, remote software version OpenSSH_5.3
20130903T125139.001620530 debug1: match: OpenSSH_5.3 pat OpenSSH*
20130903T125139.003583486 debug1: Enabling compatibility mode for protocol 2.0
20130903T125139.004404582 debug1: Local version string SSH-2.0-OpenSSH_5.3
20130903T125139.005180030 debug1: SSH2_MSG_KEXINIT sent
20130903T125139.040156868 debug1: SSH2_MSG_KEXINIT received
20130903T125139.040892755 debug1: kex: server->client aes128-ctr hmac-md5 none
20130903T125139.041596022 debug1: kex: client->server aes128-ctr hmac-md5 none
20130903T125139.042315317 debug1: SSH2_MSG_KEX_DH_GEX_REQUEST(1024<1024<8192) sent
20130903T125139.043097405 debug1: expecting SSH2_MSG_KEX_DH_GEX_GROUP
20130903T125139.150982134 debug1: SSH2_MSG_KEX_DH_GEX_INIT sent
20130903T125139.151771961 debug1: expecting SSH2_MSG_KEX_DH_GEX_REPLY
20130903T125139.203216480 debug1: Host 'git' is known and matches the RSA host key.
20130903T125139.203971960 debug1: Found key in /data2/home/geoff/.ssh/known_hosts:168
20130903T125139.204714146 debug1: ssh_rsa_verify: signature correct
20130903T125139.205477313 debug1: SSH2_MSG_NEWKEYS sent
20130903T125139.206271418 debug1: expecting SSH2_MSG_NEWKEYS
20130903T125139.206934146 debug1: SSH2_MSG_NEWKEYS received
20130903T125139.207669249 debug1: SSH2_MSG_SERVICE_REQUEST sent
20130903T125139.329978037 debug1: SSH2_MSG_SERVICE_ACCEPT received
20130903T125139.373120380 debug1: Authentications that can continue: publickey,password
20130903T125139.373917032 debug1: Next authentication method: publickey
20130903T125139.374664056 debug1: Offering public key: /data2/home/geoff/.ssh/id_dsa
20130903T125139.411515327 debug1: Server accepts key: pkalg ssh-dss blen 433
20130903T125139.539400533 debug1: Authentication succeeded (publickey).
20130903T125139.540170228 debug1: channel 0: new [client-session]
20130903T125139.541874106 debug1: setting up multiplex master socket
20130903T125139.542617874 debug1: Entering interactive session.
20130903T125139.688853716 debug1: Requesting authentication agent forwarding.
20130903T125139.689606730 debug1: Sending environment.
20130903T125139.690311631 debug1: Sending env LC_COLLATE = C
20130903T125139.691041155 debug1: Sending env LANG = en_AU.utf8
20130903T125139.691767747 debug1: Sending env LC_TIME = en_AU
20130903T125139.692502224 debug1: Sending command: echo hello world
20130903T125139.975169558 hello world
20130903T125139.975877869 debug1: client_input_channel_req: channel 0 rtype exit-status reply 0
20130903T125139.976613148 debug1: client_input_channel_req: channel 0 rtype eow@openssh.com reply 0
20130903T125139.977391876 debug1: channel 0: free: client-session, nchannels 1
20130903T125139.978166529 debug1: fd 1 clearing O_NONBLOCK
20130903T125139.978881806 Transferred: sent 2480, received 2456 bytes, in 0.4 seconds
20130903T125139.979670233 Bytes per second: sent 5685.5, received 5630.4
20130903T125139.980442841 debug1: Exit status 0

And After

20130903T125207.813170201 OpenSSH_5.3p1, OpenSSL 1.0.0-fips 29 Mar 2010
20130903T125207.813914688 debug1: Reading configuration data /data2/home/geoff/.ssh/config
20130903T125207.814665734 debug1: Reading configuration data /etc/ssh/ssh_config
20130903T125207.815460302 debug1: Applying options for *
20130903T125207.816934510 debug1: auto-mux: Trying existing master
20130903T125208.118397703 hello world

What a world of difference!

Advertisements
Tagged with: , ,
Posted in Uncategorized

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: