The Code Whisperer

I really hope this is a topic you can ignore, but I’m going to cover it anyway.

In the late 1990’s I was involved in a major project for a prestigious motor manufacturer. Having very good cashflow they weren’t adverse to spending a bit of money, so I was horrified when I found out that the source control server, with 6 months worth of design and development work on it, firstly wasn’t being backed up and secondly wasn’t in the server room but in the development team’s office situated under an air conditioning pipe that had developed a leak.

Let’s have a look at some data loss scenarios and how they could be recovered from, assuming the appropriate backup policy was in place:

Developer checks in the wrong code
Roll back to the previous version in the source control system. No loss of work.

Developer loses or corrupts checked out code
Revert to the previous night’s backup, loses up to one day of work.

Developer machine corrupt
Re-image the developer machine, get the latest source from the source control and restore the previous night’s backup, loses up to one day of work.

Source control server corrupt
Restore the source control from last night’s or later backup. Checked in code since backup may reside on developer’s machines. Worst case scenario is that each developer loses one day of work each, although this is highly unlikely.

Office destroyed
Build new source control server and restore the source control content from the last off site backup. Build new developer’s machines using image of developer’s machine held off site. Get latest source control on to developer’s machines. The amount of work each developer loses depends on how often backups are sent off site, but this could be as little as one day.

This last scenario is drastic but does happen. To be honest if the office has been destroyed there will be other factors to consider, but with the right backup strategy and a trip to the computer retailer you could be up and running really quickly.

With backing up across the Internet now available and very cheap, it’s possible to build a backup strategy that offers maximum protection with the minimum of fuss. Backups are only important to people when things go wrong, so generating them must be as effortless as possible.

Here is our backup strategy:

• Source control exists on a server in our office. Code is checked in as soon as possible provided it compiles
• The source control repository are backed up across the Internet to Data Deposit Box. The Data Deposit Box software runs on the server and detects changes to the repository, extracts, compresses and encrypts them, then securely transfers them to their secure data centre where they are stored in encrypted form
• Source code changes not checked in are backed up at the end of the day to a file server in our office. The developer’s machine has backup software that backs the files across the network and then shuts the machine down. This backup also includes Outlook PST files
• Other files such as documents and images are held on network file servers never on individual workstations. Documents and selected other files are backed up to the Data Deposit Box servers using the same technique as for the source code repository
• A disk image of a developers machine has been taken and is kept on a file server
• Copies of virtual machines are kept on the file server and backed up from developers machines weekly
• File servers are backed up completely to external hard drives, which are stored off site
• USB memory sticks are used to hold backups of this blog, software and licence information for developers products and the latest release code for our products. These are backed up to a files server and tend to go everywhere with me
• Software installation CDs are kept in a fire safe in the office

I believe this is a pretty comprehensive backup strategy and with the use of Internet backup I’m confident that we could be up and running in a different office in a matter of hours if total disaster did strike.