The Code Whisperer

Following on from yesterday’s blog entry, this entry looks at how to create the one-click build process.

First of all one very important point. Never, ever, ever build a component for distribution on a developer’s pc. The reason for this is simple. The developer’s pc will, more than likely, contain additional applications that could have a bearing on how the components are compiled. Shared executable libraries are all well and good, but it’s too easy for applications to update shared libraries and then your application to expect a certain version of a library, which then isn’t available on a client’s pc. When this happens the application either fails to run or behaves unexpectedly. I have seen this too many times to even consider taking a short cut.

Instead use “virtual pc” software such as Microsoft’s Virtual PC/Virtual Server or VMWare to create a clean environment to compile and produce installation files on. This clean environment should have the target operating system with current service packs and updates, and the development environment and distribution tool installed. The virtual pc need to have access to the source control system and possibly shared drives on the network or the host pc. Please consider the licence implications of the virtual pc - this is no different from installing on a physical pc.

Before you start you need to have a think about version numbers. There are a number of factors to take into account when determining version numbering policy:

• To make your life easier, every component released outside your organisation has a different version number
• If a release isn’t publicly available but has been released for internal testing consider whether to increment the version number or not. Some customers may be concerned if the version number jumps between releases, but often this is because they aren’t aware of how the software is built. We use version numbers in the format major.minor.build and increment the build number every time the development team go through the build and distribute process and release to the test team. By doing this we can be assured of the version on the test team’s test rigs
• If you release all components of a multi-component application then they should, in my view, have the same version number
• Version the database too, if your application has one. This will make it easier to know when to apply updates to the database and make sure that the application and database versions match. We make sure that the major and minor version number of the application and database match as there won’t be individual database changes on a build only on a minor or major version number change

To create the one-click build process follow these action points, but run through the process manually first and document it - you may need to build the product by hand one day and you’ll be glad you wrote it down.

• Work out directory structures required, taking into account whether you want to retain the executables and other installation files for each release (not necessarily every build, as there may be more than one build per release). You will need the following locations, which can be locally on the virtual pc, on a network or shared drive with the host pc:
  • The source code from the source control system
  • Executables and other files generated from the build process
  • Other files required for the installation, such as configuration files, database scripts and help files
  • Distribution files including release notes
• I would expect to have one master script that calls the individual elements. You could do this as a smart GUI, but a batch script is perfectly OK provided as much logging as possible is carried out, so that you can diagnose problems if something goes wrong. The script or application you create could also be used to create a nightly build process (which I’ll cover in a later post), so more batch like is beneficial. It’s useful to consider notification if a problem occurs, possibly by email or SMS. Create the shell of this to begin with and add or integrate the individual elements as you proceed.
• If you aren’t using a tool to automate the interaction with your source control system, you will need to write a script or application to extract the latest version of files, check files out and in and, if you want to label the source code, you need to be able to apply a label. Microsoft’s SourceSafe has a documented API so can be manipulated by a number of programming tools, including VBScript and various version of Microsoft’s Visual Studio. Although I’ve not tried it, you could use non-Microsoft development tools such as RealBasic and Delphi, but the chances are if you’re using SourceSafe you’re using Visual Studio in any case. Other source control systems such as Subversion also have APIs or cam be accessed via WebDAV extensions. So, first task is to create scripts or an application to perform the previously mentioned tasks on your source control system in a batch environment. At this stage in the build you will need to extract the latest version of source files, apply a build started label if you are labelling source control and check out files if applying a version number change.
• Next you need to build the components, changing the version number if your version numbering policy dictates this. How this is done is dependant on the development tool being used and applications that have more than one executable component will require a compile order. Some development tools, such as RealBasic, create one executable, so this step is much easier. If you are using a build tool it should work out component dependencies for you and compile in the correct order.
• Any files checked out need to be checked back in and if labelling a build complete label needs to be applied.
• Create a script to copy the executable files and any other files needed for the installation to the location determined earlier.
• Use the distribution tool to create the installation file(s) in the directory determined earlier.

You now have a distribution that can be tested and released. Once you have a public release I highly recommend you make some backups:

• On a CD or DVD burn the source code, build scripts, other installation files and if you have them project files from the build and distribution. Ideally keep this backup away from the office
• Depending on your licence agreement, make a backup of the virtual pc used and store away from the office

The reason for backing up is two-fold. Firstly there’s the normal disaster recovery aspect and secondly if you have to go back to a previous release, make a change and re-release that version having backups will make the job a lot easier and error prone.

I’m not a big fan of fast food restaurants such as McDonalds, but one thing they do very well is consistency. It doesn’t matter what country you are in, the restaurant has the same feel about it, albeit with possible a different language. The layout is very similar and the ordering process identical; if you have been in one restaurant you know exactly how it works.

However, this isn’t done just to make you feel good. Consistency is an important part in making a decision; if you know what to expect it can make a decision easier to make, particularly when it comes to food and trying to satisfy the catering needs of a young family. The other reason is speed. If you know what to do you waste less time, straight in, order, eat out. Fast food restaurants work on a high turnover of customers to make money, so everything they do to support this boosts the bottom line.

We spend several weekends and a week each year in the Dorset town of Swanage. Aside from the fact it’s a picturesque part of the country and only a couple of hours drive, the main reason we do this, and go to the same campsite every time is that we know where things are and how it works. We don’t waste time finding the supermarket or the best car park, we know that so it maximises the time we have away. We have a pot of 20p pieces for the shower, we don’t have to get change from the local shop. A weekend feels like a weekend, not a few hours in between trying to get the essential things done.

Back to software development. Let’s take a quick overview at the development lifecycle:

• Business requirements
• Technical design
• Coding
• Testing
• Bug fix
• Release
• Usage
• Support

Each of these steps requires some form of documentation and, with the possible exception of business requirements, will require reference to other documents in the lifecycle. It’s highly unlikely that one person would be responsible for each step in the lifecycle, and it’s possible that each step will be carried out by different people.

So what you don’t want is a free for all with document layout. The person looking into a bug discovered in testing wants to open up the documentation and know where to find the information they are looking for, not waste hours searching through documents.

I talked about managing programmers being like herding cats in an earlier post. This is also true with document layout. Each programmer or analyst will produce a different layout, based on their experiences of the past, which also changes over time compounding the problem. I’ve seen too much time wasted in the bug fix phase of a development looking for and through documents for information when the actual fix took minutes, or wasn’t a bug just a misunderstanding of the business requirements by the tester.

What can we do to improve the situation? Here’s my action plan for managing design documentation.

• Create templates for each type of document produced in the lifecycle. The documents must feel like the same family and as much as possible be laid out the same. Pay attention to the order of information in the document, this is key to locating details in a completed document. Use a font that’s easy to read on the screen and when printed - choosing a font that doesn’t display well on the screen will lead to more documents being printed, something you want to avoid for several reasons. Don’t forget to include copyright and distribution notices if they apply. Involve team members in the design, but have the final say. The templates will evolve over time - this is fine.
• The document template used is part of the document’s review - if it doesn’t use the correct template the document fails the review.
• Make sure the templates and the documents are easily accessible and searchable and that the design and development teams understand the importance of consistent documentation and know where to find templates and documents. There are a number of document management tools available, but sometimes a simple but well though out directory structure is sufficient. Make tools such as Agent Ransack available for searching. If it will make the location of documents easier create a document index, which can be a spreadsheet or another document.
• Consider version control software. Subversion is a good tool and using Tortoise SVN it integrates with Windows Explorer. Numerous Subversion hosting services exist, so as a bonus the documents are backed up away from the office.
• Recording bugs must be easy and the information provided comprehensive. Whatever system is used it must be possible to attach screen dumps, screen video and log files. Assuming the bug is found as part of a test plan, references to the test plan and the part of the business requirements being tested must be included. The developer correcting the bug must be able to add information about the fix applied as this may be useful in the future. I’ll cover bug tracking in more detail in a future post.
• Documents must be updated as the development progresses if they are to be of any use in the future. And don’t forget the future is just as far as the next step in the lifecycle.

If you are planning to outsource a project the quality of documentation is vital to the speed and success of the development. I’m in the process of putting together a Software Development Toolkit for small businesses that helps small businesses develop software, whether using a small in-house team or outsourcing. More on this later.

In 2005 my good friend Mark Evans and I had a stand at the NEC in Birmingham’s Classic Motor Show. Mark was launching the DVD to his latest series on Discovery RealTime An MG is Born, and I was launching our Restoration Manager software.

We had quite a large stand and wanted to have large images of the MG’s restoration and a Restoration Manager cover shot. Having found a company that could produce images on foam boards at a realistic price, I set about producing PDFs of the images. For various reasons to do with producing a large PDF with good quality images, this turned out harder than expected, until I found pdfFactory from FinePrint Software. Since then I’ve used pdfFactory exclusively and have recommended it to several of my private clients.

However, I was away from the office and needed to convert a Microsoft Word document to a PDF and didn’t have access to my trusty pdfFactory. A quick Google and I found Doc2PDF Online, which provides a free service converting files under 2MB to PDF format, which it then emails to you. I was so impressed I used it to convert the Generic Coding Standards posted yesterday.

Please note I have no connection with these two companies or recive any benefit in recommending them - I’m just a happy user!

I assume you’re creating coding standards where none exist for the company or development team. These are general tips and not aimed at a specific programming language.

Our aim in having coding standards is making code easier to understand and therefore cheaper to maintain. The document should be compact and only contain rules that will be applied at a code review. Anything else needs to be in an architectural, guidelines, procedures or specification document.

Coding standards are often used to prevent coding horrors and promote best practises, but to be honest these shouldn’t be necessary. If you have coders that are happy sprinkling their work with GOTO statements I would advise some retraining rather than enforcing standards. Standards documents are often littered with rules such as no method longer than a page, often because it’s been added in a standards document way back in the past or has been found on the Internet. Generally programmers will naturally follow these sorts of design guidelines, but taking this example, there are times when a long method is the best solution.

Best practises should be a team presentation and discussion, and if they need documenting put in the guidelines document rather than in with standards.

Rules surrounding user interface are slightly tricky to position. For some developments they could be placed in the coding standards, but for development teams with different products there may need to be one per product. I would always advise to have user interface design rules in a separate document.

Creating standards from scratch can be an uphill struggle. Here are some techniques that have worked for me over the years:

• Remember the standards can be changed, they don’t have to be perfect first time, so don’t spend too long creating them. However take into account the effect of changing existing standards below
• If you have existing standards from another development then use these as a starting point
• You may require general standards and then specific standards for each language used
• You need to get the right number and balance of people deciding the standards, so create a small standards committee of responsible and mature developers that will be able to agree rather than argue. It’s best to have an odd number of people so that a vote is always decisive
• Once complete issue the standards to the rest of the team, if possible with a short presentation. It’s absolutely crucial to get buy in from the rest of the team for the standards to benefit the development work
• Make it clear to the development team that the published standards are to be adhered to, but they are flexible and can be changed. Put into place a process where changes can be suggested and either accepted or declined by the standards committee. If declined make sure there is a good reason, which is communicated back to the person requesting the change. Remember people don’t like to be ignored, so feedback is crucial
• Wherever possible explain why a rule has been applied. This is less important for, say, variable prefixes, but vital if the standard prevents the use of certain parts of a programming language. This stops the document becoming a rule book, but instead a source of information. There may be occasions where the standards committee just doesn’t like an aspect of a programming language. If this is the case, and there is validity in including the rule, just say so

Without going into the semantics of programming languages and IDEs, here are topics that the standards should cover. Remember it’s not what you decide so much as everyone knowing. With coding standards and procedures (a future blog topic) any developer should be able to jump right in and add to a project.

• Naming standards for constants, variables, class names and methods
• Comment block template for files, classes and methods
• Comments for code changes. If you need to merge code changes between code streams or need to review a change made, this will prove invaluable. From experience this needs to include the full date of change, the name of the developer, a reference to the bug or enhancement (i.e. the unique number in the bug tracking tool) a description of the change. The start and end of the changed block of code should be marked
• Requirements for removal of code - is it deleted or commented out?
• If appropriate for the IDE the number of spaces for indentation. Sounds like a small point, but I’ve seen people struggle unnecessarily trying to amend or merge code where the indentation isn’t right.
• Location of compiled units may need specifying, particularly in a system that has multiple executables. For COM systems where there is the concept of binary compatibility, specifying the location of the compatibility object is important
• Tracing and error handling need to be defined in the standards, although it is bordering on architecture. Consistency is important in these areas and must be adhered too. This will be different for different programming languages.

And some items to avoid putting in the standards, as they are guidelines, procedures or architecture so document them separately.

• How to access data - put it in the architectural document
• Use of the source control system - put it in the procedures manual
• How to build and/or deploy the application - put it in the procedures manual

The Effect of Changing Standards Changing existing standards should be approached with caution. I’ve seen a the effect of changing the variable prefix for a data type - total chaos. Fine if all code will be brought into line with the new standard, but if not at best you have new classes and methods using the new prefix and existing code using the old prefix. And what happens if you add new variables to the existing code; do you use the old prefix to fit in with the other variables or use the new prefix?

We recently had a contretemp with a scam artist and our Chuckie Bags website. Artist is the wrong word really, as it implies some sort of skill and imagination, both of which were severely lacking in this case.

It’s not unusual for us to receive a request via email for bulk orders and we’ve had a few where they sender clearly hasn’t looked at the website in any great detail. This email stated he (and I’m assuming he) wanted to “purchase some products” did we ship to Australia and do we take credit cards, both questions should have been answered by the website.

Next he stated he wanted 200 wallets and how much would this cost with shipping.

Then how much for just the wallets as he has a courier picking up other products in the UK.

And then the final email, could we charge his card an extra