The Code Whisperer

Since 1981, aside from my own office at Operations Support, I’ve seen a very few outstanding offices and some some OK offices, but most have been very bad from a productivity point of view. As the office environment is an important aspect of developing high quality software, I was very interested in a recent article by Joel Spolsky in Inc magazine

You can read the entire article called How Hard Could it Be?: Adventures in Office Space, but this bit really caught my eye:

“There will be a reception area with a dry creek of stones and pebbles and plants that will make a great first impression on our guests. There will be a big lunchroom, because we all eat together, as well as a coffee bar, a lounge, a 180-gallon saltwater aquarium, the aforementioned shower, a library with reclining chairs for naps, two private meeting rooms, 20 private offices for programmers, 23 adjustable-height workstations for everyone else, Wi-Fi, a big screen for movies and video games, and enough glass to build the world’s largest ant farm. We will have some room to grow, finally. And in two years, if all goes well, it will be too small for us.”

Joel explains that he needs developers of the caliber that could choose to work at Google or Microsoft so needs an office environment that is as good as, or better, that these two software giants. Sounds like he’s done it.

Unachievable schedules have blighted the industry since the dawn of software development. Here’s some reasons why this happens:

• Schedule set before requirements agreed and/or understood
• Pressure from customers
• Inexperienced staff, in any combination of the industry, application or development tool
• Use of leading edge technology
• Poor estimation
• Estimate overridden by senior management
• Features added but the plan not amended
• Low estimates produced as a challenge or as an incentive to work harder
• Poor measurement of progress
• Fixed deadline, such as a sporting event, new tax year or introduction of regulations

Schedule set before requirements agreed and/or understood - This is particularly prevalent in software houses, where the primary goal is to get a customer to sign up for work. The software house doesn’t want to spend money before they get the sale, so only sufficient scoping to sell the software or enhancement is carried out. In an ideal world the software house would give a rough estimate, which would become more accurate as the analysis and technical design progressed. The problem is that customer want to know exactly when the software is going to be ready before they sign and the sales team can use the time taken to develop the software as a bargaining point, however, as we all know, this is a bad idea. So what do we do? Ideally invest time in having senior analysts and developers work out an estimate for the work, add some contingency and present that as a non-negotiable estimate that will be refined during the analysis and technical design phase. Make sure, however, that at the end of technical design the remainder of the schedule is as accurate as you can make it. Now this approach will lose sales, but for sales gained it is much more likely that the software will be delivered on time. Quite frankly a software house that has a reputation for delivering on time and in budget is going to attract the work most people want to do.

Pressure from customers - reducing the time leads to cut corners, software with bugs and demoralised development teams. The customer that suggests removing a month from a development schedule wouldn’t dream of suggesting to a surgeon that their knee operation must be an hour shorter, so why do the same with software development. My advice is it this occurs before the sale is made and you are confident with your rough estimate, walk away. Easy to say, harder to do, but once you’ve reduced your time estimate to the potential customer with out good reason, for example increasing development resource where it makes a difference or reducing functionality, your estimates lose credibility.

Inexperienced staff, in any combination of the industry, application or development tool - one of the biggest assets existing developers have is in the industry and applications your team produces. A high turnover of staff, or resourcing using a pool of developers means that a proportion of the developers may not be as productive to begin with as others. Obviously this depends on whether an existing application is being modified and how detailed the technical design is, but it can be an overlooked factor in scheduling.

Use of leading edge technology - using the latest and greatest is a developer’s dream, but can cause all sorts of delays due to bugs in the new technology or its implementation with the rest of your environment. For example I spent ages trying to get Visual Basic 5, RDO and Oracle stored procedures to work efficiently. It turned out due to an incompatibility between RDO and Oracle, with both companies blaming the other for the inefficiency. There is something to be said for using an established technology, mostly because most problems will have been discovered, overcome or not, but details available on the Internet.

Poor estimation - developers frequently underestimate the amount of time to do things and project managers must be aware of this and adjust accordingly. Overruns from one project need to be analysed and used when estimating the next. It’s important to learn from project plans, whether they are correct or not but in order to do this it’s vital that the time taken to complete tasks is accurately captured, without the developer feeling they have failed if the overrun is due to poor estimation.

Estimate overridden by senior management - senior management can reduce estimates for a variety of reasons and mostly it’s a bad thing. The only way to resolve this is the respect and relationship senior management has for the project manager producing the estimate.

Features added but the plan not amended - there seems to be a stigma about changing a plan once it’s underway. Some added features will have little or no impact on the schedule provided they are introduced at the right time, but most additional work will have some impact. Not to replan is madness.

Low estimates produced as a challenge or as an incentive to work harder - some developers, particularly those that enjoy heroics, will purposely estimate work on the low side as a challenge. It’s important that the project manager spots these and adjusts accordingly. One way to avoid this is to have two or more developers estimate task independently. It’s vital that this is carried out in isolation, particularly if one developer has better knowledge of the system than the other, because if this is done publicly there will be a tendency for the second estimate to be very close to the first. Using a tight schedule to increase efficiency in a team is just wrong and won’t work. If you need more efficiency from a team look at the other postings in this series, you should find help there.

Poor measurement of progress - it’s vital to collect information as development progresses - time taken and progress made. These are not the same. 5 hours spent on a task estimated to take 10 hours does not necessarily mean it’s half way through. I know this seems obvious, but it’s incredible how many plans are controlled in this manner. It’s also important to break tasks down as small as possible so that slippage is easily apparent. I prefer to have tasks lasting no more than two days and preferably one day as any slippage is known after two days, whereas if it was lumped with other tasks that combined took up two weeks, potentially slippage wouldn’t be discovered until late into the second week.

Fixed deadline, such as a sporting event, new tax year or introduction of regulations - if this sort of project is started late there is no simple solution. In order to deliver on time it may be necessary to increase resource, but only if it will make a difference (see Adding People to a Late Project), or reduce functionality.

The effect of office politics on any project is like the effect of kryptonite on Superman. Simply the more of it the slower the project goes until eventually the project grinds to a halt. If you saw the Sport Relief version of The Apprentice in March 2008 and witnessed Lembit Opik, Liberal Democrat MP for Montgomeryshire negotiate on behalf of his team, you’ll know exactly the point I’m trying to get across.

In my last post I talked about Geoff Polites and how he made space for people to work and protected them from things like politics that would impede their progress. This is an essential skill for successful project managers and IT directors and those that have it are rewarded with highly efficient teams. Keeping negative politics away from a development team isn’t easy, particularly in bigger organisations but failure to do so can lead to direction and time being lost in many ways.

As we’ve already discussed, having a project sponsor helps in many ways including cutting through politics, assuming the sponsor has more of a “get the job done” mindset and isn’t a political animal.

I’ve found an article called Politics-Oriented Software Development. Whilst slightly tongue in cheek, it does ring true in many places.

Before we dive into this one, lets just remind ourselves why software exists.

To solve a “problem” or gain an advantage for the person that uses the software

And that’s it. You could argue that low level software such as drivers fall outside of this, but indirectly they solve a problem, whether it’s a USB driver for a MP3 player (advantage: listen to music whilst walking) or a printer driver (advantage: print photographs at home).

You have to involve people who will use the software in as many stages of the development as possible. This can turn out to be painful and there is a danger of bending the software to meet one person’s opinion so needs to be approached with common sense and sufficient users to form a genuine consensus. Don’t forget, however, you won’t please all of the people all of the time, so beware spend large amounts of time adding a small amount of functionality that a few people will use infrequently (unless they are prepared to pay!).

This sort of activity at the requirements gathering stage is fine, the design is fluid and if you’ve produced a prototype or wire frame then the feedback will be based on what the users see rather than what they imagine. However the later in the project this feedback is gathered the more time and money it will take to change, but rather than dismiss it out of hand, look at whether there would be a suitable return on the investment.

A couple of pointers about gathering feedback. If you have a demonstrable application, consider using a video camera or for web based software a service such as CrazyEgg to record how the user interacts with the screen. In Predictably Irrational: The Hidden Forces That Shape Our Decisions, Dan Ariely carries out controlled tests that indicate how we make decisions based on certain environmental conditions. One test he has people select a free sample of beer from a list of four different beers, some with exciting descriptions, some with dull. When people publicly make their choice, they are more likely to choose a beer that hasn’t been selected by someone else at the table that choose the one that sounds the best or the same beer is chosen by each person at the table (this may be a cultural difference). However when the selection is made privately more people chose the better described beer.

I’m not suggesting that you ply users with free beer, but I am suggesting that you collect information privately possibly using a feedback form. The feedback shouldn’t be anonymous as you may need further clarification, but by removing any social convention by making comments publicly you will get better quality feedback.

To highlight the need for user input, a few years ago I wrote an application called Restoration Manager that helps locate parts from a vehicle that’s being restored over time. Having spent many hours searching for parts I had removed from a car weeks earlier, thinking I would be putting it back on in a couple of days, I felt I was in just the right position to design the software. A mate of mine who is an engineer and spends his days restoring cars and bikes and who doesn’t use a computer a great deal offered to test an early version, which was a good job as I was wide of the mark in some places. Interestingly I recruited 10 testers for a later release with the offer of free software and despite their initial interest only 3 actually downloaded the software and only 1 actually fed anything back, which was to do with the installation routine.

I’m unable to continue the classic mistakes series again today due to a problem one of my private clients is having with a software deployment. To help them hit their delivery date I’m rolling my sleeves up and attacking the problem, as Marku Alen, one of my rallying heroes from the 1980s’ would say “Maximum attack ! 110 percent!”

Typically some, if not all, of the problems could have been avoided.

• A merge from 2 code streams into another code stream had taken place just prior to the building of the software. There was no time for testing, not even the stored procedures have been compiled. The script to compile stored procedures into the database is failing in numerous places
• The deployment is split into two tasks, handled by two different departments. The department that handles the creation of the install kit using Installshield has one person that knows how to do this, and he’s on holiday
• The comprehensive operations manual hasn’t been updated for over 12 months, despite changes to the file server locations and an upgrade to Installshield 12
• An error produced in Installshield allows the MSI to be produced, but locks the files Installshield produces preventing another MSI to be generated in that directory. So every time the MSI is produced another directory has to be created and the necessary files copied over. This may have been overcome, but hasn’t been documented by the chap that normally handles it

How this could be avoided:

• Merging is a horrid task, very boring and easy to get wrong. In this case what should have happened is a merge from one code stream into another, fully test, or at least a smoke test, merge in the second code stream and fully retest. In adds 2 to 3 days to the merge, but it provides confidence in the application that currently isn’t there. Bearing in mind that this is almost a release candidate, releasing unstable code at this stage is foolhardy
• As a general rule never check code into shared source control system when that code doesn’t compile. There are downsides to this approach, for example amended code only existing on the developer’s computer for a period of time, but this risk can be minimised with a comprehensive backup plan. When you have more than one developer working on a code stream, I believe checking in only code that will compile is vital to prevent developers wasting their time getting someone else’s code to compile. The same applies to stored procedures
• Keep operations manuals up to date. The idea is that just about anyone in the development team could pick up the manual and produce, in this case, application binaries and the setup kit. Additional information, such as problems with Installshield can also be added to a departmental or company wiki