The Code Whisperer

There are books and books about planning projects, and I’m not going to attempt to replace them in one short posting!

I have a few observations over the years I would like to share.

Planning not taken seriously. For a plan to be of any use it has to be thought out, realistic, published and have tracking of time spent and progress of the task. In my opinion task need to be split down to a daily level for the plan to be worthwhile, otherwise it’s harder to know if the progress is behind the plan. If any of these don’t apply, and normally it’s most of them not just one, the planning process isn’t being taken seriously enough for it to be a worthwhile exercise.

Tasks missing. It’s easy to omit tasks from a plan and often they are common run of the mill activities that would take place as a matter of course. However omitting them from the plan has an effect - it’s time spent not allocated, which could cause the project to slip. I look at a plan as any other project document and have it peer reviewed and/or reviewed by the project team. If tasks are identified later in the project it’s crucial to add them in for three reasons. 1, you are more likely to remember them next time; 2, it allows you to adjust the plan and, if required, delivery dates; and 3, there’s an indication the task has been carried out.

Not revising estimates. Often estimates are carried out by the wrong people - sales staff, senior managers or customers. Estimates produced by developers sometimes are under the actual time required. We all know this, but revising estimates and the plan is very rare. Why?

Abandoning the plan. I’ve seen this time and time again. A project is late or chaotic, so any attempts to maintain the plan or any measured progress of the project are abandoned. The project then goes out of control, with any delivery dates a stab in the dark. It is possible to regain control of a project in this state, but take a special project manager. I have met a project manager that recovered what appeared to be a hopeless project. Possibly the most irritating chap I’ve met, but at the same time was interested in one thing - delivery of the project in the shortest possible time. He wasn’t afraid to tell the customer it would be a week late, and he delivered it when he said it would be delivered. Pure genius.

I’m sure there are more. If you have any observations on planning failure, I would like to hear them, so drop me a line at nigel(at)code-whisperer.com.

Outsourcing any part of a project to a third party is a risk, but one that can be minimised. By careful planning and management. for example having good quality technical design documents and continual communication with the third party, it’s possible to benefit from the use of a third party software, whether bespoke or readily available software.

Outsourcing allows projects to have more personnel working on it than the company has. Outsourcing is sometimes used as a way of reducing the cost of a project, although, for a variety of reasons some of which are outlined below, the lower cost of development staff doesn’t always result in a lower overall cost. Generally outsourcing is carried out away from your office, so you need to plan for more time spent communicating that you would if the development team were in your office, simply because you will probably have one point of contact and your request for information may require a return call.

Personally I think it’s vital that outsourcing isn’t used as a scapegoat for a high risk project. I have seen this happen or more than one occasion and whilst a large consultancy firm was brought in that could absorb any financial penalties, the consultancy’s staff were under immense stress to deliver the undeliverable.

Managing third party development isn’t an area I’ve had a great deal of experience of, so I’ve dug out a few tips from various books as well as my own.

• Top quality design documents are key. The documents need to be well written, detailed and correct. Peer review all documents sent out and make sure the outsourcing company understand the work required sufficiently to estimate and plan the work. One major point of failure is the outsourcing company not appreciating the complexity of the work required. This can be the fault of either party. This is of particular importance if the work is carried out away from your office as the ability to communicate face to face over small items is greatly reduced. If you are receiving the source code and planning to maintain it yourself then you must supply your coding standards and consider performing a code review on the delivered code.
• Constant communication. Make sure you are aware of real progress against your plan, not just the time taken, and have clarification that the stated progress is correct.
• Risk assessment. You will need to include the outsourcing in your risk assessment, and possibly have a backup plan
• Frequent code drops. By having points in the development where code can be delivered and tested, you will have a measurable sense of progress, or slippage. Sometimes the code drop is combined with payment to the third party an incentive for working code to be delivered on the agreed dates.

I don’t want to give the impression that outsourcing is a bad thing, it’s not, provided it’s approached sensibly and managed correctly. Having said that a number of high profile public projects, for example the NHS patient record system, have been delayed due to contractual issues with the third party developers. The same issues that cause delays and problems with an internal project seem to be magnified when a third party is involved.

Unachievable schedules have blighted the industry since the dawn of software development. Here’s some reasons why this happens:

• Schedule set before requirements agreed and/or understood
• Pressure from customers
• Inexperienced staff, in any combination of the industry, application or development tool
• Use of leading edge technology
• Poor estimation
• Estimate overridden by senior management
• Features added but the plan not amended
• Low estimates produced as a challenge or as an incentive to work harder
• Poor measurement of progress
• Fixed deadline, such as a sporting event, new tax year or introduction of regulations

Schedule set before requirements agreed and/or understood - This is particularly prevalent in software houses, where the primary goal is to get a customer to sign up for work. The software house doesn’t want to spend money before they get the sale, so only sufficient scoping to sell the software or enhancement is carried out. In an ideal world the software house would give a rough estimate, which would become more accurate as the analysis and technical design progressed. The problem is that customer want to know exactly when the software is going to be ready before they sign and the sales team can use the time taken to develop the software as a bargaining point, however, as we all know, this is a bad idea. So what do we do? Ideally invest time in having senior analysts and developers work out an estimate for the work, add some contingency and present that as a non-negotiable estimate that will be refined during the analysis and technical design phase. Make sure, however, that at the end of technical design the remainder of the schedule is as accurate as you can make it. Now this approach will lose sales, but for sales gained it is much more likely that the software will be delivered on time. Quite frankly a software house that has a reputation for delivering on time and in budget is going to attract the work most people want to do.

Pressure from customers - reducing the time leads to cut corners, software with bugs and demoralised development teams. The customer that suggests removing a month from a development schedule wouldn’t dream of suggesting to a surgeon that their knee operation must be an hour shorter, so why do the same with software development. My advice is it this occurs before the sale is made and you are confident with your rough estimate, walk away. Easy to say, harder to do, but once you’ve reduced your time estimate to the potential customer with out good reason, for example increasing development resource where it makes a difference or reducing functionality, your estimates lose credibility.

Inexperienced staff, in any combination of the industry, application or development tool - one of the biggest assets existing developers have is in the industry and applications your team produces. A high turnover of staff, or resourcing using a pool of developers means that a proportion of the developers may not be as productive to begin with as others. Obviously this depends on whether an existing application is being modified and how detailed the technical design is, but it can be an overlooked factor in scheduling.

Use of leading edge technology - using the latest and greatest is a developer’s dream, but can cause all sorts of delays due to bugs in the new technology or its implementation with the rest of your environment. For example I spent ages trying to get Visual Basic 5, RDO and Oracle stored procedures to work efficiently. It turned out due to an incompatibility between RDO and Oracle, with both companies blaming the other for the inefficiency. There is something to be said for using an established technology, mostly because most problems will have been discovered, overcome or not, but details available on the Internet.

Poor estimation - developers frequently underestimate the amount of time to do things and project managers must be aware of this and adjust accordingly. Overruns from one project need to be analysed and used when estimating the next. It’s important to learn from project plans, whether they are correct or not but in order to do this it’s vital that the time taken to complete tasks is accurately captured, without the developer feeling they have failed if the overrun is due to poor estimation.

Estimate overridden by senior management - senior management can reduce estimates for a variety of reasons and mostly it’s a bad thing. The only way to resolve this is the respect and relationship senior management has for the project manager producing the estimate.

Features added but the plan not amended - there seems to be a stigma about changing a plan once it’s underway. Some added features will have little or no impact on the schedule provided they are introduced at the right time, but most additional work will have some impact. Not to replan is madness.

Low estimates produced as a challenge or as an incentive to work harder - some developers, particularly those that enjoy heroics, will purposely estimate work on the low side as a challenge. It’s important that the project manager spots these and adjusts accordingly. One way to avoid this is to have two or more developers estimate task independently. It’s vital that this is carried out in isolation, particularly if one developer has better knowledge of the system than the other, because if this is done publicly there will be a tendency for the second estimate to be very close to the first. Using a tight schedule to increase efficiency in a team is just wrong and won’t work. If you need more efficiency from a team look at the other postings in this series, you should find help there.

Poor measurement of progress - it’s vital to collect information as development progresses - time taken and progress made. These are not the same. 5 hours spent on a task estimated to take 10 hours does not necessarily mean it’s half way through. I know this seems obvious, but it’s incredible how many plans are controlled in this manner. It’s also important to break tasks down as small as possible so that slippage is easily apparent. I prefer to have tasks lasting no more than two days and preferably one day as any slippage is known after two days, whereas if it was lumped with other tasks that combined took up two weeks, potentially slippage wouldn’t be discovered until late into the second week.

Fixed deadline, such as a sporting event, new tax year or introduction of regulations - if this sort of project is started late there is no simple solution. In order to deliver on time it may be necessary to increase resource, but only if it will make a difference (see Adding People to a Late Project), or reduce functionality.

The effect of office politics on any project is like the effect of kryptonite on Superman. Simply the more of it the slower the project goes until eventually the project grinds to a halt. If you saw the Sport Relief version of The Apprentice in March 2008 and witnessed Lembit Opik, Liberal Democrat MP for Montgomeryshire negotiate on behalf of his team, you’ll know exactly the point I’m trying to get across.

In my last post I talked about Geoff Polites and how he made space for people to work and protected them from things like politics that would impede their progress. This is an essential skill for successful project managers and IT directors and those that have it are rewarded with highly efficient teams. Keeping negative politics away from a development team isn’t easy, particularly in bigger organisations but failure to do so can lead to direction and time being lost in many ways.

As we’ve already discussed, having a project sponsor helps in many ways including cutting through politics, assuming the sponsor has more of a “get the job done” mindset and isn’t a political animal.

I’ve found an article called Politics-Oriented Software Development. Whilst slightly tongue in cheek, it does ring true in many places.

Before we dive into this one, lets just remind ourselves why software exists.

To solve a “problem” or gain an advantage for the person that uses the software

And that’s it. You could argue that low level software such as drivers fall outside of this, but indirectly they solve a problem, whether it’s a USB driver for a MP3 player (advantage: listen to music whilst walking) or a printer driver (advantage: print photographs at home).

You have to involve people who will use the software in as many stages of the development as possible. This can turn out to be painful and there is a danger of bending the software to meet one person’s opinion so needs to be approached with common sense and sufficient users to form a genuine consensus. Don’t forget, however, you won’t please all of the people all of the time, so beware spend large amounts of time adding a small amount of functionality that a few people will use infrequently (unless they are prepared to pay!).

This sort of activity at the requirements gathering stage is fine, the design is fluid and if you’ve produced a prototype or wire frame then the feedback will be based on what the users see rather than what they imagine. However the later in the project this feedback is gathered the more time and money it will take to change, but rather than dismiss it out of hand, look at whether there would be a suitable return on the investment.

A couple of pointers about gathering feedback. If you have a demonstrable application, consider using a video camera or for web based software a service such as CrazyEgg to record how the user interacts with the screen. In Predictably Irrational: The Hidden Forces That Shape Our Decisions, Dan Ariely carries out controlled tests that indicate how we make decisions based on certain environmental conditions. One test he has people select a free sample of beer from a list of four different beers, some with exciting descriptions, some with dull. When people publicly make their choice, they are more likely to choose a beer that hasn’t been selected by someone else at the table that choose the one that sounds the best or the same beer is chosen by each person at the table (this may be a cultural difference). However when the selection is made privately more people chose the better described beer.

I’m not suggesting that you ply users with free beer, but I am suggesting that you collect information privately possibly using a feedback form. The feedback shouldn’t be anonymous as you may need further clarification, but by removing any social convention by making comments publicly you will get better quality feedback.

To highlight the need for user input, a few years ago I wrote an application called Restoration Manager that helps locate parts from a vehicle that’s being restored over time. Having spent many hours searching for parts I had removed from a car weeks earlier, thinking I would be putting it back on in a couple of days, I felt I was in just the right position to design the software. A mate of mine who is an engineer and spends his days restoring cars and bikes and who doesn’t use a computer a great deal offered to test an early version, which was a good job as I was wide of the mark in some places. Interestingly I recruited 10 testers for a later release with the offer of free software and despite their initial interest only 3 actually downloaded the software and only 1 actually fed anything back, which was to do with the installation routine.