Estimating software is really hard

Are software estimates valuable? It's hard to say. We, as programmers, feel that estimates are very difficult to obtain, usually inaccurate and therefore something of a waste of time. Managers, on the other hand, have a certain budget available and often a certain time frame, too. They want to know if a project will fit into those constraints or, if they are a particularly talented manager, what subset of the blue-sky-dream requirements can be completed within that budget and time. Estimates are the way budgets turn into project plans.

The problem is that software estimates are always a tricky beast. Software isn't like constructing a building. It's more like designing a building for a swarm of robots to build in seconds for no cost. All the effort in software is design effort, not construction, and design is the hard part to pin down.

When you design a building, you have some solid requirements to work from. It will be this tall, with this many floors, we want this many units or offices inside, and this many car parking spaces. That, in turn, leads to some other calculated requirements - to serve that many floors, you need this many lifts. Fire stairwells are this size and regulations require two of them. Concrete can support this much weight per unit surface area. You know what you're getting into.

Software, so far, doesn't have any of that, and might never do. Also, by the time you're done finding out how large the project is, that's part of the design work. If, for instance, you're trying to measure business software by the number of database tables required, then you need to find out what tables are needed and count them. Finding out what tables are needed, though, means gathering all the requirements and designing tables to store the relevant data and oh, look, now you've done a database schema to estimate how long it will take to develop a database schema. It's almost as if the nature of software rebels against being quantified before it is completed. Unfortunately, "I'll know how much it will cost when I'm done spending the money" isn't a satisfying or wise answer to the question of budgets.

So what can we do? Maybe skilled managers have better intuition about how large a project will be, based on some imprecise metrics. If I know we're dealing with five kinds of business data here, then we'll probably have (for instance) 50 tables (10 per data type) and 10 major processes to write (interactions between the major data types), and from there we can guess how long that will take (assuming we know our team well).

The best way, probably, is to say "This is what we want to get done. We have up to this amount of money, or up to this time to do as much as we can. How close might we get to perfection?"

Mokalus of Borg

PS - I have to plead ignorance on how other industries estimate their work.
PPS - Except sometimes when I saw managers give a gut-feel guess and everyone called it good.