This is the first of three blog posts about what I've learned running my little machine learning shop inside a 10K employee SaaS giant and through the previous 18 years of my software career.
In my long and storied experience one of the main ways things go wrong when delivering software is when the activity you have planned does not match the work you have to do.
Sizing software building tasks is hard. The work in building software varies from the mundane and simple to the arcane, unknown and complex - and it is entirely possible for size estimation to be as costly as the work itself.
It is easy to end up with the wrong plan for the work - and that's when you run into heaps of problems.
If you don't get it right, you're either under estimating or over estimating. We hear more about the under estimated activities - the deathmarch projects, and the cost overruns - but the converse problem is as common place and equally bad - it's the type of mistake organisations make, the origin of $10K toilet seats.
None of this is rocket science - but I thought I'd write it down since it explains so much of What Went Wrong when I look back at projects I've been involved with that have gone wrong.
What you did (Actual) vs What you should have done (Optimal)
|Actual ↓ / Optimal →||Project||Process||Task|
I decided to map it out in the table above. I've defined three levels of complexity of the activity you're undertaking
For the big problems you build a project - a full setup with communication, management, feedback, tracking, stakeholders, plans, dashboards. Projects are designed for the ad-hoc and at scale.
For a smaller problem, perhaps you do enough of them that you have a repeatable process you can rely on. There are well known steps, and well known concepts of handover from one step to the other. There might even be convenient tooling for you. If you do a lot of projects maybe you're really running a process instead when you do them
For the really simple we'll simply call it a task. It's a small thing that one or perhaps two people can finish in a short amount of time without the effort of coordinating or communicating too much about it.
When you went for a project, but should have used a fixed process everything's a question again. The repeatability of the process is gone - and of course you know have to deal with the cost of running the project.
When you rigorously pull even simple things through a tedious process, you spend a lot of time doing process theater - satisfying the process, not the task.
A classic is involving too many people in solving simple problems - invariably everyone will have an opinion - and invariably, the color of the bikeshed really isn't that big a deal.
The easiest way to undershoot is forgetting to plan altogether - this is the road to death march projects. But there are subtler ways too - like trying to steer a complex project, that requires a ton of signoffs and handoffs to succeed through a simple process. You end up blocked - either during the work, because you're lacking elements of the plan or - even worse - at the exit when there's just no fit between what you did and what you should have done.
There's also the more commonplace of not just checking the boxes - so you end up with a half baked solution instead of the more considered solution delivered by a well laid out process.