Ordering Some Estimates… Did You Want Testing With That?

I have been participating in many of the discussions around software estimates for the past year or so. The #NoEstimates discussion has been quite a debate, and to be honest I get a lot of the points being made by both sides:

“We need estimates to secure our business agreements”

  • I completely agree that in some industries and markets, not providing an estimate will simply prevent deals from being made (I worked with commercial airlines and military… trust me it IS a show stopper. Projects span years and incremental investment (which I feel is a great model) is not something that this market is ready for)
  • Often when trying to decide between opportunities, providing an estimate in combination with a business opportunity summary is how decisions are made

“You call them estimates, but you ink them in as deadlines”

  • I agree that the current use of estimates (based on my own observations) as a tool for sizing and planning projects truly sucks.
  • I have personally witnessed time and time again how estimates are converted into delivery plans, and inked into contracts.
  • I have seen how teams are demoralized when they are attacked for altering estimates when new information emerges. “How did you get this so wrong?”

 

I found myself asking this question the other day. If you truly believe that you can estimate the work required for an upcoming project, and that your estimate can stand the test of time (in terms of scope, effort and expected value to the purchaser/investor/user)… then why would you need any testers? Or any testing at all for that matter?

Image

 

Now, I am sure that we have all witnessed estimates that get calculated with some sense of science. Some even consider “How much time is needed for PROPER testing?” (the magical mathematics for this is typically quite beautiful and impressive. Much like unicorns) Lets pause however and consider why we are testing in the first place.

  • We are hoping to learn about the system under test.
  • We want to deliver additional information to the team about the behaviour of the system under test.
  • We consider it essential to better understand what it is we just built, and to explore the many ways that a user might interact with the system.
  • We are concerned that once we get something the way we want it, further development might destabilize it.
  • We are hitting a new market and are afraid about performance given new conditions and scale.
  • We are trying something new and honestly we are both excited and scared.
  • We just expect the unexpected.

Since I have already opened this can of worms, I feel that I might as well extend this concept to getting iterative feedback from users, demonstrations to investors/stakeholder and even code reviews. Like testing, these are things that we feel strongly about as “Really good things to do” when the desire exists within the team to create something great and something to be really proud of. They are all excellent sources of feedback and information to constantly revisit our existing plan and adapt as we go. Smart people can do amazing things if you keep providing them better information.

So, for those who agree that testing is intended to introduce and uncover new information that cannot honestly be precisely predicted (or therefore estimated) and that where this new information will most likely add to (or alter) the planned work, then what was that first estimate about? Why do you trust it? How did you estimate unknown discoveries at unknown junctures within the project? Why are you willing to sign a contract and commit (often with penalties) to it?

For me, I think the next time that I am in a discussion where estimates are being used to plan delivery dates, I might just ask that simple question: “So can I just confirm… you DON’T want any testing with that?”