Thursday, May 24, 2012

Could manual testing work?

Let me tell you about my experience with software testing. In many projects (but not all) there has been a group of testers. Their job has been to test the system after the developers are done, file bug reports, wait for fixes, and then re-test. It all looks sound on the surface, right? However in practice this classical setup has a number of  problems :
  1. The tester isn't part of the team. He needs written documentation to know what to test. Somebody has to write this very detailed documentation.
  2. Testers I've worked with have had absolutely no clue on how to automate even the simplest test cases. Instead they have spent lots of time doing monkey work.
  3. The fact that there is a specific phase for testing means the developer, consciously or unconsciously, gets lazy with his own testing. That results in many bugs.
  4. If there are many bugs most of the tests will probably be impossible to run. The tester will have to wait for the developers to fix the bugs.
  5. Developers will quickly hack the bug fixes since we're already out of time.
  6. Nobody will know when or if it will be possible to deliver anything to the actual users.
 The  solutions  are pretty obvious. In the same order as above:
  1. Put the tester into the developer team. The tester will now have a chance to write tests even before code gets written.
  2. The tester is a programmer capable of automating tests.
  3. There is no testing phase. Stories are done only when they have been tested.
  4. No waiting for fixes will occur.
  5. No quick hacking will occur.
  6. Delivery is on demo day.
Problem solved. Relief.


Actually... Yes, I think so!

In my current project (which more or less follows Scrum) however we have yet to set up automated testing at a story level. The test cases so far have been completely manual. This fact gave rise to a discussion today among team members that I summarize like this:

"The thing I'm working on now affects more than just one story. I'll probably need to re-test quite a few stories and forms. What if the next story is similar? I'd have to re-re-test those same things again! That's going too take to much time. What we need to do is to have a code-freeze a number of days before sprint ending. We'll use those days to test the whole system and fix whatever bugs turn up."

Yes, that would do it.

 No! Panic Button! Incoming Godzilla! Iceberg on starboard! The drachma has arisen from the dead!

If we do this, we re-introduce a testing phase at the end of the sprint. Having a testing phase leads to sloppy developers, which leads to many bugs, which leads to end-of-sprint chaos!

The only really good solution is to automate the testing. But since we haven't done that we need a compromise solution. Suggestion for manual testing:
  • Multiple story implementations that affect the same existing functionality get tested together and just once. Before code-freeze. Fix for: Time
  • There is a code-freeze and the extra days will help us confirm the done-ness of stories. The testing will cover functionality we didn't even think about when doing story specific testing. Fix for: Regression testing
  • Only stories that are completed according to the DoD before the code-freeze occurs, may be included in the sprint delivery. No stories are allowed to enter into the state of Done after code-freeze. Fix for: Bad habits
That might work. Let's see if I get a chance to try it out.


Overath said...

Dont forget to convince the product owner of the importance of test automation;)

Orimark said...

I just reviewed your posting and found it interesting. Generally Manual Testing means the program is checked manually for the errors and then it is corrected. As the technology is developing the people prefer Software Testing rather than manual testing. I gained some idea by reading your writing. Hope to read more of your post in future…!