A dear friend of mine asked this question about his start-up idea. Having worked on a dozen ideas from prototyping to production, my take on it is a definite “NO”. Here is why:
The most important aspect of a prototype is its flexibility for validated learning. It is tempting to implement it like a mature production-ready project but this is also the number one reason for running out of resources.
Creating the product right instead of creating the right product is the number one resource waster.
Here is two reasons why you should not automate tests for your prototype:
1. Up front costs
Setting up automated testing has an upfront cost: you have to spend quite some time into finding the right tool and configuring it. Most of the benefits of automated testing comes to effect when you have a continuous integration (CI) server as well. And that usually follows with continuous delivery (CD). These are all unnecessary for a prototype when you’re not even sure if the requirements.
The problem is vaguely defined, therefore the solution architecture is not solid enough to justify the cost of tests.
Start-ups have limited resources that should be prioritized wisely. Quality is important but most of the basic tests will be done during product evaluation anyway (call it smoke testing if you will).
2. Slower validation
Apart from the time spent on learning the test framework, writing tests and keeping them up to date is a time-consuming process. This hinders the agility that is required for prototype validation. That being said, I'm an advocate of TDD and highly recommend unit tests before writing code that is going to live longer in the project. But TDD is not the ultimate development methodology either. The rule of thumb is: if the code is likely going to be changed when the project goes into production, don’t bother with automated testing.
When should I do automated testing?
If you have a team of developers who constantly implement new features, you might want to have good test coverage on the currently validated features and make sure that new development does’t break them.
This also has a side benefit: since updating tests has a price and will increase the scope of a feature’s implementation, developers may think twice before introducing new features that is going to deprecate older ones.
Update from test a specialist
I have the luxury of working with a veteran tester at my current company. After talking to him, I need to add 3 points to this post:
- If you happen to have testers during during prototyping, if there is anything that is decided, it is best to inform the them so they can prepare themselves in parallel.
- He said he is always involved in the project when there is a solid spec and requirement. i.e. after the prototyping phase is over.
- Writing tests first and then writing the code that pass those tests (test driven development or TDD) is a good methodology for developing products that are of coding nature e.g. when the product is an API or a library.