Everyone thinks they know how to test… Most people don’t. And most people think they know what automated testing is, it’s just that they never have time for it.
Into this gap steps an interesting white paper, The Automated Testing Handbook, from Software Test Professionals, an online SW test community. You have to register for a free community account to download it, but I did and it’s A great 100-page conceptual introduction to automated testing. We’re trying to figure out testing right now – it’s for our first SaaS product so that’s a challenge, and also because of the devops angle we’re trying to figure out “what does ‘unit test’ mean regarding an OS build or a Tomcat server before it gets apps deployed?'” So I was interested in reading a basic theory paper to give me some grounding; I like doing that when engaging in a new field rather than just hopping from specific to specific.
Some takeaways from the paper:
- Automated testing isn’t just capture/replay, that’s unmaintainable.
- Automated testing isn’t writing a program to test a program, that’s unscalable.
- What you want is a framework to automate the process around the testing. Some specific tests can/should be automated and others can’t/shouldn’t. But a framework helps you with both, and lets you make more reusable tests.
Your tests should be version controlled, change controlled, etc. just like software (testware?).
Great quote from p.47:
A thorough test exercises more than just the application itself: it ultimately tests the entire environment, including all of the supporting hardware and surrounding software.
We are having issues with that right now and our larval SaaS implementation – our traditional software R&D folks historically can just do their own narrow scope tests, and say “we’re done.” Now that we’re running this as a big system in the cloud, we need to run integration tests including the real target environment, and we’re trying to convince them that’s something they need to spend effort in.
They go on to mention major test automation approaches, including capture/replay, data-driven, and table-driven – their pros and cons and even implementation tips.
- Capture/replay is basically recording your manual test so you can do it again a lot. Which makes it helpful for mature and stable apps and also for load testing.
- Data driven just means capture/replay with varied inputs. It’s a little more complicated but much better than capture/replay, where you either don’t adequately test various input scenarios or you have a billion recorded scripts.
- Table driven is where the actions themselves are defined as data as well – one step short of hardcoding a test app. But (assuming you’re using a tool that can work on multiple apps) portable across apps…
I guess writing pure code to test is an unspoken #4, but unspoken because she (the author) doesn’t think that’s a real good option.
There’s a bunch of other stuff in there too, it’s a great introduction to the hows and whys and gotchas of test automation. Give it a read!
the agile admin 