Now that I have put
myself in the camp of those who agree that the number of test cases does not
matter as long as all the test conditions are covered efficiently and
effectively, it seems to be a good time to find out the technical background of
the claim.
What is being 'efficient & effective'?
As I mentioned in my
previous post also, you can ignore the number of test cases you write and
execute if you are efficient and effective at designing the test cases. So again, what is being efficient and what is being effective? As Peter Drucker puts it in simple terms, "Efficiency is doing things right; effectiveness is doing the right things". In our context of test case writing, to be efficient you have to write an adequate number of test cases, so that no extra time is spent on repetitive or
invalid tests. To be effective you have to cover the right set of test conditions.
So how hard can it
be? As it turns out, 'not much'! The best way is to utilize some of the many
test design techniques.
Test case design techniques
There are many
techniques in common practice when it comes to the matter of designing and
implementing an efficient and effective set of test cases.
Equivalence
partitioning and Boundary value analysis (EP & BVA) are a combination of
two most used black-box test designing techniques. As Kanif Fattepurkar correctly states here,
"This technique is used to reduce an
infinite number of test cases to a finite number, while ensuring that the
selected test cases are still effective test cases which will cover all
possible scenarios."
Decision tables (AKA
cause-effect tables) help to test combinations of inputs and other conditions
in addition. Given the number of combinations between all available test
conditions, this technique helps to systematically determine which of those
combinations to be tested.
Given the system
under test gives different outputs for the same inputs depending on changes
made by previous inputs, (which make it a 'finite
state system') a selection of these states and transitions can also be
considered for testing. Identifying critical paths among the possible set of
transitions helps to determine the level of testing that should be incorporated
The latter two
techniques also have the by-product of supporting both the development and test
teams to get a better understanding of the system
How many of those techniques are out there?
There are many other
techniques such as Use case based testing, Coverage based testing,
Error-guessing and Exploratory testing to name the most commonly used. Besides,
most of these are considered as 'black-box' techniques whereas many white-box
and other techniques are also available.
The most important
point would be that it is always up to you - the one who do the designing - to
select which techniques to use.
Assumptions, Assumptions everywhere!
The
most important point is, almost all these techniques seek refuge of assumptions in order to mark their
level of success. Following are a couple of those:
- In EP, it is assumed that all the 'members' of a given partition are processed the same way. Which in turn translate to the point that if one member passes the test so would the others - and vice-versa
- Also, both EP and BVA techniques assumes that a given failure is caused by a single defect, not a possible combination of multiple defects ('single fault assumption')
So, as you can see
applicability these assumptions to the test conditions at hand along with the
ability of test designers to correctly identify the required inputs to use the
techniques (such as identifying the boundaries and partitions correctly) has considerable impact on the end results.
'WIIFY'
What's in it for
you? Well, two major things:
One - the number of
test cases you have to write and execute become less. Yet, rest assured, you
have not missed anything important because:
Two - coverage
obtained over the identified test conditions will be at an above average level.
That in the sense, you have covered all important and 'could-go-wrong'
conditions.
So at the end of the
day, if you properly use the techniques (probably a combination of multiple
techniques, rather than just 'one-fits-all') you will get to write and execute
a lower number of test cases without missing (ideally) anything important.
