Sometimes you want to write a test for a business rule that's based on multiple variables. In your goal to cover the rule thoroughly, you start writing tests for each permutation of all variables. Quickly it blows up into something unsustainable. With n variables for the business rule, you get 2n permutations/test cases. This is manageable with 2 variables (4 test cases), but at 3 variables (8 test cases) it becomes ridiculous, and anything beyond that feels immediately uncomfortable.
I've noticed myself using an alternate pattern for this, which results in n+1 test cases instead of 2n. It gives good coverage despite not testing all permutations, and I actually think the test cases themselves become easier to follow. The approach looks like this:
- Write one test that covers the positive case, where all the variables are in the correct state to make the thing happen.
- Write one test per variable, to cover the negative case where the variable's value would cause the thing to not happen.
An example: imagine we have a message that we only want to show when (a) the user is logged in, (b) they have green hair, and (c) they like bananas.
The "all permutations" approach would result in 8 tests:
when the user is logged in
when the user has green hair
when the user likes bananas
the message appears
when the user does not like bananas
the message does not appear
when the user has brown hair
when the user likes bananas
the message does not appear
when the user does not like bananas
the message does not appear
when the user is not logged in
when the user has green hair
when the user likes bananas
the message does not appear
when the user does not like bananas
the message does not appear
when the user has brown hair
when the user likes bananas
the message does not appear
when the user does not like bananas
the message does not appear
You can see in this list of test cases the inspiration for the alternate approach — there is only 1 of 8 tests that test for the positive case. The rest are all combinations of values that turn the message off.
The alternate "turn it off one variable at a time" approach results in 4 test cases:
when the user is logged in, has green hair, and likes bananas
the message appears
when the user is not logged in
the message does not appear
when the user has brown hair
the message does not appear
when the user does not like bananas because they are some kind of monster
the message does not appear
We have zero test cases for multiple fields being turned off, but that might be a good thing — if multiple variables are turning the message off, you can't really say with 100% confidence that the test is passing because both fields are being checked.
I think I tend to evolve toward this latter approach even without thinking about it, but now that I've become aware of this tendency, I've been using this as a starting point instead of the "eventual evolution."