I tend not to like TDD for a different and serious reasons. TDD seems to encourage bad trial and error programming practices where developer blindly modifies the code until it passes the tests instead of reasoning about correctness based on algorithms and specifications. TDD does not consider that even a program that work correctly in the current environment for any possible set of input data is still incorrect if it violates contracts defined in APIs and protocols. Remember how the correct update to the memcpy implementation in glibc broke programs that incorrectly used it for overlapping memory regions.
My problem is more that I never know what to test. I mean, suppose I have some nontrivial code -- one of my latest random projects contains this:
def permute(tpl):
"""Gives the set of valid permutations for this tuple."""
if len(tpl) == 0:
yield ()
else:
for t in permute(tpl[1:]):
for n in range(0, len(t) + 1):
yield t[0:n] + (tpl[0],) + t[n:]
It's a recursive generator-driven permutation engine. Technically I suppose the order of the permutations doesn't matter, so long as they are all distinct and there are n! of them. Is that what I should be testing? That is, should my test code read:
permute_test = tuple(permute((1, 2, 3, 4)))
assert(len(permute_test) == 24)
for i in range(0, 24):
assert(len(permute_test[i]) == 4)
for i in range(0, 23):
for j in range(i + 1, 24):
assert(permute_test[i] != permute_test[j])
...? And if so, how does that help me write the original function? Or am I supposed to test a base case and one or two recursion steps, so that it helps me write the function, but "hard-wires" a particular order?
You're right to want to avoid duplicating your code inside your test. In this case, I'd work out some simple permutations and test against those values, sorting the results so order doesn't matter.
In the doctest, I include a few toy examples, stuff I can verify by hand, and which helps the user understand what I'm doing.
In the unit tests, I try to do things in the style of Haskell's Quickcheck (the greatest testing library out there). Generate some random values and test properties (e.g., len(permute(x)) == factorial(len(x))).
Not to sound snarky, but sounds like you're doing it wrong. You're right that the canonical example of TDD is to write a test that fails and write code until the test passes, but that, IMO, is an over-simplification that ignores the motivation behind TDD. You should write tests whether or not you do TDD, but TDD ensures proper test coverage. If you write tests first, you ensure every line of code is covered by a test.
Otherwise you have to do what I had to last month, which was go back—after the fact—and write unit tests for a large chunk of our code base because we didn't have an automated way of verifying that our logic worked with different (read: non-happy path) data sets.
