How many test harnesses are too many?
Before I start, let’s agree on what a test harness is, taken from the Wikipedia description of test harnesses: In software testing, a test harness or automated test framework is a collection of software and test data configured to test a program unit by running it under varying conditions and monitoring its behavior and outputs. It has two main parts: the Test execution engine and the Test script repository. While I know I have a tendency to disagree with people about definitions of things, I think this is a definition that we can, if not agree on, at least not violently disagree on. Based on this definition, we have four test harnesses in use here at work, three of which, curiously, were written by me. Those three, not coincidentally, were written to address serious limitations in Perl’s testing infrastructure. Christopher Humphries commented to me on Twitter that four harnesses seems like a red flag and he’s right! Unfortunately, this is where we’re at and the following explains why.
