Yesterday, we had a great alt.net øresund meeting at BestBrains in København. Sune held a great presentation about TDD, its pros, and cons. Actually, despite being a TDD fan and user, Sune is very insightful and realizes that TDD isn’t a silver bullet. It could actually decrease your changeability if done wrong for example.
I’ve personally experienced some of the pains Sune described and that we discussed during the open space session that followed. Actually, previously the very same day Martin N Jensen and I tried to start attacking some of them by complementing our TDD efforts with a BDD way of defining higher level requirements in the form of stories and scenarios.
During the bicycle/train ride home, i thought some more about these issues. Only two days before I had to fix a lot of tests that utilized mocking/stubbing when an API changed that didn’t affect the system behavior in any way. That got me thinking – what the frak am I doing?!? No business value in that activity…
My not-IRL-tested idea goes something like this:
You have two sets of unit tests.
- Stable BDD tests that drives the SUT (System Under Test) by only looking at the highest level of state (i.e. the database), performs an action, and then verifies the resulting state/output on the highest possible level. These tests corresponds one-to-one with the requirements if using stories as requirements
- Instable “classic” TDD tests that drives the design of the inner workings (classes and their interactions, individual methods, etc.) of the SUT
My idea definitely requires UFD (UpFront Design) more according to the lean way of developing software as opposed to some agile start-immediately approaches.
If you start with a set of fairly stable stories, and implement them as BDD tests in one way or another (we’re using StoryQ at the moment), you consider these tests as stable meaning it should require some thinking before changing them.
The much more fragile TDD tests are considered something that could be thrown away (likely in parallel with writing new ones), or rewritten if still needed, when the inner workings are refactored (e.g. when replacing Castle Active Record with Frog.Net Sune, or in my case when removing specialized repositories in favor of one generic the other day).
I feel that if I have my stories implemented as executable acceptance tests, then I would really get
- true confidence when undertaking big refactorings
- tests that could actually be read and understood by peers, and possibly some other stakeholders
- regression test suite that verifies the SUT’s behavior, i.e. that the stakeholders get what’s agreed upon
As a potential benefit I also get extremely readable status reports of every story on e.g. the build server every time an ok commit is made. This is a huge benefit IMHO since it allows you to remove the possible need for a complex task tracking system but still gives your PM (and all others) complete real-time status, allowing you to work with a whiteboard and post-its as a kanban board.
