Archive for October, 2010


I got into an interesting – and as it turned out – quite refreshing conversation the other day about testing.

My view about tests is that, quite simply, they only make sense if you know what you are doing and have knowledge and insight. If you don’t, you’re in trouble: through no fault of your own, particularly.  Do you have the background and context to interpret a set of tests?  What if you don’t? It’s a question that many people find themselves facing on a regular basis, and isn’t a ’problem’ that can be systematically ‘solved’ as such. Nor is it some kind of obscure side-issue – it is at the heart of application development. Once we accept it, the oft-quoted Kent Beck panacea of “Writing the tests first” takes on a somewhat different meaning. What if you are not in a position to write the tests?  What happens then Mr Beck? He offers no explanation. Furthermore, getting to a point where you can write tests, might involve alot of time, effort and hard work. Or – to put it another way – analysis. However you dress it up, it is necessary to acquire knowledge. This doesn’t happen overnight. We are deluding ourselves if we think otherwise.

You don’t instantly gain the ability to write tests. So “starting with the tests” should be an aim. Not a policy. In fact, starting with the tests if you’re not ready to do so can be downright dangerous. Or at least pointless. I have seen many tests that don’t do anything useful at all. Others that have preconditions so obscure, Babbage, Turing and Lovelace themselves wouldn’t be able to decipher what is going on. What is the point of that? “Don’t code until you have tests” is reasonable enough, but anything more and you’re on shaky ground.

Many in the agile movement will counter what I have just said with things like “you can learn from the tests”, “we can ask the users”, “we already know”.  They always seem to make a massive assumption that there is an endless supply of knowledgeable people within and outside the team that are at our beck-and-call to answer everything.

This is rubbish.

There are obviously exceptions. Teams that have worked on something for long enough to know it inside out. In this situation, great. Good luck. My earlier post about whether you understand the tests in the first place, never arises. For most people though, this isn’t the case. We have to learn.

And what about those poor users? They might also need to learn, or be taught, about the system and it’s processes so they can do their jobs? They realise there is no point in contacting the IT team, since IT expect you to answer their questions. Not the other way around. It doesn’t surprise me that the relationship between IT and users is often fraught – many in the agile movement seem to view the users as Old Testament Prophets with all the answers. They don’t seem to realise that users have to go through a learning curve of their own and that IT should be supporting them in this.

Learning about an IT system is no different to learning about anything else. And you don’t learn by looking at tests.  If you are learning Geography, your starting point isn’t the examination papers.  It’s the text book.  Furthermore, If you are asked to write examination questions and you don’t know the subject, then the last thing to do is attempt to write the questions. You have to start by going away and learning and coming back when you are ready.  It’s no different in IT. You should write tests only when you are able to.  And to support this, you have to have a body of knowledge, artifacts and ‘truth’ to refer to.

So no Mr Beck.  You don’t start with the tests.  You start with knowing what you are doing.

Astonishingly there was agreement in the group…

I have recently had to take over some work from a colleague, and the acceptance criteria might as well say something like this. Preconditions that are virtually impossible for the uninitiated to pick up.  How will the ‘Given’ be achieved, pray tell?  Proof, if ever any were needed, that User Stories often don’t give you the whole story (to coin a phrase) and have to be accompanied by something else. It amazes me how many people still don’t get the message.