Announcing the TDD TestHelpers opensource project
Whenever I start working on a project; I invariably find myself writing a collection of TDD test helper methods. I quick survey of other TDDers reveals the same; and thus the birth of my latest opensource project, TestHelpers (http://code.google.com/p/testhelpers/).
The aim of the project is to centralise all those little test helper methods you end up creating into a useful assembly you can use to jumpstart your next project. Things like:
- Comparers
- Generic object comparers
- DataSet comparers
- Test Data generators
- Builder pattern
- Automocking containers
For example, I’ve just added an “AssertValues” functor; which helps you check whether the values of who object instances are the same.
One area I keep using asserts like this is in integration tests; where I want to check that the objects I’m persisting to the database via my ORM actually end up in the database in a non-mangled form. In this case, I new up entityA, persist it, reload it into entityB and then need to check that all the values in entityB are the same as those in entityA.
A standard Assert.AreEqual will fail, because entityA and entityB are different instances. But, my helper method AssertValues.AreEqual will pass, because it checks the (serialized) string values of entityA and entityB.
Here is another, simpler example to illustrate the concept.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 | [TestFixture] public class StandardObjectsTests { public class StringContainer { public string String1 { get; set; } public string String2 { get; set; } } [Test] public void ObjectsWithSameValue_ShouldBeEqual() { var stringContainer1 = new StringContainer {String1 = "Test String1", String2 = "Test String 2"}; var stringContainer2 = new StringContainer {String1 = "Test String1", String2 = "Test String 2"}; Assert.AreNotEqual(stringContainer1, stringContainer2); AssertValues.AreEqual(stringContainer1, stringContainer2); } } |
I’m sure you have a bunch of similar helper methods lying about your projects.
How about contributing them to the TestHelper project?
ALT.NET; London; 13 Sept 2008
Intro
Debate over what ALT.NET is; should it have a set of guiding principles like the Agile manifesto?
Continuous integration & deployment
There seemed to be 3 major areas where people encountered difficulties doing continuous integration & deployment.
- Configuration files
- DB schema migrations
- Data migrations.
- Make sure that your config files are small. and contain only that config data that changes often (DB connection strings, file paths etc). Put all your “static” config data into separate files (DI injection config etc).
- Consider templated config files; where specific values are injected during deploy process.
- Keep all config in simple text files in source control.
- Migration techniques borrowed from Ruby on Rails - generate change scripts by hand or using tools like SQL Compare; and then apply them using a versioning tool like dbdeploy.
- Take backup before data migration.
- Ensure app fails quickly if is a problem; cause if data has changed since deployment then cannot rollback.
- Consider apps upgrading themselves and running smoke tests upon startup - and refusing to run if there is a problem - this technique is used by established opensource projects - Wordpress, Drupal, Joomla.
- Build a DSL incrementally during short iterations. Gives you opportunity to refine, fill in gaps, and train whole team to use same language.
- Without a DSL, acceptance testing via the UI testing becomes brittle, as you end up specifying your tests at too low a level, (click button A, then check for result in cell B); rather than having a translation from acceptance tests in a higher DSL language to specific UI components.
- Consider prioritised tests - have a set of facesaving tests / smoke tests that always work, and ensure major things are still working (company phone number correct? Submit order button still work?). Acceptance tests can be thrown away if they have served their function of evolving the design / team understanding.
- The acceptance testing trio - Developers test for success - thus automated testing only tests happy flow - still need exploritory testing by someone with testing mindset; what happens if you do weird stuff? Tester must have domain knowledge. Business - what are should happen? Don’t let developers be forced to make up business rules?
- Ensure all layers of stack (tests, manuals, code, unit tests) use the same DSL language.
- How do you get workable acceptance tests - see Requirements Workshops book
- Short iterations - more focus, incremental specs, opportunity to discuss missing test examples.
- Key is having a ubiquitous language encoded as a DSL (domain specific language) - develops over time, enables automated accpetance tests,
- Sign off against acceptance tests (Green Pepper tool - capture & approve acceptance tests)
- Talk: The Yawning Gap of ?? doom - infoQ, Martin Fowler
- Avoid describing these activities as “testing” - people avoid because testing has low social status.
- Discussion around the difference between DDD; where we treat the concepts & actions as central; vs DB centrered design, where we’re thinking about the data as central, and UI centered design, where the screens are considered central.
- Concensus was that domain shouldn’t be tightly bound to the DB, or the UI.
- Ideas around passing DTO objects up to view (UI, webservices etc), andchange messages bad from view, indicating how the domain should be changed (rather than passing the whole DTO, where you don’t know what has changed).
- Defined as Dan North’s Given, When, Then
- Is it any difference from Acceptance testing? Only that it is better branding, because BDD doesn’t have the word “testing” in it; which prevents people being switched off hearing the word test when discussing specifications.
- BDD is writing failing acceptance testing first; before writing code.
- Unit testing is ensuring that the code is built right, but acceptance testing / BDD ensures that the right code is built.
- Toolset is still immature. Fitnesse .NET & Java tooling is most mature toolset. Many BDD tools (other than Ruby’s rSpec) have been started and abandoned (nBehave, nSpec etc)
- BDD is not about testing, its about communicating and automating the DSL. Be wary of implementing BDD in developer tools (e.g, nunit), which prevent other team members (business, customer, testers) from accessing them.
- Refactoring can break fitnesse tests, because it isn’t part of the code base.
- Executable specs (via acceptance tests) are the only way to ensure documentation / test suites are up to date & trustable
- Agile is about surfacing problems early (rather than hiding them until its too late to address them). So when writing acceptance tests up front is difficult; this is good, because you are raising the communication problems early.
- The real value is in building a shared understanding via acceptance criteria; rather than building automated regression test suite.
- Requirements workshops can degenerate into long boring meetings. To mitigate this problem
-
- Vitally important to have a Moderator - whoe’s role is to keeps it focused & moving along. Injects energy into the room. (BA’s often play this role soon)
- Stare test - disinterested; you’re talking to the wrong person.
- Requirements Workshops - idea’s for making the specification meetings less painful.
- Run requirements gathering as open space meetings - whoever comes is who should be making the decisions
Using Acceptance Criteria & Text based stories to build a common domain language between business and developer
Image via Wikipedia Besides precisely pinning functionality, writing text based stories has another - and some would argue more important - benefit, developing a shared domain language between the business & developers.
A large part of developing a new software application is defining and codifying a business solution. To do this, both sides of the equation must be molded to fit the constraints of the other - the business process needs to be expressed in a precise manner that can be automated in software, and software must be molded to fit the use cases of its users.
The mismatch between the way the business sees the solution, and the way the developers view the solution becomes painfully obvious about half way into the project, when you start to try to match what data fields are labeled on the UI, and what they are called in the database / object model.
I’ve worked on what should have been simple projects; where maintenance is a exercise in hair pulling as you try to figure out what data input results in the weird output in a report.
The root problem is a lack of a shared domain language. Projects naturally evolve domain languages; and unless guided, you can guarantee that the language in the customers requirements spec and that in the code base will diverge rapidly.
Sitting developers, testers and the customer together to produce written text user stories following Dan North’s classic BDD story structure goes a long way towards resolving this issue.
Talking through how functionality will work, and being forced to formalize it by writing things down helps the domain understanding and language to evolve naturally, influenced equally by the customers domain understanding, and the constraints the developer must work within.
Its vital that this is done before coding begins for the following reasons:
- All stakeholders have been indoctrinated in the same domain language
- Names for domain concepts are at hand when the developer needs them, resulting in better named domain objects.
- Both the developer and customer know exactly what functionality is expected; helping to keep both focused on solving the right problems.
- Facilitate ongoing conversations as the solution evolves. Evolving a shared language is difficult, and better done at the beginning of the project whilst everyone’s enthusiasm is high. With that hurdle out the way, ongoing conversations are easier, and the temptation just to guess, or devolve into an us vs them mentality is greatly reduced.
During release planning, the high level “As a [x] I want [Y] so that [Z]” is probably sufficient, with the “Given, When, Then” acceptance scenario’s being fleshed out at the beginning of each sprint.
Specifying your functional requirements as text stories leads to some exciting opportunities:
- Your “unit” of work is immediately available, and understood by all. This makes prioritizing which stories to include, work on next, or drop much easier
- Its possible to turn the stories into executable specifications.
The Ruby community has made the most progress in the latter opportunity, with their rSpec story runner.
Consider the possibilities of the following development practice:
- The team begin by specifying text stories & acceptance criteria.
- The testers turn this into an executable spec, with each step “pending”
- The developers then work from the executable spec, coding the functionality to make each step pass one by one
- When the customers receive a new version, the first thing they do is execute the stories, to see exactly which functionality has been implemented, and prove that all is still working as expected.
At any stage its possible to see how far the team is (how many steps pass?), speculative complexity is reduced because the developers are focused on developing only that which the test requires, and all the while a suite of regression tests are being build up!
![Reblog this post [with Zemanta]](http://img.zemanta.com/reblog_e.png?x-id=505a2f3c-65c4-460b-a75f-24129ace91c2)
100% code coverage is only part of the QA story
Luke Francl of Tumblon has a nice summary with backing research showing how unit testing is only part of the QA story.
Its important to realise that developers by nature will only test the happy path; hence its likely that those diabolical edge cases will remain untested by a developer.
Another important factor is that most bugs come from missing features or requirements; and its impossible to unit test what isn’t there.
See more at http://railspikes.com/2008/7/11/testing-is-overrated; specifically Luke’s create Venn diagram showing how different types of testing (unit, user, code review etc) uncover different types of defects.
This isn’t to say that unit testing isn’t worthwhile; it just frees your testers up to concentrate on the unexpected, non-logical things that users are sure to try use your software for 
Recommended reads
