This year, I again had the great opportunity to attend the Agile Alliance Functional Testing Tools Workshop (AAFTT), the day before the Agile 2010 conference in Orlando. Organised by Jennita Andrea, Elisabeth Hendrickson, Aimme Keener and Patrick Wilson-Welsh, it was once again a wide variety of participants with a passion for testing and testing tools.

It was also awesome to have Rachel Davies (co-author of Agile Coaching) facilitate the session.

First of all, the the agenda was set:

We then did a round circle stating our name, role, claim to fame and hopes from the workshop. Due to the high percentage of Canadians in the workshop, our locations quickly turned in Canada-location jokes (OK, you had to be there…)

Rachel then asked us to re-check the goal of the workshop. She suggested that we should check this with the group, even though it is usually set before the workshop. She asked the organisers to clarify the goals before re-checking what is in and out of scope for the workshop

The goal was: To reflect on the state of the practices and to identify ways to move forward.

In scope for the workshop was ATDD, marketing, pre-requisites, how we teach, effective ways we talk about terms and use of tools to support practice. Out of scope was TDD, terminology / ontology and test after.

Pleasure and Plains of ATDD

We then broke into a number of groups to draw pictures of our pleasures and pains in relation to ATDD.

This resulted in the following:

ATDD Retrospective

We then did an ATDD retrospective, which revealed a rich history!

Open Space

After lunch, we conducted 2 open space sessions.

Sales and Adoption Strategies

In the first session I attended was on Sales and Adoption Strategies which was hosted by Mark Levision:

• Elisabeth Hendrickson suggested that ATDD does not get traction because we need to bring everyone to the table, however it can give power to a struggling agile adoption, we need to understand the fears and concerns of people and we need to have skills to talk to all of the audiences
• Jim Cornelius said that managers think they want BDD for everything, yet we do not want to express everything in Given When Then format
• I told the story of using ATDD on non-software development teams, as well as some of our challenges in getting test team buy-in from our testing teams to tools such as Concordion
• Patrick Wilson-Welsh told a story about him introducing story testing to a team with no skills in testing, yet they were a rock star development team, they used JBehave and a thin TeamCity wrapper too convince them of the value
• it was suggested that continuous integration needs to be in place to give ATDD adoption teeth
• Andreas Ebbert-Karroum told a story of success when his team lost track of what they wanted to show at the end of the sprint, the team then wanted to do it, the barrier has been a steep technical learning curve, they are using Robot Framework but are always finding pieces missing (and wonder why in this day and age they seem to be the first person finding these issues)
• we then heard the example of simple DSL Lookup screen (the one where you type a phone number to see if you can get DSL coverage) at BellSouth which was an example of an extremely simple user interface and lots of business logic, they used Fitnesse, as they found it was hard to test manually
• Bob Galen suggested the need for effective grooming sessions to start collaboration, create a shared vision at the organisation level not just at the developer level, and the fact you need grass roots and top level support
• Mark  Levison told all of his executives to read Agile Testing and now he is getting buy in and getting questions about how it might work
• Jason Montague suggested his success has been through just in time communication, need to teams to get it working and sell the results for you

Pleasure and Pain with Tools

The second session I attended was on Pleasure and Pain with Tools hosted by Elisabeth Hendrickson:

• Express expectations in English and relate to underlying specifications
• unit testing successful because it is in my language, my idea, nobody else involved – all you need to do (sounds easy) is to change the way of thinking, and it works for both test first and after
• Robot Framework allows a test to be written in text, you can link from the wiki to the version control system and read directly from the trunk
• test from a photo of the whiteboard using ApprovalTests – put a whiteboard photo in the test directory, get the system under test to create a bitmap, first manually match the photo and once the customer is happy then get the system to diff automatically moving forward
• Sikuli uses images for tests – push a button that looks most like the bitmap in the test
• ATDD frameworks facilitate collaboration, like Cucumber, then you need a driver
• there was some discussion of Fitnesse versus Slim and the fact that the GPL2 code prevents adoption
• Jason Huggins suggested that Concordion / Robot Framework / Fitnesse are too much bondage, as a developer all I need is assert
• the question of testing Android was raised, with Jason Huggins suggesting Robotium
• White is a .Net test framework
• Given When Then loses detail if business people afe solely thinking that way
• it is awesome that most of the tools now support tables, given when then and other more recent test tool approaches
• too many tools are solely web focussed, Robot Framework is one exception although there are others
• wikis are not comfortable for non-technical users (Elisabeth Hendrickson called this the last mile problem), many tools do not have command completion or refactoring support and output from the tools is not always management friendly (who usually want to know number of tests passed / failed)
• to get ATDD tool buy in you need to get from scratch to test infected in 20 minutes
• need to discover you are done sooner than you expect
• a good analogy is why we have brakes on a car, not so you can stop but so you can go fast, the same stands for tests

Lightning Talks

We then kicked off a round of lightning talks:

Every Acceptance Test Should Look Like An iPhone Commercial (Jason Huggins)

I had seen this last year at AAFTT, but good to see Hugs was still passionate about it and has some updated examples this year

• Ruby on Rails worked because of the compelling video
• key technology is now YouTube
• the early iPhone commercials built excitement
• Google demo – a screenplay, sent to colleagues for acceptance, is more compelling than standard text

How Does ATDD Work with Kanban (Matt Philip)

• Elisabeth Hendrickson’s  diagram of  ATDD posted in war rooms
• a queue is envisioned as some persons work, at the end of the process, needs discipline and rigor

Principle of TD (Llewelyn Falco)

• need to do something and then verify it (do verify)
• benefits of a test – specifications, feedback, regression, granularity

Structure of Scalable & Maintainable Test Suite (With Robot Framework) (Andreas Ebbert-Karroum)

A copy of the model is available in the blog post: How to Structure a Scalable and Maintainable Acceptance Test Suite

• if I have stable platform and libraries, tests should be stable
• the resource layer is the layer that should be variable (in the middle)\
• Given When Then (GWT) tests grouped into themes, import file shield tests from business object changes

Testing Circle (Llewellyn Falco)

• discuss story on whiteboard
• then becomes a story in written firm
• then becomes code
• result then looks like whiteboard

WWII & ATDD (Brian Marick)

• places with good infrastructure had their infrastructure pounded to rubble in World War II but the USA did not, therefore we have no fast trains, no fast Internet, bridges that fall down, etc.,.
• all of XP is just what Ward Cunningham does naturally
• spend too much time figuring out how to survive with legacy code, we should go back to the small!

Open Space Summaries

• Why Have We Not Yet Discussed ATDD (J. B. Rainsberger) – came to conclusion that it is the same as BDD, How Test Driven Development Works (And More!)
• Adoption and Sales Strategies (Mark Levison) – small pilots under the covers, ping pong developer tester ATDD, automation directors and managers and developers have different perspectives, talk at an enterprise level, semantic patterns of needing to slow down versus switching everyone all at once, how do we maintain momentum, always feel we are the first person tacking the issue
• Facilitating Conversations (George Dinwiddie) – describing the problem rather than the solution, getting the language right, personas, myth of single product owner
• Coding Dojo (Llewellyn Falco) – wrote tests for Yahtzee
• Start a Business Example to Be Used for Many Tools (Mark Levison) – looked at example that Brian Marick wrote in Ruby, but still needs work
• Fixtureless Testing (Declan Whelan) – fixtures break cadence in flow, especially when a tester needs to talk to a developer, wiki storage format not optimal as not easy to refactor, should be stored as code so we can refactor and reuse, want language to be close between system under test and acceptance test (use the domain language)
• Pleasure and Pain with Tools (Elisabeth Hendrickson) – see above for notes
• Workflow – Activities and Deliverables – built up a workflow

Wrap Up

We then wrapped up with the future of AAFTT Program, in a discussion led by Jenitta Andrea:

All in all it was a great day. Lisa Crispin was taking some video so if the quality was good I will help her get it uploaded. I also have more pictures up on Flickr for those that are interested.