The Agile Alliance Functional Testing Tools Workshop (AAFTT), held the day before the Agile 2011 conference in Salt Lake City, was once again one of the highlights of the conference. Organised by Jennita Andrea and Elisabeth Hendrickson, it was as always a wide variety of participants with a passion for testing and testing tools. Here are my notes from the day held on August 7, 2011.
Energy Kickoff & Networking
The session was facilitated by Ainsley Nies, and all of the official session notes are stored on the AAFTT wiki: http://aaftt.agilealliance.org:8080/display/AAFTT/agile2011.
We started the day with some networking and sharing some areas of passion. Some of these included:
The theme of the AAFTT is: “Advancing the state of the art and the practice of Acceptance Test Driven Development”.
Ainsley started walking the circle to explain the day and how open space works, but frankly it make me feel a little dizzy! She went on to explain that
Harrison Owen invented the open space idea as he noticed the real content at conferences was the passionate conversations. The rules of open space are:
whoever shows up are the right people
do not hang on to pre-conceived ideas
it starts when it starts
discussion does not need to be over until it’s over
wherever it happens is the right place
The law of mobility and responsibility (also known as the law of two feet) is if you are not learning or contributing where you are, go some place where you will. Also, butterflies and bumblebees cross pollinate ideas.
Finally, we were warned to be prepared to be surprised.
Developers are testers and testers are developers – how do we dissolve and combine the roles
This was the first session that I attended.
there are two mindsets - offence and defence, testers are defence
job is not to find defects but to prevent defects – build quality in
define quality and what does it mean to us
startups don’t often have the problem – multiple skills required
what is the biggest impediment – are we missing the skill
there is no team of quality anymore – drive quality through the organization
functional testers tend to exploratory test and drive from the UI, technical analysts tend to multiple-skill
you need to have a team focus and a product focus
don’t start with practices but start with a common vision (eg. zero defects)
fear of losing identity if you dissolve roles
understanding the historical roles sometimes helps understands why things are the way they are
need time – Lisa Crispin mentioned that in her company they were going out of business because the system was not good quality, so management were smart to support the initiative
helps if everybody on the team has experienced the entire value chain and needs to understand the value of everybody’s piece of the chain – tendency to optimise the piece of the chain you understand
developers often underestimate the precision of data and scenarios and developers underestimate the difficulty of some requests
personality issues often get in the way
mostly about having the right people – need to let some people go
we assign labels to roles which create barriers – break down on teams but need to break down at the HR level
payroll is also an issue – need to compensate for people taking on more responsibility
need to put queue limits on the testing queue to drive behaviours
pairing with developer if they do not understand the scenarios
some people have the questioning mindset, some have the practical focus – need both to make sure you ship a quality product
mini waterfall problem – long tail feedback loop, change workflow that developer needs to work with tester, avoid lean batching problem
Jennitta Andrea led this session about the work so far in this space:
Last Mile Tools
Elisabeth Hendricksen led this session on tools that are attempting to solve the problem at the last mile.
NUnit – Liz Keogh – were using Fitnesse but added another level of complication, wrote a DSL that separates tests to make it easier read, WiPFlash is the automation tool, examples are on the website, can call the fixtures from another testing tool like Fitnesse, capture scenarios on a wiki first to get the best out of the automation tool
SpecFlow – Christian Hassa – similar to Cucumber, scenarios written as steps that are bound to execution, uses Gherkin parser (this is a plus as a number of tools use this)
SpecLog – maps of your product backlog, capture results of collaboration with the business ( Jeff Patton’s story maps), data stored in a single file, stories are initially mapped to a feature file but ultimately get linked to a feature tree
SpecRun is under development currently, not bound to SpecFlow or test runner/execution, currently Windows only
Limited Red – Joseph Wilk – uses the probability of failure to run those tests first in Cucumber, can then get failure statistics at a feature level, working on a refactoring tool at the moment
Relish – publish Cucumber features to a website
The Smallest Federated Wiki – Ward Cunningham – JSON for data scrubbing, thin columns to display well on mobile, refactoring is the number one edit so allow it to drag and drop refactor, fit for any analytic or outcome-oriented endeavor, sponsored by Nike, under very early development, meant to take spreadsheet data to the next level
Mary Gorman led this discussion.
business rules – conference website has rules, such as group pack for 5 registrations, what happens to the sixth person, what if someone pulls out
need to capture these to describe what our system does
business rules manifesto – Mary gives a copy to everyone she work with separation of concerns – a rule is separate from the action which makes the process more brittle and more difficult to test
rules are a form of requirements and live beyond the building
one process is to extract the rules of a legacy system and then the regression tests – code archaeology
the business does not always know the rules of the system or how they got there – rules get added to the system over time or evolve and documentation is unlikely to get updated
one insurance company had spent $100 million dollars to bring in a business rule engine, returned investment in two years due to being able to be able to look for conflicting rules
put analysis of rules in the hands of developers for way too long
simplest part of business rules is having a glossary
rules engine enables our rules in productions, and use examples to ensure the engine works correctly
testing could look like this – given this data when these rules are applied then I expect this output
you need both rules and examples to test them – you need enough examples for now, need to be different paths, decision points, infliction points rather than different values
examples are not as expressive as arithmetic, but they are not as understandable
lots of rules that we do not think of as business rules because they are baked into the process eg. security access, database schemas
“business logic is not” (Martin Fowler)
you can’t read English as if it were rules, so we need to use examples
the worst systems are the ones that do not have a manual override, humans are usually the best at determining this
lots of business rules change due to jurisdiction
something will always fall to the bottom – rules need to be valued on risk and value – where is the tipping rule
rules are the expression of intent
Mars issue – crashed, six week window too costly to fix
guts to keep it simple – reporting system (Ward Cunningham) – resisted urge to put in a formula system, wait for requests from users, got 6 requests, sold system based on simplicity of the system
As with any conference, there are always sessions you would have liked to have got along to.
Richard Lawrence led a discussion on Static Analysis for Gherkin which turned into a discussion on design patterns for Cucumber.
George Dinwiddie led a discussion about conversations between roles:
My mate Jason Montague led a session on Building Conditions Conducive for ATDD Adoption.
We shared some takeaways in the closing circle, he were some that stood out at me:
issues with dealing of people was a theme
what are good ways to express a large amount of test data
challenge to get corporations over the hump to release data, plus have good tests and examples around the rules
testing needs to be a nation, not just a community
it’s time we got more respect in our organisations, it’s time we show more respect to those we work with
teams need to dependent on the production of the build
federated wikis could help solve the test ownership problem
As for me, my comment was the day had renewed my energy again. ATDD is hard, and as a community we need to try harder.
Finally, I recorded a
short audio podcast for The Agile Revolution wrapping up AAFTT.