Sanjiv Augustine on Scaling Agile, No-Management and Agile 2015 Executive Forum

InfoQSanjiv Augustine talks about his new book “Scaling Agile: A Lean JumpStart”, reinventing organizations and the implementation of no-management at LitheSpeed and the Agile 2015 Executive Forum.

SanjivSource: Sanjiv Augustine on Scaling Agile, No-Management and Agile 2015 Executive Forum

AAFTT Workshop 2012 (Dallas)

Agile AllianceThe Agile Alliance Functional Testing Tools Workshop (AAFTT), was one again held this year the day before the Agile 2012 conference in Dallas. Despite there being only a small group there this year, the discussion was still open and free flowing under the facilitation of Matt Barcomb and the organisation of Joseph Wilk and Elisabeth Hendrickson.

From Agile 2012

We created an agenda for the day:

From Agile 2012
From Agile 2012

Here are my notes from the day:

Enabling Non-Programmers

George Dinwiddie led this session which turned into a lively discussion! I had proposed what I thought was a related session on Specification By Example and had combined them, but the conversation never really had a chance of getting onto that topic!

From Agile 2012
  • George expects the business people to be able to read and understand the tests
  • non-programmers should not be writing automation, it is the programmers responsibility
  • wants to be able to extract working tests into a step definition rather than needing to rewrite in Ruby (George Dinwiddie)
  • there is a difference between a specification and testing (Christian Hassa), this is a fundamental shift
  • building a DSL – talk about terminology and how we explore our domain – essential step
  • you don’t create a DSL, you build it
  • not a problem with the toolset but our training in thinking in a procedural way rather than an example way of thinking (Corey Haines
  • testers new to automation create large scripts because it’s their only hope in creating some sort of repetition (@chzy), it does not take a lot of effort and most business people are open to working this way
  • enable non-programmers by getting them to come work with us every day (Woody Zuill)
  • George is helping people make a transition, don’t want people to throw away what they have,
  • ideal is not to have step definitions call step definitions, Cucumber community is becoming a community of programmers and are moving away from this
  • Robot Framework is more keyword driven, more aligned to non-programmers, you can also make a mess, “it is a double edged sword” (Elisabeth Hendrickson)
  • testers like to test the negative cases, should they be expressed at a high level or expressed as a unit test by pairing developers and testers
  • if you are testers and you cannot write simple Ruby scripts, then you have no place on my team (Corey Haines), this opinion is probably shared by the Cucumber community (George disagreed…)
  • need to use the same design patterns in both Robot and Cucumber (@chzy)
  • in an environment that is test centric and BDD, Cucumber is the tool (usually environments with little to no QA),  in a business centric environment where you an get the business involved Robot Framework is your tool
  • Corey works in environments where there is very few Cucumber specifications per scenario, backed by lots of unit tests
  • Cucumber came out of environments where the team is predominantly developers, hence the desire to drill down to Ruby code sooner
  • at a large household name company – theyexpect testers to be more technical, happening more in the industry, eliminated the role of tester due to different pay grades (@chzy)
  • moving traditional organizations to a collaborative way of working is hard (@chzy)
  • wants simple refactorings that are are a bridge from one place to another (George Dinwiddie)
From Agile 2012
From Agile 2012

Not Testing

Joseph Wilk led this discussion on thoughts that are coming from the Lean Startup movement.

From Agile 2012
  • at a startup Joseph was at, tests were taking up to 8 hours to run and costs for distributed architecture was high
  • Forward Internet (London) – let developers do what they want – by not testing they could be faster and more interactive than their competitors – did testing in Production, a risk that sometimes things could fail – testing should not block deployment
  • in some situations it is just worth hacking it out, particularly in a lean startup
  • if it is faster to rewrite rather than maintain it, then don’t write tests (Fred George via Corey Haines)
  • a big question of this is the skill level of your developers – do you have the skill level to make the choice to not do it (Corey Haines), primary impact of success is the skill level of your developers
  • cost of failure?
  • complexity is in the eye of the beholder
  • Etsycheck error rates in Production (and decide whether to roll back or not)
  • Scribd – were having trouble with test speed and found out the developers were scared of breaking the PDF (which is the heart of the business) – they separated the PDF out to speed up development (so developers weren’t worried about breaking it)
  • quick delivery – need the quick feedback cycle to make this work, simulate production
  • need effective tests – small suite of tests that are 5-10 minutes long
  • test what you are most scared of
  • Silicon Valley’s issue is hiring – Facebook is stealing developers from Google because they hire good people and enable them to just hack it out
  • 2 software industries – small companies and large corporations, very different worlds
  • question everything – can only do this if you have experienced it before and understand it
  • need a model to help others adopt this
From Agile 2012

What Are The Better Ways To Specify Tests With Large Test Data

I unfortunately did not get to this session as it was running at the same time as the No Testing session, but here is the output from that session.

From Agile 2012
From Agile 2012
From Agile 2012

Deliberate Test Practice

Brandon Leiran led this session, trying to see if there was a testing equivalent of coding katas.

From Agile 2012
  • weekend testing group – choose a target, collaborate on Skype on their findings
  • Wikimedia Foundation – looking at ways crowd source testing to test infrastructure (rather than content) – more on this initiative to be announced in the near future
  • why is it any different to coding katas? Safer and smaller so you get more practice, practice collaboration too
  • organise a community like a book club
  • code roast – put the code up and everybody critiques it, be careful not to attach to a person!
  • get practice at driving different interfaces – Triangle Tester exercise, parking lot calculator
  • hard to practice test automation as it takes a lot of time upfron
  • take time to do charter writing sessions or test different items like cheap toys (how would you test this toy?)
  • demonstrate value of quality using simulations eg. origami games
  • add tests to open source – many of the existing tests are average
From Agile 2012
From Agile 2012

Holes / Editors

Chzy led this discussion to discuss holes in the existing frameworks.

From Agile 2012
  • the HTML report from Cucumber is very average – chzy is releasing a new gem based on discussion from a recent testing conference in Austin
  • editors – people now bundling these in TextMate, Eclipse and Visual Studio
  • JetBrainsRubyMine has gherkin support and refactoring support for Ruby, plus a lot of support for steps in Cucumber
  • big picture view of feature coverage – would be cool to map this to Sonar, suites represent functional areas, tags to represent cross-cutting concerns
  • SpecFlow is trying to map to story maps using SpecLog
  • Relish allows you to create higher level specification of your scenarios
  • there is a plugin for Cucumber that allows github integration
  • Thucydides has a built in feature coverage report
  • Twist has Cucumber support
  • test data management – FactoryGirl gem – build up snapshots but want to be able manipulate values down the stack, Faker, ActiveRecord
From Agile 2012
From Agile 2012

AA-FTT – The Future

Elisabeth Hendrickson led this session as part of her handing the leadership over to Joseph Wilk.

From Agile 2012
  • mission is to advance the state of the art of functional testing tools
  • community building is the best way to spend the money, tool builders and tool users
  • Yahoo group is main repository of knowledge, current wiki probably needs to be moved
  • need people who have time and energy and interest to take this forward
  • biggest issues with wikis is managing all the wiki spam
  • have a leadership issue to curate the content and grow the community
  • the other options are to create static content, like business analysts and leadership
  • important to have a knowledge repository that at least captures outcomes
  • would like have more organised meetings worldwide
  • is our mandate just functional testing? It has really been just about “agile testing”
  • probably need to rewrite the charter
From Agile 2012

Wrap Up

We finished up the open space by writing what action we were taking from the day and giving them to another participant to keep us honest (mine was to write this post!)

From Agile 2012

Another good open space, and good to catch up with many of the leaders in the testing community once again.

Podcast

I recorded a short audio podcast for The Agile Revolution wrapping up AAFTT.

AAFTT Workshop 2011 (Salt Lake City)

Agile AllianceThe Agile Alliance Functional Testing Tools Workshop (AAFTT), held the day before the Agile 2011 conference in Salt Lake City, was once again one of the highlights of the conference. Organised by Jennita Andrea and Elisabeth Hendrickson, it was as always a wide variety of participants with a passion for testing and testing tools. Here are my notes from the day held on August 7, 2011.

From AAFTT 2011
From AAFTT 2011
From AAFTT 2011


Energy Kickoff & Networking

The session was facilitated by Ainsley Nies, and all of the official session notes are stored on the AAFTT wiki: http://aaftt.agilealliance.org:8080/display/AAFTT/agile2011.

From AAFTT 2011
From AAFTT 2011

We started the day with some networking and sharing some areas of passion. Some of these included:

The theme of the AAFTT is: “Advancing the state of the art and the practice of Acceptance Test Driven Development”.

From AAFTT 2011

Ainsley started walking the circle to explain the day and how open space works, but frankly it make me feel a little dizzy! She went on to explain that Harrison Owen invented the open space idea as he noticed the real content at conferences was the passionate conversations. The rules of open space are:

  • whoever shows up are the right people
  • do not hang on to pre-conceived ideas
  • it starts when it starts
  • discussion does not need to be over until it’s over
  • wherever it happens is the right place
From AAFTT 2011

The law of mobility and responsibility (also known as the law of two feet) is if you are not learning or contributing where you are, go some place where you will. Also, butterflies and bumblebees cross pollinate ideas.

From AAFTT 2011
From AAFTT 2011

Finally, we were warned to be prepared to be surprised.

From AAFTT 2011
From AAFTT 2011

Developers are testers and testers are developers – how do we dissolve and combine the roles 

This was the first session that I attended.

From AAFTT 2011
  • there are two mindsets – offence and defence, testers are defence
  • job is not to find defects but to prevent defects – build quality in
  • define quality and what does it mean to us
  • startups don’t often have the problem – multiple skills required
  • what is the biggest impediment – are we missing the skill
  • there is no team of quality anymore – drive quality through the organization
  • functional testers tend to exploratory test and drive from the UI, technical analysts tend to multiple-skill
  • you need to have a team focus and a product focus
  • don’t start with practices but start with a common vision (eg. zero defects)
  • fear of losing identity if you dissolve roles
  • understanding the historical roles sometimes helps understands why things are the way they are
  • need time – Lisa Crispin mentioned that in her company they were going out of business because the system was not good quality, so management were smart to support the initiative
  • helps if everybody on the team has experienced the entire value chain and needs to understand the value of everybody’s piece of the chain – tendency to optimise the piece of the chain you understand
  • developers often underestimate the precision of data and scenarios and developers underestimate the difficulty of some requests
  • personality issues often get in the way
  • mostly about having the right people – need to let some people go
  • we assign labels to roles which create barriers – break down on teams but need to break down at the HR level
  • payroll is also an issue – need to compensate for people taking on more responsibility
  • need to put queue limits on the testing queue to drive behaviours
  • pairing with developer if they do not understand the scenarios
  • some people have the questioning mindset, some have the practical focus – need both to make sure you ship a quality product
  • mini waterfall problem – long tail feedback loop, change workflow that developer needs to work with tester, avoid lean batching problem
From AAFTT 2011

ATDD Patterns

Jennitta Andrea led this session about the work so far in this space:

Last Mile Tools

Elisabeth Hendricksen led this session on tools that are attempting to solve the problem at the last mile.

From AAFTT 2011
From AAFTT 2011
  • NUnitLiz Keogh – were using Fitnesse but added another level of complication, wrote a DSL that separates tests to make it easier read, WiPFlash is the automation tool, examples are on the website, can call the fixtures from another testing tool like Fitnesse, capture scenarios on a wiki first to get the best out of the automation tool
From AAFTT 2011
From AAFTT 2011
  • SpecFlow – Christian Hassa – similar to Cucumber, scenarios written as steps that are bound to execution, uses Gherkin parser (this is a plus as a number of tools use this)
  • SpecLog – maps of your product backlog, capture results of collaboration with the business (Jeff Patton’s story maps), data stored in a single file, stories are initially mapped to a feature file but ultimately get linked to a feature tree
  • SpecRun is under development currently, not bound to SpecFlow or test runner/execution, currently Windows only
From AAFTT 2011
  • Limited RedJoseph Wilk – uses the probability of failure to run those tests first in Cucumber, can then get failure statistics at a feature level, working on a refactoring tool at the moment
  • Relish – publish Cucumber features to a website
From AAFTT 2011
  • The Smallest Federated WikiWard CunninghamJSON for data scrubbing, thin columns to display well on mobile, refactoring is the number one edit so allow it to drag and drop refactor, fit for any analytic or outcome-oriented endeavor, sponsored by Nike, under very early development, meant to take spreadsheet data to the next level
From AAFTT 2011
From AAFTT 2011

Business Rules

Mary Gorman led this discussion.

From AAFTT 2011
  • business rules – conference website has rules, such as group pack for 5 registrations, what happens to the sixth person, what if someone pulls out
  • need to capture these to describe what our system does
  • business rules manifesto – Mary gives a copy to everyone she work with
  • separation of concerns – a rule is separate from the action which makes the process more brittle and more difficult to test
  • rules are a form of requirements and live beyond the building
  • one process is to extract the rules of a legacy system and then the regression tests – code archaeology
  • the business does not always know the rules of the system or how they got there – rules get added to the system over time or evolve and documentation is unlikely to get updated
  • one insurance company had spent $100 million dollars to bring in a business rule engine, returned investment in two years due to being able to be able to look for conflicting rules
  • put analysis of rules in the hands of developers for way too long
  • simplest part of business rules is having a glossary
  • rules engine enables our rules in productions, and use examples to ensure the engine works correctly
  • testing could look like this – given this data when these rules are applied then I expect this output
  • you need both rules and examples to test them – you need enough examples for now, need to be different paths, decision points, infliction points rather than different values
  • examples are not as expressive as arithmetic, but they are not as understandable
  • lots of rules that we do not think of as business rules because they are baked into the process eg. security access, database schemas
  • “business logic is not” (Martin Fowler)
  • you can’t read English as if it were rules, so we need to use examples
  • the worst systems are the ones that do not have a manual override, humans are usually the best at determining this
  • lots of business rules change due to jurisdiction
  • something will always fall to the bottom – rules need to be valued on risk and value – where is the tipping rule
  • rules are the expression of intent
  • Mars issue – crashed, six week window too costly to fix
  • guts to keep it simple – reporting system (Ward Cunningham) – resisted urge to put in a formula system, wait for requests from users, got 6 requests, sold system based on simplicity of the system
From AAFTT 2011
From AAFTT 2011

Other Sessions

As with any conference, there are always sessions you would have liked to have got along to.

Richard Lawrence led a discussion on Static Analysis for Gherkin which turned into a discussion on design patterns for Cucumber.

From AAFTT 2011
From AAFTT 2011
From AAFTT 2011

George Dinwiddie led a discussion about conversations between roles:

From AAFTT 2011
From AAFTT 2011
From AAFTT 2011

My mate Jason Montague led a session on Building Conditions Conducive for ATDD Adoption.

From AAFTT 2011
From AAFTT 2011

Closing Circle

We shared some takeaways in the closing circle, he were some that stood out at me:

From AAFTT 2011
  • issues with dealing of people was a theme
  • what are good ways to express a large amount of test data
  • challenge to get corporations over the hump to release data, plus have good tests and examples around the rules
  • testing needs to be a nation, not just a community
  • it’s time we got more respect in our organisations, it’s time we show more respect to those we work with
  • teams need to dependent on the production of the build
  • federated wikis could help solve the test ownership problem

As for me, my comment was the day had renewed my energy again. ATDD is hard, and as a community we need to try harder.

Podcast

Finally, I recorded a short audio podcast for The Agile Revolution wrapping up AAFTT.