The inaugural Agile Alliance Meetup in Brisbane kicked off on May 13 at the Hilton, Brisbane with a talk by Kristan Vingrys (Global Test Lead for ThoughtWorks) talking about the changing role of a tester. I have had the good fortune to work with Kristan a couple of times this year, so was looking forward to catching up again as well as seeing this talk.

Among other announcements, Robin Mack is the Brisbane liason for Agile Alliance Australia and they are currently looking for volunteers, sponsors and speakers.

The following are my notes from the session, the slides are also available online:

• testing – find out what system is and what it does
• testing is not about putting a quality assurance stamp on software before it goes out the door, its about understanding the quality of the application
• traditional testing – V model – independent, based on a feature complete system, exit criteria from each phase
• collaboration – more related to agile – single velocity for entire team (not just developers), co-located and interspersed team, loose boundary of roles, work closely with developers (not throwing information over the wall)
• should not be iterative testing (testing 2-3 iterations behind), independent or separate velocity
• stop finger pointing, no longer a gatekeeper
• tester should be able to check out code, try it out whenever they want to
• tester should have direct contact with business to break down direct animosity with the developer
• measured by value added to the team not by defects found or number of tests written
• test early – make sure the story is testable, do all testing earlier (including non-functional testing such as UAT)
• Acceptance Test Driven Development – building quality in, not testing it in, this is not just about automation
• handover – developer should call tester over to ensure that the requirement has been met, give the tester the keyboard for 2 minutes to see if they can break it before check-in
• exploratory testing – difference between exploratory and ad-hoc, it should have a plan, allows to execute test cases while learning the system, a good approach to test the tests, looking for different ways to learn about and break the system
• automation – a good way to continually do regression testing, it is not a test strategy, is not just tests (can also be used for automated assisted testing, trawling through log files, etc), automation is code so treat it like code (not using good behaviours will end up as brittle code)
• unit testing – benefits the tester, know what comes from the developers has been tested to a certain level
• continuous integration – push the regression suite so that it becomes the responsibility of the team
• pairing – way to transfer knowledge, to and from the tester, focus on higher value testing
• co-location – can talk directly to the team about issues or further understanding
• standups – understanding what the team is doing, what they are working on, know areas of system to spend time on or stay away from
• automated functional testing – not running the same test again and again and again
• ratio of testers to developers – hard to gauge, depends on the skills, not everybody on the testing team needs to be a tester
• test analysis – what are we going to test, understand requirements, architecture, test execution (by a human or a machine), environment management (testing the right thing, a stable environment)
• “you can’t find a defect if it doesn’t exist”
• troubleshooting – being able to track down a problem, find the artefacts and hand it off to the correct team to fix
• every role on the team should contribute to testing in some way
• metrics that you collect change when you move from waterfall to agile testing
• generating a test report should not take very long – should be an indication as to whether someone needs to have a conversation with me, answers questions about the state of the test team
• people make assumptions on vanilla metrics

Metrics:

• points in test over time – number of stories that are in the testing column on the storywall, indicate if the team is keeping up with testing activities
• outstanding defects over time  – how defects are tracking over time
• number of bounces per story point – bounces back and forward between the development and the testing columns

• not all defects need to be logged – only needs to be captured if it is not going to be fixed straight away, better to spend time on automated test to check defect will not come back rather than logging it
• be aware of fake defect debt – when a story gets pushed through and defect/s get raised to be fixed later
• add value by getting team focused on quality

Then there were some questions from the crowd:

• environments – want enough environments to avoid test scheduling (where possible), happy to do UAT in an earlier environment if it means you are getting early feedback that is useful
• tester velocity – should be focused on the velocity of the team and the definition of done, investigating using Kanban techniques to set a limit on a column (perhaps for testing to enable team flow)
• automated tests – depends on where business logic, for AJAX you need to do more UI tests, main issue is getting tests running as quickly as possible and getting fast feedback, prefer not to test via the UI because it is very fragile
• skillsets – pulling a team together will look at troubleshooting and may need to hire somebody just for this task, a team with developers that strong minded makes it hard for a tester to get their opinions across
• test plans – does not necessarily need to be a document, the important thing is that there is a shared understanding, should never be signed off because it should be constantly revisited
• skilled team of testers and developers – who should write the automated tests – look at skills, should not be written by those whp do not understand programming or test analyst – answer what I want to test and how do I go about developing it
• regression test definition of done – business need to articulate the level of risk they are willing to accept, understand where the business value is in the system, finding a defect may encourage you to spend more time