Day 2 at Agile Australia 2010 and another full day of sessions and hallway discussions.

Keynote: Martin Fowler

Martin Fowler gave a “suite” of keynote presentations, including a great impersonation of “Uncle Bob” Martin! He wrote about his trip on his blog.

• agile manifesto has been way more successful than they thought when they wrote it
• semantic diffusion – the message of agile has changed from the original thoughts, but that is the price of success
• The New Methodology
• agile is about adaptive planning and people first
• predictive planning requires stable requirements – very rarely happens as requirements change
• “a late change in requirements is a competitive advantage” – Mary Poppendieck
• plan should be a current understanding of where things are
• adaptive planning needs evolutionary design – the technical side is important
• Frederick Winslow Taylor – father of planning in factories, invented scientific management, invented to control factory workers who were considered lazy and stupid

• if integrating hurts, do it more often
• every developer should commit to the mainline at least once a day – avoid the big scary merge
• Continuous Integration article and Continuous Integration book by Paul Duvall
• continuous delivery
• commit tests need to run in less than 10 minutes – need to stub out databases, look out for slow connections
• acceptance tests are more end to end but they run much slower, you don’t want to wait at commit time
• all environments should be automatically configured, binaries should be stored in an artefact repository
• commit > acceptance > performance > deploy
• deploying to production should not really be treated any different to other environments – sometimes you may need to have a manual deploy step – this should be a business decision because software should always be production ready
• Continuous Delivery book by Jez Humble and continuousdelivery.com

• notion of tradeable quality – trading quality for speed / cost – makes sense when buying a car but software is different
• external quality and internal quality – internal quality is invisible to customers, customers can choose external quality because they can see it
• design stamina hypothesis – can trade design for speed but only up to a certain point – this hits you in weeks not months
• technical debt term speaks to non technical people – paying off the debt and reduce the interest costs
• technical debt cards at least help you have the conversation
• inevitable that you will look at your code in future and think you should have done it better – this is still technical debt

You Can Take Your Agile Maturity Assessment And…

I was asked to be the MC for this session and had the privilege of introducing Marina Chiovetti and Jason Yip from ThoughtWorks. Their slides are available here.

• easy to get started in agile – invest in training, coaches and experienced practitioners – but after a year or two, how do you know if you are investing in the right thing
• looking for what’s really happening – what projects are hiding behaviors that we want to fix
• you can look agile and game the results – what are you looking for, practices are not everything
• create a culture for problem solving and learning – why don’t we have maturity assessments for these things?
• poster child project so people can look at it and see it really works
• successful project, everybody is happy, Prozac is cheaper, are you actually delivering?
• data is important but more important to understand the facts – verify the numbers reflect reality
• spider charts – depends on who is assessing, self assessment is only against what you know, so new teams score themselves high because they only know what they know, experienced assessors also look differently so you will get different results
• disconnect between doing agile and getting to profit – what are we using agile to get better at?
• tick boxes versus understanding the situation – will only change for the assessment or don’t understand why they are doing the change so the change won’t stick
• uncover the behaviours you don’t want to see, even if the teams are successful
• different teams need different measures of success – measure the drivers of success which will be different for each team
• different assessors will yield different results
• agile maturity used like a stick – use across organization, not to manage
• describe your true north (lean) – provide direction so you can determine your next step
• what does the organization value in measures
• make work visible – window into the process
• celebrate failure
• enterprise 2.0 bullseye

Monster Builds

I also had the privilege of introducing Chris Mountford from Atlassian, his slides are available here:

• manager was developer – mxd
• long builds cause low velocity – context switching, task entanglement, build breakage confusion
• tame monster builds through optimization – measure so you know what to fix, look at network and disk access, use repositories like maven, co-locate servers, then measure again
• matrix culling – remove things you don’t want to support – browsers, operating systems, databases, dropped user editions, ancient application servers, old versions of Java – Atlassian dropped build from 60 to 18 hours
• power to wait ratio – unit to functional tests – be selectively continuous
• parallelise your tests – Amdahl’s Law – benefit of running more processes in parallel
• false negatives are expensive – run builds against known good versions of the system to check to see if the infrastructure is broken

Building Quality Into The Lifecycle – All The Way Through

Sharon Robson delivered this presentation, her slides are available here:

• agile makes it real, now, here
• nobody knows what testing is, it’s hard to define
• in agile you are all testers you just don’t know it yet
• ISO 9126 defines software quality – her favourite standard
• cannot define done until you can define what is good
• need to define the what if questions up front, and we also need the answers – what are the things the product should not do
• a good metric for government departments – “will it make the newspaper” or even worse “will it make the front page”
• technical debt – “don’t apply for the mortgage”
• testing is a growth industry – the amount of regression grows every iteration
• testing is about information, risk and making sure it is good
• testing is pervasive – we should be testing everything all the time – process, people, tools, requirements
• everyone in an agile team is a tester and knows what is right and wrong – just need to empower them
• test all your development artefacts – look at everything with your tester eyes on
• agile testing techniques are no different to traditional testing techniques
• story cards are not relevant as artefacts – tests should tell you what the system does
• we live in a world of regression, it is the responsibility of the entire team
• automation is a must have – don’t automate everything but you need technology to allow you to cover a lot of ground very quickly

Open Space

I sat in for a while on the open space topic of testing in agile, which was led by Rene Maslen. Some good conversation and discussion in this group.

Panel: Paying Down Technical Debt

It was great to be asked to be on a technical panel, especially with the likes of Martin Kearns, Adam Boas and Andy Marks with Marina Chiovetti looking after the moderation. Probably most nerving was seeing Martin Fowler sitting in the front row!

Marina initially set the scene for technical debt. Ward Cunningham first drew the comparison between technical complexity and debt in a 1992 experience report:  “Shipping first time code is like going into debt. A little debt speeds development so long as it is paid back promptly with a rewrite… The danger occurs when the debt is not repaid. Every minute spent on not-quite-right code counts as interest on that debt. Entire engineering organizations can be brought to a stand-still under the debt load of an unconsolidated implementation, object-oriented or otherwise.” The Agile community has faced a lot of hard questions about how a methodology that breaks development into short iterations can maintain a long-term view on issues like maintainability. Does Agile unintentionally increase the risk of technical debt?

We then kicked into a number of questions:

1. What does technical debt look like?  How would I know if I saw it?
2. What are some of the common causes of technical debt on teams you’re working with?
3. How do you prioritise technical debt against new feature development?  How do you construct a roadmap for paying back technical debt?
4. What is the cost of tech debt?  How can we monetise it?
5. I’ve heard of one team that had a technical debt backlog that was prioritised beside the product backlog.  The CEO set a target that the tech debt backlog couldn’t grow beyond 10% of the size of the product backlog, and as soon as it got bigger than 10% the team had to start focusing on paying down the tech debt.
   So, how do you monitor technical debt and manage it so that it doesn’t spiral out of control?  Can you give us some examples or techniques you’ve used?
6. Technical debt sounds pretty, well, techie.  How can we expect business/product owners to understand its impacts?  And how can business owners align their objectives to managing tech debt?

Distributed Agile Projects

After the close of the conference, I had a nice long chat with Siva Dorairaj from Victoria University of Wellington about his PhD research into Distributed Agile Projects. I look forward to seeing the outputs from his research.

Agile Australia 2010 was held for its second year earlier this month at the Crown Conference Centre in Melbourne.

Once again I was honoured to be offered an opportunity to present, but I also had the pleasure of being a conference organiser, an MC for speaker sessions on both days, the opportunity to be part of panel discussion as well as running a workshop with Phil Abernathy on the day prior to the conference for those new to agile. The conference attracted 450 attendees from 140 different companies.

One of the difficulties of assisting in the organisation of a conference like this (and I had a similar experience co-producing a stage at Agile 2010) is narrowing down the large number of proposals into a program that will meet the expectations of the intended audience. There was much discussion and debate throughout the process, but I think we ended up with a well-rounded program that covered a wide range of topics, experiences and session types. I owe a huge thankyou to Rachel Slattery from Slattery IT for giving me the opportunity to have a greater involvement this year and the ability to work with the other organisers Phil Abernathy, Adam Boas, Keith Dodds, Martin Kearns, Dave Thomas and Zaz Teoh.

My notes and summaries are below:

Pre-Conference Workshop: First Steps In Agile

I ran a pre-conference workshop for agile beginners with Phil Abernathy entitled First Steps in Agile, which was based on some of the introductory material that is available in the courses offered by the Agile Academy.

In 3 and a half hours we ran through the fundamentals of agile, and had them creating focusing questions, success sliders and story cards before getting them to play with Lego to deliver an agile project.

Executive Breakfast: Accelerating Agility for Executives and Managers

Jim Highsmith delivered this breakfast seminar to special guests and conference speakers.

• there is no more normal
• efficiency vs responsiveness – which one is your model – Walmart vs Google
• business agility – a key resource is David Sull – The Upside of Turbulence
• if agility is a key bus driver, create a Chief Agility Officer
• there are a number of executive levers
• quality matters, Michael Mah from QSM is a good agile data source
• adaptive quality – can you deliver good quality code in the future, key job for executives is to champion a technical debt reduction strategy
• do less – reduce work in progress, requires a lot of discipline
• eliminate marginal value – Standish have found that 64% of features are rarely or never used
• motivate – autonomy (self organizing teams), mastery (getting better at what we do), purpose (working on value)
• doing agile vs being agile – difference between half assed and half done, executives that not only support agile but understand what it means and be part of it
• “Be the change you wish to be in the world” – Gandhi
• agile leaders – it’s OK to be wrong but not OK to be uncertain, need to balance between up-front and just-in-time
• focus on value not the constraints, what is the capacity to release
• agile 101 – need to start with the rules based prescription, but you can’t get stuck there – move from prescriptive agility to adaptive / adult agility
• salesforce.com – 60,000 tests, all one code base trunk
• all agile models are flawed because the world is a complex place – you need a hybrid approach
• quality is the key to future value
• “It’s kind of fun to do the impossible” – Walt Disney
• Martin Fowler said accumulation of technical debt comes in weeks not months, observed over time that it slows you down

Keynote: Beyond Scope, Schedule and Cost – Optimizing Value

Jim Highsmith delivered this keynote, the slides are available here.

• there is no more normal, it is all change – important to adapt to that change rather than following a plan
• 95% removal of defects on a project is where benefits accrue – good companies in the US average 75% so we have a way to go
• focus on speed and you will get low quality and low speed, focus on quality and you will get speed
• a doubling of staff traditionally quadruples defects, so need to focus on quality
• error feedback – errors put into system when fixing other errors
• create opportunities by doing less – scope is a terrible measure of progress
• do most valuable first and choose the right cut off point, most value for least cost
• value engineering – estimate in both cost and value – calculate cost from the bottom up and value from the top down
• say to product owners – “if you don’t have time to estimate value, we don’t have time to estimate cost”
• process maturity models only look at the process and not the value
• parking lot diagram (FDD) to show what is valuable – only goes green when it is deployable – done done
• value and priority are different – sometimes you may need to mitigate risk or deal with dependencies
• may want to prioritise technical debt stories different to customer stories
• every iteration is the product releasable, focus on releasability and does it meet the vision
• agile triangle – value (releasable), quality (reliable, adaptable), constraints (cost schedule scope)
• being responsive – product, adaptive people, agile process, architect with a vision
• strategic questions – can we release today, value-cost ratio, product quality, within acceptable constraints
• getting funding – business cases still very traditional, need to look at funding models and change them over time, find ways to capitalise expenses or deliver early to get the attention of the CFO – ask CFO what appeals to them about agile

Keynote: Delivering Business Value with Agile

Jeff Smith from Suncorp delivered this keynote. I was pumped because he mentioned my name at least twice!

• agile – fundamental change in how work gets done
• good is not good enough, great is not good enough, only being the best is good enough
• change has always been there, but it is quite disruptive right now
• need to change people from thinking that we can’t do things – what can we do?
• don’t need to change your strategy all the time – you need to adapt it to get value but keep your values consistent
• attract, develop and retain good people is the role of good leaders – job is not to motivate, nurture their passions
• shift your thinking – we don’t know best, look at your partners and competitors
• course correction over perfection – it’s OK to make mistakes
• simplify the way people work together to simplify output
• trust and respect the key values – trust takes character (integrity and intent) and competence (capability and results)
• fear of failure is actually masking the fear of criticism
• a great culture – people love what they do, love who they work for (people leave leaders not companies), love who you work with
• collaboration – contribution, good instruction and feedback
• best thing about Yammer is that it is a flat communication structure – increases productivity and removes cascading messages
• strive for level-less leadership – wisdom of the crowd, removes the communication waterfall
• need coaching, like sports teams, to get better
• constraint is a leaders ability to communicate and direct change, not the business ability to absorb it
• Suncorp is twice as productive as they were a year ago – measure performance by the number of productive business changes into production
• inspire people to flourish personally as well as professionally
• need to break down hierarchies, constraint of thought is caused by management
• remove fear about changes by involving people in the process

Old Tricks for New Dogs and New Tricks for Old Dogs

I was asked to to be MC for this session, and had the privilege of introducing Shane Hastie from Software Education. His slides are available here and you can also download his Project Toolkit Matrix.

• dudes law – value = why / how (David Hussman)
• context – sociology (trust, safe to fail, communication), organization, geography
• babies and bathwater – focus on which one it is, what is of value and what can go down the drain
• all the old tricks still have value
• new tricks – value stream mapping, story mapping, user stories (fundamental agile but may not be the right tool), BDD, pragmatic usability (everybody’s responsibility), paper prototypes (don’t craft a long term artifact, craft for conversations)
• agile constants – TDD (every project should do it), refactoring, continuous integration, automated testing, pair programming, sustainable pace, retrospectives – these are just the givens
• introduced a matrix on how to select the right tools (see link above), and encouraged feedback

It’s the Culture Stupid

I also had the privilege of introducing Rob Thomsett, who’s talk was one of the highlights of the conference for me, his slides are available here.

• modern management which dates back to late 19th century, has reached the limits of improvement
• companies make cultural change because they are hurting or need to – explains hotbed of agile in Europe
• most high level executives believe projects take too long to deliver, project management techniques are bureaucratic
• watermelon projects – “green on the outside, red on the inside”
• don’t under estimate cultural change when you go agile – will make you feel naked
• welcome to agile, welcome to a battle – the war has been raging for 40 years
• outsourcing as payback (referenced in Radical Project Management) – issues is there is no payback
• culture is about values – it’s about what you believe in, it’s sucked into you through life
• standing over somebody and asking for an estimate is an ambush – then play guess the number, price is right, doubling and halving game
• Estimation Games – most downloaded article on the website, but is has made no difference
• Fubini’s Law – every agile person should read it

Continuous Deployment

Kane Mar presented this session, his slides are below:

• all started with continuous integration – the best paper is the original whitepaper by Martin Fowler
• continuous deployment is continuous integration on steroids
• who’s doing it – kaChing, Pirum, imvu, WordPress, Flickr, wiredreach
• continuous deployment gives you quick feedback from customers, avoid waste (reduce cycle time)
• easier in some environments such as web, but it is possible everywhere
• requires investment in infrastructure and testing
• adoption – get continuous integration, stop the (commit) line, simple deployment, realtime alerts, test!
• avoid allowing broken tests, unrealistic expectations

Lightning Talks

There were a number of scheduled lightning talks, followed by some ad-hoc talks which I unfortunately missed (including one from my colleague Rene Maslen). From the couple of talks I saw:

Big A vs Small a (Tania Broome)

• little a is about delivery, philosophy, set of principles
• big a is about empowering teams and giving them the tools to get there, it’s a statement of commitment to change
• shi, ha, ri – practice the rules, question them then transcend the rules because it is who you are
• big a agile is shu, risk that we stop here

Agile at Home (Ben Arnott)

Ben is another one of my colleagues, and he presented a great talk about how he uses agile at home with his family.

Building an A-Team: I Love It When A Team Comes Together

The session I presented got an awesome turnout (standing room only and people sitting at the front around the stage)! The slides are available in a separate post as well as here.

I wish somebody had got a photo!

Other Stuff

I spent a great deal of time during sessions and breaks talking to many different people about a range of many different things. In particular, I spent the majority of the last session having an awesome chat about agile testing with Veenu Bharara from Atlassian.

ThoughtWorks hosted an open house in their Melbourne office after the last session which again was an opportunity to speak with far too many interesting people to mention.

The final day of Agile 2010 was all about keynotes. Before they kicked off, the Agile 2011 committee was announced (to be held in Salt Lake City), as were the Gordon Pask winners (it was awesome to see Elisabeth Hendrickson take the award along with Liz Keogh). They also thanked the organisers and sponsors (it was a first for me to have my picture up on the big screen for this reason!)

Here are my notes:

Keynote: Product Centric – A hot new trend for IT organizations

Dave West from Forrester Research delivered this keynote.

• difference between a methodologist and a terrorist is you can reason with a terrorist
• success of any process is a movement – passion and energy
• IT departments are acting like product companies – it’s insane
• pragmatism is important – one size does not fit all
• software is a key catalyst to innovation – was cost and quality but now innovation, speed and agility are top of CIO minds
• software is getting more complex – space shuttle has 2 million lines of code whilst the new S class Mercedes is 20 million lines of code – for new cars, software is 10-15% of the R&D cost
• projects still fail, agile is seeing 70% success however
• disconnect between software and the business – we usually try to bridge the gap with a PMO
• mess of many – we tend to automate business processes and end up with a mess between the business and software processes – we spend a lot of time testing because we don’t know the impacts or understand them, not because we changed a lot
• Agile is the predominant methodology (35%) although the second most popular was no methodology at all (30%), iterative 21% and waterfall 13%
• CSM means you have just gone to your first disco but that doesn’t mean you can dance
• we build better software when we know WE have to maintain it
• organisations moving to product-centric operating model
• people – learn more about the business, better developers are 10-20% more productive than average developers as they deliver real business value
• organisation – measure as a team (need to eat, argue and drink together), aligned to a business process
• process – release and software process is usually separate because the infrastructure is constrained and allows development to deliver, designers being dropped into a team force you to think in a different way
• need clear business traction – need to work with the business to solve problems – deliver a product that creates business value
• measure by business value, be incentivised by business results, learn about the business not just technology
• separation between IT and the business gets in the way of speedy cadence
• much simpler world – simplify the budget process, cut the red tape to get projects started
• IT to business is too complicated – make it simpler, simple value chains and exploit it the simplest way you can
• learnt when they built RUP they made it too complicated

Keynote: Stuck in Ha

Ron Jeffries and Chet Hendrickson delivered this keynote. I found it fun, but I wonder if people who were not familiar with Ron and Chet’s style would have walked away a little confused…

• shu, ha, ri – you are doing it wrong
• scrum is so simple if you actually did it it would have to work – but it is not enough
• we spend a lot of time trying to change the process, we need to change the situation
• we tell first timers to inspect and adapt – unfortunately it can take long time if you are just picking practices
• context is just the decisions that were made before you
• agile skills project – map of things you need to know to do agile well
• Alistair Cockburn is also mapping knowledge
• need to explore the whole map
• Scrum is a doorway to agile, it is an entry point not a destination
• idea from conference – there is lots of ideas out there, and we need to look at every darn one of them
• doing it better is not harder, it is easier — do it better!

Keynote: ADAPTing to Agile for Continued Success

Mike Cohn delivered this keynote, the slides are available here.

• agile is the second wave of OO – many of the luminaries came from here
• the number of lines of code in Ford vehicles has jumped from 2.5 million lines of code in 2005 to 10 million lines of code in 2010
• claim that agile is 5-10% faster – very few teams have achieved this, this is not the norm
• agile is a standard set of things – people started with waterfall-but, it’s not about strict adherence, continual
• process of inspect and adapt and become the best we can be
• doing Agile is easy – getting better results but it is not easy
• agile is not something you become, it is something you become more of
• need to raise the bar on each other
• don’t need to be perfect, just need to be 1% better than your competition – Google is not the perfect search engine, if they were we would just hit I’m feeling lucky

ADAPT

• awareness that there is room for improvement
• desire to change
• ability to work in an agile manner
• promote early successes to build momentum and get others to follow
• transfer the impact of agile throughout the organization so that it sticks

• awareness, desire and ability is what we need to look at when others frustrated, communicate the problem, use metrics (maybe metric per employee), expose the new people and experiences, start with a few practices
• desire – create a sense of urgency, communicate a better way, take a test drive and build momentum, overcome waterfallacies
• ability – coaching and training, hold others accountable, just get started don’t wait
• promote – agile safari at Google to go and see a team doing it right, to see agile in the wild
• transfer – break out of organisational gravity otherwise you will get sucked back to status quo (plus daily standups)
• need to iterate towards agility – use an enterprise transition community in conjunction with improvement communities (formed around the passion of a small number of people)
• create an improvement backlog – create communities and tackle improvements
• return next year more agile than you are today!
• let’s make it software development and not agile software development

The Happiest Place On Earth

And with the conference wrapped up at lunch time, I used the rest of the day to visit Disney Hollywood Studios and Magic Kingdom!

Day 4 at Agile 2010 and another full day of sessions followed by the conference dinner at Epcot. Here are my notes:

How to Own a Really Big Complex Product

Mike Cottmeyer presented this session which is also available on Slideshare.

• product owner is responsible for stewarding the team, creates and prioritizes the backlog, elaborate requirements, communicate vision, represent customer, QA the product
• product owner is a single wringable neck, one person responsible for what we want to build, with multiple product owners who decides what to build?
• embodiment of traditional roles – product manager, project manager, business analyst, quality assurance, management, user experience and a team member – overloaded!
• complex product – intersection of multiple products we need to deliver against
• need to think about our organisation differently – challenge is always to get a cross functional team that has all the parts of software, hardware, etc (there is a scale where this does not work)
• many to many teams working on products – teams may have good velocity but the organization or product does not – sequence backlog across team to ensure goals are being met
• performance indicator is not velocity of the team – performance of one one team can starve performance (if one team is not pulling their weight)
• typically we have features that spread across multiple teams, not always working on the same thing because they are blocked or not synced
• intermeshing features leads to time blowouts and context switching – end up running late, dropping features and all hands on deck
• look for teams that are the constraint – remove their non essential work so that they are getting value out
• need to take systems thinking approach and look for team constraints
• look for patterns as to who is fulfilling the product owner role – often tends to be the development manager
• need to express the product owner capabilities at team and organisation level
• scrum of scrums have turned out to be dependency management meetings for scrum masters, need to do the same for product owners – make sure that product backlogs are not starved, include architecture or a senior technical resource in this “product owner team”
• manage and respect the dependencies between teams
• recommend taking a look at Agile Management for Software Engineering: Applying the Theory of Constraints for Business Results by David J. Anderson
• kanban – can help the organization constraint problem, express the enterprise flow of value
• architect – creates strategic technical constraints for teams
• it’s about team, team, team, conversation, conversation, conversation – need to trust the organisation has ethics to achieve decisions – most executives make hairbrain decisions because they don’t have the right data

Refactor Your Retrospectives

Rachel Davies delivered this tutorial, a copy of the presentation is here.

• Project Retrospectives: A Handbook for Team Reviews by Norman Kerth was the original text on retrospectives in 2001
• not originally part of scrum – a small part in the Agile Project Management with Scrum book
• for a good overview of retrospectives take a look at the Agile Retrospectives book as well as Agile Coaching
• heartbeat retrospectives- challenge that 2 weeks not long enough to do things, what to do differently is sometimes not enough to ask
• more people doing continuous retrospectives, especially with kanban
• helps if you limit the number of actions – short amount of time and not enough time to do it
• prepare for your retrospective – bring in data, book rooms, bring equipment, etc…
• after the retrospective – publish the outcomes, make them visible
• have the participants prepare

Smells

• too much time living in the past – not creating actions
• judging – spending more time on what happens rather than the cause, and just making analysis rather than actually making changes
• cloudy thinking – spend more time thinking about practical steps to fix the problem
• fixing symptoms – talk about the real problem
• blaming – criticizing others, especially those not in the room, need to steer the conversation back to what the team can effect, can we build bridges or invite them to the retrospective, get multiple teams together to tackle issue
• unconnected ideas – have people who are keen and not keen, need buy in from team that it makes sense
• thinking too big – ambitious action lists, can we fix in the next iteration, decrease velocity, pick the top few, decompose large actions
• no owner – team recognizes problem, need somebody excited to change it, should not always be the scrum master because the team are not taking responsibility and action
• invisible actions – make the actions visible to the product owner, hard if the product owner is from management
• activities that trivialize the team – vary retrospectives but respect the team, sometimes have fun activities and sometimes need for discussion, be aware of things happening in the environment
• picking on people – maintain safety, ensure meeting is about process and work not individuals

• important retrospectives affect some kind of change
• respect people are bringing time to the meeting
• do it differently – respect visual versus post it’s versus standing versus sitting
• a bridge between sprints – should be half looking back and half looking forward
• are sometimes about improving our working agreements
• turn the topic to “why can’t we do retrospective actions”
• don’t rush – allow time, get the story out and listen to the team, need about an hour
• gather the data in a timeline – written as well as verbal, everybody participating
• get people to draw what it felt like working on the last sprint
• dot voting – need to build consensus, but can also lead to disagreements
• encourage many solutions to an idea
• make things visible, write them down, bring them to the next one
• use agile techniques to run and build into plan
• read books on facilitation and try different techniques

Kanban Explained: Seeing, Not Hearing Constraints

Jon Stahl delivered this presentation to a packed room, the slides are available on Slideshare:

• kanban is another practice – you don’t eat the whole buffet, you just pick what you need
• at Toyota doors are requirements but also are physical inventory
• 3 rules – strict queue limits, pull value through, make it visible
• set limits for active states, not wait states, relative to your team size, no more than 2-3 in wait state or per person
• use value stream and FIFO (First In First Out)

• everytime a developer stops work to help QA, put a tick mark, we now have a discussion point
• waiting point queues are for slack in between, just for buffer
• show and tell column for feedback, when we reach the limit schedule while it is still fresh, also may be a signal to deploy in kanban
• use bingo doppers to mark features – it is just fun
• can change the priority for items in the wait queue to help
• push through a feature
• make your queue signals visible (rules) eg. if architecture = new, trigger architecture review meetings
• what goes on card – date card created, work in progress (WIP) start and done dates to measure lead time and cycle time, then create a metric from a realtime average
• when you get a flow issue from your metric, do a root cause analysis, throw it in the pile for next retrospective
• you don’t want things to flow backward, need to call a plumber! – write on the back the date it went backwards

• Disney are good at wait time – track your average wait time to done (customer delight) in your analysis queue and your backlog (should be the same number of WIP as your kanban board)
• a card is a token, a symbol of work, gives you courage to have a conversation, there is only so much WIP
• takes about 6 weeks to figure out your WIP when starting up
• there is no “always the case”, this is it
• estimates – extra small (XS) < 1 day (1), small (S) 1-3 days (2), medium (M) 4-6 days (4), large 7-10 days (8)
• every implementation – the business is the bottleneck
• team signals – put the team rules up on your board, put meetings up on wall so you check it is adding value
• retrospective kanban board – when you hit a limit, have a retrospective, keep a retrospective kanban board for actionss
• flow is continuous, no open or close (show and tell and retrospective triggered)
• scrum versus kanban, who cares – worry about pace and consistency
• standups – how are things flowing, talk about blocks, take turns talking to the board, walk the board and talk when you need to – stops the ceremony
• Jeff Patton talks about an express lane – have signals and make it hard
• look at Kanban by David Anderson new book, Limited WIP Society, the Yahoo! kanbandev group
• distributed teams – have a board in each location, get them to measure and give data back
• cross team communication is 25% waste

Bloody Stupid Johnson Teaches Agile

Rave reviews of this session by James Shore and Arlo Belshee led to an encore performance, which more than lived up to the hype. There is a great review of the original session written up by Dave Nicolette on his blog.

The play was very entertaining (and way too true in some regards) and the audience participation was spot on. I heard a rumor the original session was recorded (I certainly hope it was as it is well worth a viewing).

Project Management Stage

Once again, I dropped in on the sessions that were happening on the Project Management stage throughout the day, there were some real standouts that in hindsight I wish I had attended. We had a number of talks in multiple rooms.

• Institutionalizing Scrum (Brian Bozzuto and Michele Sliger) – this was a well received talk and it was great to meet Brian in person as we had collaborated on the reviews for the stage.
• Save 76% – Joys and Pitfalls of using Virtual Worlds for Remote Teams (Bill Krebs) – this was a talk I was keen to have on the stage, and it was awesome to see Bill demonstrate in real time a virtual standup in Second Life and Telepresence with State Farm.

• Project Vital Signs (Stelios Pantazopoulos) – it was great to meet Stelios and I got to spend some time with him at various stage throughout the conference. This is an excellent presentation with different ways to demonstrate the health of an agile project.

• How an Agile Project can Fail and what to do about it (Ahmed Sidky and Sara Medhat) – Ahmed was the co-author on Becoming Agile and it was good to see him again this year at the conference, and to see him speak from real like experiences.
• Estimation Games – Play to create and destroy good estimates (Pascal Van Cauwenberghe) – Pascal came highly recommended for this talk on the stage.

• Agile Projects – Beginning with the End in Mind (Bob Galen) – this was another good session, and it was good to finally meet Bob this year (he was also at the AAFTT workshop I attended earlier in the week).

Conference Dinner

In typical Disney style, this was a well orchestrated affair of dinner, a show followed by Fireworks at Epcot. Unlike usual conference dinners, it was a little unfortunate that there was not time to mingle and chat afterwards. However, the entertainment was awesome, particularly the celtic rock band Off Kilter, and the night concluded with the 10pm fireworks at Epcot followed by 2 hours to experience the attractions at the park (Test Track baby!)

Day 3 at Agile 2010 and a full day of sessions including my own in the first session. Here are my notes:

I’m The Business & Agile Was My Idea

The session I presented got a good sized audience and the session feedback forms were mostly positive. It was awesome that many people that attended pulled me up in the hallways and commented on how much they enjoyed my talk (makes it all worthwhile). The slides are available in a separate post.

The following photo was taken as I was setting up (before anybody showed up).

This photo I took just after my introduction (I told the audience at the time that I needed to prove to my boss that people showed up to my talk!)

Beyond Sprint 0: Using Collaborative Product Discovery to Plan Agile Projects

Jeff Patton presented this session, the slides are available here.

• simple agile process is an effective delivery tool
• definition of ready – thrash a lot in a sprint, set a sprint goal
• how do you decide what to build?
• how do you know when you’re ready to start building?
• figure out walking skeletons – the thinnest thing you can build that will walk
• balance discovery with delivery – developers are good at delivery but how do we understand the right product to build (just because it is burned down doesn’t mean it’s good)
• difference between context and solution – business strategy, customer segments, user goals, product usages, regulatory constraints and business process are contexts that help you build the solution
• target the solution to maximize outcome not output – have a starting context which leads to great ideas which we then increment and deliver – check we get to the resulting context which is an outcome with happy customers – focus on outcome not output
• backlog describes what to build but not always why
• velocity measures output not outcome
• difference between choosers and users (those who buy and those who are stuck using it)
• outcomes – look at goals, users and the activities and tasks that they do – trim your backlog and eliminate activities, which will eliminate customer segments which will eliminate goals
• deliver collaboratively – green light projects like you would a movie – collaborative workshops with the right mix of people (people who know things, care about the outcome, people who will get it done and people who can and will make decisions)
• “pair up in threes” (Yogi Berra) – need less structure, decisions will come quicker
• simple models – avoid words that go into the air, visualize so people share understanding and get to a better idea
• 2 people at a whiteboard is the most effective and rich communication channel
• affinity diagrams – clump
• chronological models – how things play out over time, left to right
• decomposition and adhoc charts
• blend your models and annotate – no particular model, rough and simple
• document models using photos or short video
• time box and sprint through discovery – small team to plan and focus this activity, 1 to 3 weeks ideal, months is usually analysis paralysis, start with morning standup and end with daily standout, multiple 60-90 minute collaborative workshops at beginning of the day and then paired small groupd work to spike code, etc, weekly outcome review at the end to share what you have delivered – high visibility inside team and share outcomes outside the team
• check – are we confident in our release plan, what is our confidence
• collaborative product discovery practices
• build simple lightweight pragmatic personas – learn what you don’t know about your users – who they are, their objectives and values
• interview users to build value – understand that you are building the right thing – we should behave more like doctors than waiters – we are trying to help people, ask question about them
• gemba – in lean, the scene of the crime, research informally and in context (visit users and learn)
• build a story map to describe a personas experience, build a scenario to hypothesize a solution
• build a timeline map to show flow across the system
• design studio technique to visualize user experience
• validate solutions with a simple prototype session with users
• start with a product goal – what is the outcome
• collect underpants… profit – many projects in software look like this

Testing Olympics! Teams Compete to Solve Agile Testing Challenges

This was a practical workshop led by Nate Oster:

• squads of 6 , gold, silver and bronze voted by other squads
• separate symptoms from causes
• guess my dysfunction game – agile testing cocktail party – cards with dysfunction, guess before time expired, based on Party Quirks from Whose Line Is It Anyway, use a buzz in bell

• crushed under the weight of tests, lack of technical skills, crushed between wall of tester and developer, brittle user interface
• second game was agile public safety posters for Acme Widgets – not practices but identify root causes

Agile Program Management: Another Approach to Large Projects

Johanna Rothman presented this workshop on the Project Management stage. The presentation is available here.

• concurrent projects going for one product or managing phased projects are out of scope – harder issues to solve
• how do you manage interdependent or relatively independent teams – issues manage up down and sideways, understanding status, managing backlog and architecture decisions
• teams commit, not functional managers
• product owners manage what-to-build risk
• program managers use agile approaches
• timebox requirements to 1 iteration
• not keen of many iterations of iteration zero

We then kicked into an exercise to simulate product managers, using materials to create paper flower baskets. The following are the creations of the group I was a part of:

Project Management Stage

Throughout the day I made sure that I dropped in on the sessions that were happening on the Project Management stage that I co-produced with Greg Smith that I did not attend or present above. All of the talks were well received by the audiences:

• Estimating and Managing Fixed Price Contracts For Agile Projects (Mark Balbes) – it was great to meet Mark and looks like the presentation was well received
• Case Study: First Time Implementation of Agile at a NY State Government Agency (Pat Guariglia) – this was a great case study and it was great to finally meet Pat in person as we had collaborated together on the reviews for Project Management stage

Day 2 at Agile 2010 in Orlando kicked off with the conference overview and keynote. Jim Newkirk reported that the conference had 1,400 attendees from 38 countries to attend a selection of 227 sessions. Here are my notes from the day.

Keynote – Agile 2010 – An Unplugged Retrospective on the Agile Decade “Mirror Mirror on the wall are we really the most beautiful of all?”

Dave Thomas presented the conference keynote:

• testers were getting drowned at the bottom of the waterfall, developers and analysts were in the clean water
• only answer you can give as an estimate is the wrong answer
• if you only do one thing, put continuous integration in place
• agile has delivered improved productivity – some, but not the 5x as some were hoping
• software is still built by people, business analysts have been missed largely by agile, the product owner role is overloaded
• sprint zero – the original scrum paper said it may take 3 months, innovation can’t be done in a single timebox
• collective ownership – no piece of code should have one owner, use four eyes, not always 2 people holding the mouse, don’t be religious because pairs don’t always work but use four eyes even if traditional development
• TDD – was originally test driven design, most significant piece of engineering this decade, has solved some amazing problems, acceptance tests worth much more money than unit tests if you have to make a trade off, parametric / random testing to test hard things like compliers
• an XP manager is an unemployed smalltalk programmer
• agile is successful because the methods and tools people are here!
• your method is the set of practices that work for your business – practice first
• if you can’t beat the card, you will only do it worse with tools
• shut down the laptop in meetings, should be about people and interactions not talking to a machine
• certification – important that people acquire bodies of knowledge, but is only the starting point, favor concept of apprenticeship because it is about confidence
• return to craftsmanship – cooperative education programs in universities, passion for programs that look good, practice the kata to retain your confidence, learn the keyboard chart before you pair
• hire them young , they don’t know what can’t be done
• clean code removes technical debt at it’s source
• big ball of mud design pattern – it’s our mess, we create the garbage and assets we cannot fix
• refactoring is a wizards work, the tools are toys, often we are tied to the data not the code
• lean – so simple it is so hard, all you need is the ability to think, if you can’t measure it you can’t improve it , need to measure whole value chain, bottlenecks usually not where you think they are
• cumulative flow much better than burndown, see where bottlenecks are
• it’s about the practices stupid
• read “The Mythical Man Month” (Fred Brooks) – you ship your organization
• need a technical ladder alongside your management ladder – recognize distinguished engineers, don’t have to have people reporting to them, they become the coaches and walk around, they may make as much as a senior executive and get access to the executive team
• communities of practice – best way to learn, done in own time, mark of a professional that gets off their butt and learns in their own time, reward these people
• scrum and agile does not make the difference in the large, lean thinking makes the difference
• automated tests are an investment
• employ smart people who have failed before (they have seen the movie)
• if you have a bad backlog it does not get better, have tangible testable requirements, without an acceptance criteria you have a wish not a requirement, non-functional requirements are nonsense as they are all requirements, if you can’t measure or test it they are not requirements
• envisioning – build requirements through design to make it tangible, many good requirements are not expressed in text
• Apple spends more on envisioning than any company in the world
• estimates – stupid questions that managers ask, answer should be “buzz off, here’s some points”
• always respond to an estimate question with a range, want the manager to be responsible for the risk, can do an activity estimate to get a risk window, play the estimate game frequently and early
• architecture – we all do this, it is not a job, architecture should be an interface and let the code draw the diagrams
• customers buy features, but components keep the code base alive, rank component backlog on feature level

Beyond Agile

• it is still perceived as an expense, learn about teams pain, agile is fragile as it depends on sustainable leadership and discipline, good leaders have magic powers
• software is hard and there is too much of it
• skills are in short supply, good luck finding a crack graduate, have processes that are not dependent on anyone, teach six hats early, give people reasonable tools (Google App Inventor for building Android applications)
• need to get away from low level languages (Java is just a VM language), need higher level expressive languages (read structured interpretations of computer programs), make applications data driven
• adopt better business practices – pair business and programmers, we spend months on budgeting without blinking an eye so improve this
• winning cultures and teams work well and do not follow a process

Agile Business Intelligence – 12 Critical Steps to Success

Ken Collier presented this session. This was a good session, unfortunately the good stuff was at the end and Ken ran out of time at the end for the a-grade material and the myriad of awesome questions.

Ken started by getting people to stand in the room on x and y axis – x (across the room) for agile experience and y (fromt to back) for amount of business intelligence knowledge.

• why don’t we hear stories about how business intelligence has turned a company around – have done a lousy job of getting business value to customers
• data modellers love to model data – overbuilding of systems as they model for future or unperceived needs
• core values and principles apply – envision –> speculate –> explore –> adapt –> close
• TDWI (The Data Warehouse Institute) – lots of buzz about scrum but it is not the complete picture
• highly recommended reading – Agile Project Management (Jim Highsmith)
• shared community of planners, doers and users – responsibility shared
• business intelligence customers – don’t often know what these systems do, so we need to show them features early (eg. OLAP reports, visualization dashboard)
• iteration zero – opportunity to get infrastructure in place, logical/conceptual data models in place, explore user stories with customers, system discovery and understanding source system data
• advocate 60-90 day release cycles – should not be all needs, focus on a division or goal and create a bounding box
• in database development historically work in a shared environment and when we think it is ready we migrate into production – cannot experiment, explore, do good unit testing
• sandbox development – need to check in artifacts into version control, use continuous integration, use virtual servers to mitigate cost
• with 4GL tools for continuous integration – recently used Hudson, as long as the tool supports a command line API to run the modules that are created by the tools, AnthillPro has plugins that help
• DDL/DML to keep in sync with developers – build from the metal up using the continuous integration server, try to move toward push button deployment that will build the server, need to be able to smoke everything on a regular basis (scorch the earth build) to keep in sync
• what about production systems? – can’t recreate the data, customers have reports, it holds the data and is the one point of truth, need to maintain consistency, read Scott Ambler’s Refactoring Databases, use continuous integration to build and rebuild, practice deploying using your technique of choice, blue/green deployment, run parallel until confident
• think creatively on how to deliver value in short iterations – feature spike and then build
• a story is done when it adds value to the customer and goes all the way back to source system data – stories quickly become epics in business intelligence
• customer collaboration is usually poorly done – don’t just rely on the business analyst, need real user stories
• if you are proud to show your customers features, you have waited too long to show your customer features
• data modellers argue they need months up front – wrong headed, nothing unique, model in increments and prove in code
• agile is not a substitute for technical excellence
• should always be able to deploy into production
• automate everything
• unit testing – look at DbFit, fixtures for fitnesse, previously used SQLUnit using Ant, test data not features, write tests first and then ego

Improving Customer Conversations

Esther Derby delivered this session:

• not so much how to have a conversation but how to interview
• open and closed questions – closed questions are yes/no but open questions invite conversations
• 5W questions – ff asked in the past, you get how customers are using the system, what problems, etc… whereas present questions give different perspectives such as who else should I talk to, etc… and discover possibilities (would and could questions)
• try and go and see your customers where they work and how they use your software
• points are questions that are used to confirm a specific piece of information- yes and no closed questions
• priorities and preferences – confirm information using multiple choice
• probes – probe more deeply, follow up questions to get below the surface, interviewers are usually good at this
• always asking questions to understand why, but why questions often put people on the defensive
• alternative to ask something like what was the thinking behind this, what are the reasons, etc…
• meta questions – questions about the questions, are there questions that I am not asking that I should be asking, useful when customer assumes you have domain knowledge but you don’t
• 3 cardinal rules – never argue with the customer (if you win you lose the customer and if you lose you look like a dummy), don’t turn an interview into a training session (you will never come back to the interview and looks like it is about you and your vast knowledge) and don’t talk more than the customer (no more than 30% of the time)
• always write questions in priority order so you get through the important questions
• make some connection with the customer so it doesn’t feel like you are just mining for information – ask how long they have been with company, etc…
• preparation helps and builds confidence for the interview – makes good use of your time and theirs and show you value their time
• useful to test questions before you take them on the road – our minds work in different ways, make sure that you are asking understandable questions
• customers do not know what they want and don’t know how to speak in requirements and stories, questions can give the customer ideas
• when a customer starts leaning back in chair or shows body language, ask if asking to many questions and who else might be able to answer questions
• set what you are going to achieve when you are making the appointment
• if the customer is technical and tries to tell you how to design, ask questions about what this will gain, need to understand how their world works
• questions not to ask – leading questions (obvious), bias questions (of the interviewer, feel manipulated, wouldn’t it be better to use .Net), wouldn’t you agree, compound questions (questions in questions, interviewee does not know where to start, one question at a time)
• start with more general questions, most familiar and most important to your goal and then drill down
• 10 kinds of people (2 kinds of people in binary) – think outloud (extravert – will keep talking until you interrupt) and those who like time to consider an answer (introvert – looks for a break in the conversation) – mixed pairs are tricky, if not answering your question directly, give them time to think (8 seconds)
• over the phone give people time to answer
• when asking a question, never be the next person to speak
• after the interview take notes and review them to see what you are learning about the features
• almost never do transcripts for interviews because human language is full of useless words, may paraphrase for later
• get somebody to takes note if possible
• what else is a useful question
• if they don’t something like ask them what they would like instead

Life’s not a Beach, it’s a Game: Innovation Games® for Agile Teams

Luke Hohmann and Cory Foy led this workshop to introduce a new innovation game.

• companies play games
• fuel innovation – what you know, know what you don’t know (requirement gathering) and don’t know what you don’t know (innovation games)
• Prune The Product Tree – branches are features and releases radiate up the tree, prune the apples off the tree
• the new game – Draw your nightmare scrum master, product owner or developer

Project Management Stage

Throughout the day I made sure that I dropped in on the sessions that were happening on the Project Management stage that I co-produced with Greg Smith. All of the talks were well received by the audiences:

• Building a More Accurate Burndown: Using Range Estimation in Scrum (Arin Sime) – it was great to meet Arin and the parts of this presentation that I attended covered this topic well
• Scrum and Kanban – Like Chocolate and Peanut Butter (Damon Poole) – the presentation is available here, it was great to meet Damon before the talk as well as at the AccuRev stand, they got me hooked on Reese’s Peanut Butter Cups however (I wish you could get them in Australia!)
• Fixed-Bid, Fixed-Feature, Fixed-Time Agile Development (Dan Rawsthorne) – it was great to meet Dan, his presentation is available here
• Agile for the Practical Project Manager (Greg Smith) – Greg has presented a version this talk to rave reviews across the USA, and watching the audience and questions at the end of this session, this version of the talk was no exception

For the third year in a row, I was lucky enough to get a talk accepted at Agile 2010 and the support from Suncorp to send me along to speak. I was also honoured to assist Greg Smith in co-producing the Project Management stage for the conference. Whilst its a been a few weeks since the conference (I took some time afterwards to explore Florida and the south-east of the United States) I wanted to ensure that I posted my notes.

The sessions I attended on day 1 were as follows:

Patterns for Customer Interaction

Linda Rising led this session, and whilst I was looking forward to seeing Linda speak, this session was way to drawn out to fit the 180 minute timeslot and as a result I left part way through. The customer interaction patterns are available on Linda’s website, here are my notes for the part I was present for:

• common sense is not very common
• Christopher Alexander, a building architect, came up with the idea of patterns, gives us a name for something so that we can talk about that idea
• Einstein got 10 hours of sleep per night, the average these days is 7 hours, short term memory drops when less than 7 hours

Foundation patterns

• it’s a relationship, not a sale – we are always focussed on the the task af hand, but handing over the solution is just the beginning, a customer does not know until they see it
• know yourself – believe in your own capabilities, agile is a methodology that starts by knowing it’s values, what is your mission
• know the customer – learn about your customer, know as much about them as you can
• build trust – difference between trust and respect

Agile Estimating & Planning: From Basics to Brain Stumpers

Mike Cohn presented on the Project Management stage to a standing room only crowd (I was sitting on the floor at the side of the room). The presentation is available here.

• release planning – looking ahead 3,6 or 9 months
• sprint planning – break down, 2 hours of planning
• good plan – when done you can look back and be happy with it
• “do the planning, throw away the plan” – Mary Poppendieck
• anchoring – tendency for an estimate to be as long as the estimate you last heard
• estimate size then derive duration – separate the steps, estimating the size is hard (try it out, compare to other examples)
• story points – how long will it take based on complexity and uncertainty, story points are relative – example walk to a building in 1 unit of time, next building in 3 units of time, holds true if I am walking or on crutches
• relative estimation with 100 stories in a backlog – get team to randomly read the backlog, start with a couple of the smallest stories – find a good 2 and a 5, then compare as you go
• calculate confidence in historical velocity, throw out observations after 8 or more sprints (the top and the bottom sorted velocity) (Mike has a calculator for this)
• sick and vacation for most teams – normally can predict that things will be the same now and in the future
• common sense trumps all when calculating velocity
• to determine the finish, give management a velocity range – low velocity will definitely get to, aim for the medium and the most we can hope to achieve
• velocity for a new team – forecast velocity
• get a backlog, break stories into tasks and hours, agree they can commit, repeat until iteration full for the first sprint
• may want to pull items off the backlog randomly
• if you can’t use a real team, find a substitute team
• best option is to use historical data or stall when somebody wants an answer of when
• work will be done in natural order, typically always turns out logically
• story points only for stories, assume right person will always do the work, estimates will balance out across the team
• get team to do personal math on how many hours they can commit to the iteration
• drive by commitment, not by velocity (backend the velocity)
• story points are a useful long term measure, not short term
• use exercises to get team useful in estimating, takes Mike 1 day
• velocity usually stabilizes within 3-5 sprints
• express point velocity as a range, take a guess, calculate a range percentage based on other teams standard deviation

What happens to velocity if team size changes?

• if team size changes are consistent, don’t do anything
• stop changing team members around – study that teams need a new team member every 3 years to stop being stagnant
• keep spreadsheet of percentage changes over 3 iterations – team from 6-7 people went down 10 percent, then down 5 percent and then up 13 percent
• fixed scope and time
• multiply low and high velocity by date and give a range
• everybody is doing some sort of contract development, going with bottom velocity makes you a short term hero with long term risk – trade off of short term hero and long term risk, where do you want the pain?
• long term estimation > 6-9 months, treat like a budget, look st epic feature level stories and see if it us credible to do it in a quarter
• look at velocity and see if cards look too big
• typical successful team can do about 1 to 1.5 stories per person per sprint

Uncertainty

• what’s my realistic worst case – 90% certainty, what’s my best case and find a buffer in the middle of the overall
• give 50% and 90% numbers
• to get a buffer add all of the 50s and some of the 90s – sample buffer
• do this when there is a high cost of being wrong – front page news, heads will roll – do not use all the time
• if you don’t have experience, stall and get some
• re-estimate when things take longer than you thought – re-estimate those cards when you know they are bigger and you know where they are, don’t reestimate random screwups, get customer to reprioritise

Dependencies

• which card do you do first, whole backlog should represent the entire size, work in natural order, if wrong we switch points, if horribly wrong might need to re-estimate
• do technical stories rarely, pulls value away from story, only if large infrastructure beast that might need it’s own sprint

Throughout the session, Mike ran a great example of estimating, giving the audience some trivia questions and getting us to answer with a range estimate (for example, how may airplanes did the US military own in 1913?). It’s amazing how far off some of our estimates were! (if you were wondering, the answer to the example was 23 aircraft, I was way off!) There were also a number of other exercises around estimating that can be seen in the presentation.