At the recent YOW! Night in Brisbane (as well and Sydney and Melbourne), Lindsay Holmwood (the Head of Technology at the DTA) presented “Breaking the Cylinders of Excellence”. It was a rare experience to hear the story of how the DTA is using cutting edge development practices to help the government catch up with, and even exceed, the public sector.
- DTA – aid transformation in government, small agency
- Delivery hubs in Sydney and Canberra – help identify and plug capability gaps in teams
- Prototype of how government services could work gov.au/alpha
- Digital Service Standard – 13 characteristics on what good looks like in government, useful in organisations as well
- Cloud.gov.au – government cloud service, usage growing, continuous delivery pipeline (which is a major change for government who are used to 2 changes per year)
- The unit of delivery is the team – not about individuals, but the team – borrowed from GDS
- Government is slow, but government is designed to be stable, they cannot fail, they have characteristics that are resistant to change
- Myth that organisations must choose between speed and reliability, high performing organisations deploy more frequently, have shorter lead times, fewer failures and recover faster, but they also have a greater profit
- Want to deliver like a startup but be stable like a government
- Not a lot of cross pollination between departments currently
- Read the policy! – quite often the process is not mandated
- Document what works and doesn’t so it becomes a repeatable pattern – ie. running a meetup inhouse, don’t tell me I can’t do it, tell me how I can run it without being thrown in jail!
- Stick with technologies the government is comfortable with if you are changing the delivery engine
- Security matters – prevention is a battle you will always lose, detection is your best defence – aggregate and log in one place, identify threat signatures, etc
- Embed security people on big services so it is part of the architecture
- Proactive testing between different governments around the world on similar platforms
- Simplest security breaches make the most mess – infected excel macros, leaving free USB keys in the foyer that are malware infected
- Need to put user needs first – alpha mockup using tools like Jeckyll, then beta then live
- Lots of people strictly interpret the design and delivery guides – they are guides not rules!
- Create a longer runway by pulling tech forward – turn down the volume of design, turn up the volume of tech
- If it hurts, do it more often!
- Fixed cost delivery with agile is a thing, agile is a way to de-risk in the government
- Don’t put manual testing on the critical deployment path – have special skills on hand for accessibility, performance and security
I recently completed reading Enterprise Software Delivery by Alan W. Brown. I read and reviewed the book for InfoQ and the following is my brief review and key notes from the book.
This is a book that sums up any large organisation with a software delivery function. It certainly matches the experiences in my last company but also matches the many case studies I have heard from my colleagues over the years.
It starts by defining enterprise software delivery, noting it uses the word delivery and not development. This is on purpose as software development is only useful when in the hands of the end-user.
The key focus areas are:
- software supply chain and factories – a large portion of the book points to understanding the entire delivery process and understanding how to deal with distributed teams (whether they be within the organisation or partnered or near or offshore)
- collaboration – the importance of collaboration and the use of collaborative delivery environments (CDE) and collaborative application lifecycle management (CALM)
- agile – the importance of agile and approaches to rolling out and scaling in the enterprise
- quality – the importance of building quality in as well as quality of the team and its relationships
- governance – measuring your delivery and managing your team and partnerships effectively
- future – trends that are starting to effect enterprise software delivery including cloud, multisourcing, crowdsourcing and mobile
The Agile portion of the book had me either in agreement or in a couple of cases opposition. I could understand the viewpoint of the author — for example the use of software testing factories which I think removes a lot of the collaboration that is possible with Agile testing techniques and tooling. The author is the CTO for IBM Rational in Europe and as a result a lot of IBM thinking comes through in the ideas. Whilst this is not a bad thing, it is useful to remember that the IBM approach is not the only approach.
The book has lots of case study material (mostly revolving around the financial industry and Danske Bank as well as IBM itself). There are a number of figures and illustrations throughout that demonstrate the points that the author is trying to make (like examples of metrics, processes and plans) and and these alone make the book a valuable asset for those times when you are looking for an example to get started.
This is a book that all technical leaders (including developers, architects and CIOs) should read if they are involved in software delivery in a large organisation. This was a book that I admit I struggled with a little as I read it, which I think was because I had spent so much time in a large organisation that had a continuous focus on governance, sourcing and agile. But on reflection, there were lots of ideas and trigger points throughout the book that are calls to action. Of course, for those just venturing into this domain, it is a recommended read.
Overall, this was a worthwhile read and a book that I would summarise as being very good for lots of ideas rather than a playbook that you follow from beginning to end.
Here are my key notes from the book:
- agile “steering” leadership style – frequent course corrections based on testing, measurement, and trends in defects and cost-of-change
- economic pressures have focused attention in enterprise software delivery organisations on cost-cutting activities, severely stretching existing delivery practices
- CRASH Report – examined 288 deployed enterprise applications from 75 organizations (with more than 108 million lines of code) and found structural errors that on average would require an investment of more than $1 million per application to fix
- must learn how to balance the relationship between necessary investments in enterprise software delivery and the business value these investments bring to the organisation
- inefficiencies and misunderstandings are a major source of errors, frustration, and waste. Clear approaches to communication and coordination greatly improve the team’s performance
- early and continuous attention to quality is essential
- adoption of an industrialised approach places the focus on cost efficiency, reuse, and standardisation in enterprise software delivery
- best-in-class product and services companies are those that have built a strong competency in enterprise software delivery—approaching this as a core business process. Their attention is placed on enabling innovation, lowering costs, and managing change
- Application assembly optimization (AAO) – consist of centres of competency, technology assembly centres, collaboration and measurements and lean processes – uses work packets to deliver work (self-contained unit of work, the receiver designs, plans, and executes the work requested and measured in real-time)
- software testing factories – particularly effective in organisations that are more complex, with numerous departments, vendors, and locations – cost-efficient and effective testing on-demand, standardised business-driven test processes, centralises test equipment, common approach to metrics and measures, centralisation of test service requests
- important insights for an industrialised view of enterprise software delivery – new ideas needed around monitoring progress and managing status, focus measurement on broader SLA’s across vendors, transparency across the chain is critical, choose different partners for specialist tasks and to reduce risk and increase flexibility and competition
- an extreme version of multisourcing is the use of crowd sourcing for component delivery
- collaboration is king – trends – globalisation, outsourcing, faster time to market, agile, end user demands
- organisational arrangements for collaboration – internal partnerships (many companies operate software development as a separate legal entity), supplier partnerships (strategic alignment between both parties), outsourced partnerships (formal SLA between parties)
- global approach – reduce labour costs, skills in short demand, gain access to emerging markets
- points of friction – “energy is lost in their execution which could otherwise be directed to more creative activities that contribute directly to the completion of the project’s mission”
- time starvation – “typically never enough time to complete everything on an individual’s to-do list”
- collaborative application life-cycle management (CALM) focuses on the synchronization of the entire team and the hand offs that occur across domains
- balance delivery capabilities – time to market, productivity, process maturity and quality
- the approach to agility – “an iterative and incremental approach performed in a highly collaborative and self-organizing manner with just the right amount of ceremony to frequently produce high quality software”
- investment should be placed in techniques that support and encourage software to evolve – critical phase comes after first release
- evolutionary view – distinction between these two activities is blurred to the point that development and maintenance are just two aspects of the same need to create and deliver the system
- agility possible when: high-bandwidth communication and coordination, decisions based on real-time information, continuous measurement and transparency into individual, team, and project progress
- impacts on the scaling of agile – geographical distribution, compliance requirements, technical complexity, team size, enterprise discipline, organisation distribution, organisational complexity, domain complexity
- “My experience with several large enterprise software delivery organizations suggests that agile approaches can be successful in almost every kind of development team and in almost all organizations”
- Agile observations: 1) roles most challenged by agile approaches are Executive Managers, Product Managers, and Project Managers; 2) plan at multiple levels and adopt a “measure and steer” approach; 3) don’t use agile methods for everything, match to optimize success; 4) adapt and harden the agile process as the project gets closer to delivery
- National Institute of Standards and Technology (NIST) – 80% of development costs are spent on identifying and correcting defects
- almost two thirds of errors are a result of requirements and design problems, misunderstandings, and omissions, while more than 80% of error discovery occurs in acceptance testing or once the software is placed into production
- improving quality – automating repetitive tasks, discovering more defects earlier in the life cycle, closer alignment between roles, lowering risk
- “the quality of the enterprise software is a direct reflection of the organization’s activities, practices, skills, and morale”
- “the evolution of the role of the software testing factory is toward a utility view of testing as a service that can be readily acquired on a pay-per-usage model or a results-based scheme” – performance and load testing, security testing, compliance testing
- important aspects of governance – establishing chains of responsibility, authority, and communication and establishing measurement, policies, standards, and control mechanisms to enable people to carry out their roles and responsibilities
- challenge lies in knowing which metrics are more meaningful in a specific context and how to use those metrics to change the way an organization performs it tasks – one cannot control what one cannot measure
- a well-chosen set of metrics clarifies the goals of the organization
- measure what you can, what you value, what drives behaviour and what optimises results
- decisions that may be considered optimal in one area of the business (e.g., software testing) may introduce challenges in other areas (e.g., deployment management).
- Walker Royce – analogy between enterprise software delivery and movie making – early stages of the process involve significant “scrap and rework” as part of the creative process and later in the process there must be a great deal of focus on reducing rework to eliminate inefficiencies
- Royce’s ten management principles for agile software delivery
- through-life capability management (TLCM) – evolutionary acquisition – each increment provides useful and supportable operational capabilities that can be developed, produced, deployed, and sustained. Major enhancements and preplanned product improvements are managed as separate increments.
- supply chain operations reference model (SCOR) – ensure that the supply chain is operating as efficiently as possible and generating the highest level of customer satisfaction at the lowest cost
- lean is a philosophy that seeks to eliminate waste in all aspects of a company’s production activities: human relations, vendor relations, technology, and the management of materials and inventory
- Bell’s Law – a new platform emerges about every ten years to offer improved functionality in new areas, accompanied by lower prices for existing capabilities compared to those of the incumbent platform
- cloud-based services are usually viewed as operational expenses because they are “leased” for the time they are needed – the switch from capital to operational expense can be very attractive from an accounting perspective
- “you ship your organization”—the way you organize individuals and teams will undoubtedly have significant impact on the structure and effectiveness of the system delivered