Two hidden assumptions of agile

Key takeaways

• The founding members crafted the agile manifesto to synthesise how they tackled a particular set of problems, specifically problems where responsiveness to change (agility) is valued more than efficiency and minimising opportunity cost is more valuable than minimising cost.

• Enterprises should organise differentiating capabilities for breakthrough knowledge acquisition to uncover differentiated value. While optimising utility capabilities for operational excellence to deliver efficiencies.

• Agile was designed to power differentiating capabilities and operates on two key assumptions that are not explicitly stated.- 1) Agile is best applied to problem domains where experimentation is central to uncovering and sustaining differentiated value, and 2) The technology stack supporting the differentiating capabilities is optimised for experimentation.

• Agile methodologies have produced notable successes because forward-thinking organisations applied agile principles to narrow but deep problem domains linked to their growth strategies.

• Agile, like many successful ideas, suffered from semantic diffusion. Agile gets marketed as a universal methodology suitable for all problem types within an organisation. Taking Agile outside its original context and conflating it with portfolio management has diluted its value and may have caused more harm than good.

Organisations must apply agile techniques to advance competitive advantage, but doing so requires understanding the (implicit) assumptions baked into the agile manifesto. In this post, I'll deconstruct these assumptions and make the case that waterfall and agile methodologies are complementary for most organisations.

scale down the problem before scaling up the solution. - Tom Geraghty

Experimentation is central to advancing competitive advantage

Viewing agile as a specialised practice for advancing competitive advantage pays greater dividends than viewing it as a generic, project management methodology.

Agile lends itself to problem domains where:

• The best practice is yet to be discovered,

• Market research highlights valuable unmet customer needs,

• Rapid experimentation enables cost-effective hypothesis validation,

• An incremental approach best uncovers opportunities while balancing risk and delivering differentiated value.

The Cynefin decision-making framework is an excellent tool to guide the applicability of agile.

Cynefin framework groups problems into four categories.

Cynefin Category	Description	Examples	Appropriate Responses
Complex	The domain of emergent practices and creative thinking	SpaceX recognised that it was dealing with a complex problem (as opposed to complicated) and optimised for discovery by running various safe-to-fail experiments to accelerate breakthrough knowledge acquisition.	Adopt agile methodologies.
Complicated	The domain of good practices and expert thinking	Large engineering projects like constructing dams, bridges, and buildings can be classified as complicated. These projects require specialised knowledge and expertise. While there are always multiple good practices to choose from, and one expert might do things differently than another, both would achieve the goals in the end. Fundamentally, the outcomes are still predictable.	Adopt waterfall methodologies.
Clear	The domain of best practices and entrained thinking	Heavily process-oriented domains like support centres can be classified as clear. Call centre executives can resort to clear & well-defined processes to handle problems. There is unanimous agreement about the best course of action, and the response tends to be a process. Adhering to best practices and standard operating procedures delivers effective outcomes.	Adopt standard operating procedures
Chaotic	The domain of rapid response and action bias	The early days of the COVID-19 pandemic and the incubation phases of many start-ups can be classified as chaotic. People are most receptive to novelty and directive leadership in this context, and many innovations have emerged from chaotic situations.	Leverage design thinking and lean start-up techniques.

Applying Agile techniques to solve complex problems while leveraging more conventional methodologies e.g. Waterfall for other problem categories delivers the best results for organisations.

The technology stack supporting the differentiating capabilities is optimised for experimentation

Agile is about breaking down complex problems into a series of experiments, where each experiment is described as a "story". Several experiments may run in parallel requiring small, focused changes in the supporting solutions.

Users receive these changes at relatively regular intervals. User engagement is studied to derive feedback. When feedback is positive, the subsequent experiments double down on the original hypothesis aiming to scale value. The team pivots from their initial hypothesis when the feedback is negative.

Agile assumes the supporting technology is amiable to small, focussed changes and rapid feedback loops.

Minimising the time lag between hypothesis development and validation is critical for agile success.

Agile techniques require solutions to enable speed over time, not just short-term expediency. When architecting solutions to support differentiating business capabilities, "evolvability" is one of the most critical Architecturally Significant Requirement (ASR). Applying architectural tactics like modularity and loose coupling help improve evolvability.

Advanced engineering practices like CI/CD, A/B testing, GitOps, and Chaos engineering also optimise evolvability and reliability, which is crucial to agile success.

Attempting agile with solutions lacking evolvability is a recipe for failure.

Conclusion

Agile techniques are best applied to uncover and scale value optimally while simultaneously driving down associated risks and downsides.

Waterfall methodologies are better suited to pursue more predictable outcomes where driving efficiencies is a critical success measure.

Mary Poppendieck's views on the misapplication of agile

Every large agile framework that I know of is an excuse to avoid the difficult and challenging work of sorting out the organization’s system architecture so that small agile teams can work independently. You do not create smart, innovative teams by adding more process, you create them by breaking dependencies.

What we have learned from the Internet and from the Cloud is very simple – really serious scale can only happen when small teams independently leverage local intelligence and creativity. Companies that think scaled agile processes will help them scale will discover that these processes are not the right path to truly serious scale. - Mary Poppendieck