Agile ≠ Lean
How many times have you heard the question: “What are the pros and cons of Agile?” Unfortunately, that question is put in the wrong way.
Let’s start on the highway.
Hey, we have built a new superfast highway. Admittedly, it is only single-lane. But in every other aspect it is premium quality. The first car to show up on the road has a maximum speed of 70 km/h. It’s no problem here to drive at that speed. The travel time is predictable and minimal, given the capacity of the car and the road.
Then, one more car appears with the same maximum speed. Since both cars travel at the same speed, they don’t interfere with each other. Problems arise when the third car turns up. It has a top speed of 110 km/h, and the two slower cars become blockers. Resources can’t be used to their full capacity. It’s even worse when a really slow car with a maximum speed of 30 km/h enters. The speed and the length of the queue fluctuate. And when even more cars enter with varying capacities, the speed even falls below the speed of the slowest car on the highway. Sir John Kingman explained this phenomenon in the 1960s.
“In heavy traffic the equilibrium waiting time will be large”
Sir John Kingman (1939–)
In 1962, Sir John Kingman wrote that in his famous paper on queues in heavy traffic. And he actually constructed a formula describing the average waiting time in a highway queue.
The problem is the variations and they come in many flavors.
There are three kinds of variations that always exists in enterprises:
- The number of orders varies with time.
- The capacity of our resources varies with time.
- Every order is special. Everyone wants to make their own customized order.
When the cars arrive at the highway with varying frequency, the queuing quickly gets frustrating. And it will be even worse when the acceleration capacity varies among the cars and when the drivers are headed to different destinations. It appears as the more we use the resources, the less value we get. Let’s visualize this formula in a diagram.
On the horizontal axis, we see how much we use our resources. I apologize for sometimes referring to humans as resources in this essay. It’s only done when describing a view of the world where humans are said to be interchangeable. What I mean in those cases by “resources” is our computers, our employees, our offices — all the stuff and people we have that help us create value.
The vertical axis shows how long we have to wait or, in other words, how fast the refinement into value flows through our system. The higher the waiting time, the worse the flow.
Let’s start with the dashed curve that follows the x-axis. It shows that without variation, we don’t need to worry about resources being used too heavily. The more we use the resources, the more value we create. One hundred percent usage gives us twice as much value as 50%. The cars can run at maximum speed when there is no variation.
Then comes the solid curve where we have moderate variation. If we don’t use more than 75% of the resources, the flow is quite good. But when we push the machines beyond that, the whole flow halts. The cars are forced to slow down and are trapped in slow queues.
The third curve shows that in the case of high variation, we only get something done if we use no more than a fraction of our resource capacity; otherwise, the velocity on the highway hits zero.
Flow and resource utilization are two very different approaches that obviously need to be balanced against each other. We will compare these two with a third approach: agility. Before I define agility, let me explain what it is not.
Three approaches
Resource, Flow, and Agility. Here are my three definitions:
- Resource utilization: When we use all our resources to the maximum for what they are best at.
- Flow: We minimize time from idea to cash without decreasing quality and despite variation within limits.
- Agility: Ability to adapt to unexpected changes and wisely influencing the new situation to our advantage.
Let’s compare the three approaches:
- Resource utilization — as many cars on the road as possible moving at their top speeds.
- Flow — the cars that actually are on the road, reaching their destinations as quickly as possible.
- Agility — ability to adapt if something unforeseen happens, such as a rock rolling out onto the road.
No enterprise can do with just one of these three. We need to find the balance
and, above all, understand in what contexts one approach is better than the others. For the rest of this essay, these three approaches will be referred to as
Resource, Flow, and Agility.
Best context — where does each approach fit the purpose?
Three approaches, each suitable for a special context. When we know exactly what needs to be done, it’s time to optimize the usage of our resources. We’re exploiting a failsafe bonanza. We know what is required of us. We have done this many times before. No devastating surprises await us. Let’s just focus on our little puzzle piece.
Flow optimization is the best match, however, when we know what to do but have variation within limits.
- The number of orders varies.
- The capacity of our resources varies.
- Every order is customized.
We must apply systems thinking. The whole is the sum of the parts.
Finally, we need agility when things we can’t even imagine might happen. We know why we exist, what our purpose is. We also know that our environment will evolve. But upfront, we have no clue how. All assumptions about ourselves and our environment may need to be revised. Disruptive events may put our existence at risk. We must have complexity thinking. The whole is more than the sum of the parts. Do you remember how Lewis Carroll reasoned back in the 19th century?
“Would you tell me, please, which way I ought to go from here?”
“That depends a good deal on where you want to get to,” said the Cat.Lewis Carroll (1832–1898)
Lewis Carroll wrote this in his famous 1865 novel. In an enterprise, we make decisions all the time.
Decisions — when and based on what?
When should we make decisions? And on what basis? With a resource-utilization approach, we make decisions as early as possible to get clarity. We base decisions on calculations about the future, and we follow industry standards.
If we want to maximize flow instead, we might defer crucial decisions until they start to influence other decisions. That makes knowledge about variation, for example, an input to our decision making. Historical observations form the basis of our decisions.
To increase our agility, we avoid speculative decisions. We make decisions on the go at the last responsible moment. When new knowledge emerges, we don’t mind changing the decisions we already made. Our decisions are based on what is happening right now. Assumptions that history repeats itself should be challenged.
Another interesting comparison is how differently we handle dependencies with these three approaches.
Autonomy — how dependent are we on others?
Autonomy is when a team or department have discretion granted in their work. If utilizing resources as much as possible is the overall goal, then compliance with industry standards makes resources more interchangeable. Local deviations increase costs. To achieve synergies, departments in the organization are controlled and measured from the level above them. Alignment is the guiding star. Each group has one person recognized as a hero if the group meets the goal described in their plan. This person is also held accountable, in case the task assigned to the group is not done as expected. To make handovers between teams efficient, we put much effort into maintaining and managing the dependencies between teams.
If we want to optimize the flow instead, we must not forget that the cycle time of a subprocess is only one part of the entire throughput time for a system. Optimizations should improve the whole flow, from idea to value. Cross-functional teams can take on broader problems and thus reduce waiting time and knowledge loss in handovers. Support teams with more general proficiency are ready to help with anything at short notice. We call this the Andon system.
The third approach is agility. That’s where the entire organization works in the same direction, but each department decides how it uses its resources. Since the departments don’t have individual goals, they don’t compete with each other. If the people in one department create more value by helping people in another department, then they should do that. Lines on the organizational chart represent habitat, not accountability. This enhances the ability to adapt, even when something unexpected happens. We prefer generic rather than specialized resources. This is true for people as well as machines and financing.
The three approaches don’t only differ when it comes to interdepartmental work. There is also a difference in how groups collaborate internally. Why is diversity and independence important?
“The best collective decisions are the product of disagreement and contest, not consensus or compromise.”
James Surowiecki (1967–)
James Surowiecki gave many examples of that in his fascinating book titled
The Wisdom of Crowds and with the subtitle Why the Many Are Smarter Than the Few and How Collective Wisdom Shapes Business, Economies, Societies, and Nations.
Diversity — heterogeneity or homogeneity?
Do we want heterogeneous or homogeneous teams and departments? It depends on what problem we are facing. When we repeat exactly the same tasks over and over again, we want to optimize resource utilization. Only through specialization do all resources achieve maximum performance for their tasks. The sum is maximized when the parts perform at their maximum.
Departments consisting of one type of specialists create clarity. If, on the other hand, the work varies a lot but within limits, we should keep an eye on flow instead. Handover waste is reduced by cross-functional teams who can take on broader problems. As I mentioned before, we also want generalist support teams, who can increase the organization’s capacity given large variations. Long experiences with the same process help employees recognize uncommon situations.
Agility is supported by people with T-shaped knowledge and groups with diversity. Individuals may have deep knowledge about something. But above all, make sure they also have general knowledge about many things. Curiosity about stuff you currently aren’t working on is king. Pair up with an expert and do tasks you’re not familiar with. Actively strive for cognitive, cultural, field, and other types of diversity to become more generic as a group.
Overall, these practices improve our ability to adapt to new conditions.
So, diversity sometimes improves the quality of decisions.
Quality — how good?
What quality should we aim for in our work? And how can we achieve it? In recurring tasks without variation, we achieve the highest quality automatically when all resources are utilized to the maximum for what they are best at.
When it comes to flow optimization, I already said in my definition that we should do as little as possible to achieve excellence. Quality trade-offs are thus never acceptable. But we do want to eliminate all waste that doesn’t contribute to maintaining our level of quality.
To optimize adaptability, we do just enough to achieve sufficient quality. Gold plates are speculative. Solving many problems is preferred to achieving excellence. Can we improve quality by following standards, and if so, what standards?
“The nice thing about standards is that you have so many to choose from.”
“Furthermore, if you do not like any of them, you can just wait for next year’s model.”Andrew S. Tanenbaum (1944–)
Andy Tanenbaum wrote that in his seminal book on computer networks. We can either stick to external standards or develop our own. As a matter of fact, we may even skip standards altogether.
Standards — how we do it here?
For well-known problems with no variation, it’s a good idea to follow established industry standards, even with core business. Inventing your own rules is a waste when someone else has already solved this problem. However, our organization might differ from others.
When we want to optimize our own flow, it is important to identify and standardize what turns out to work well for us. We must standardize and stabilize before we begin to improve a process. Before the process is stable, we can’t measure the wholeness objectively. Too many parameters vary.
Standards per se do not enhance our adaptability. Instead, every employee and every department is empowered to constantly inspect and adapt. Co-evolving with the environment creates improvements. We create value based on what we see now, not what has historically proven to work. Rather than rules, we stick to heuristics.
An organization might not only be concerned about how, but also why. When is it important to know why we exist at all?
Purpose — why do we exist?
It’s easy to go in the wrong direction if we solely focus on tiny everyday-tasks. In an organization where resources are used to the maximum, everyone knows what their own job is. Each department receives unique instructions
describing their expected output. These instructions are broken down into sub-instructions assigned to individual employees. This may force several projects to need the same limited resource. Management on a higher level then decides where to allocate this attractive resource.
In the flow-optimization organization, however, it’s more important that everyone understands the benefit for the customer. We minimize the number of simultaneous purposes. By definition, we want the time from idea to customer value to be as short as possible without lowering quality. Little’s law shows that the waiting time increases when we multitask.
To be able to adapt to sudden changes we couldn’t even imagine upfront, optimizing flow or resource utilization is not enough. When we have a shared, coherent story about why we exist, we don’t need to change that story when conditions change. Instead, we focus on how to best proceed in the same direction given the new circumstances and try to exapt our tools and other resources. Exaptation is when something that serves one function
comes to serve another. Think of bird feathers, which were first used for temperature regulation but were later adapted for flight.
These are three different ways of looking at organizational purpose. Planning around them is vital.
“The work of every workman is fully planned out by the management . . .
not only what is to be done
but how it is to be done
and the exact time allowed for doing it.”Frederick Winslow Taylor (1856–1915)
Frederick Taylor was very clear when he wrote the book Principles of Scientific Management, in 1911. Grammatical tense significantly influences how we plan. Are we planning based on historical measurements? Are we planning based on what we observe right now? Or are we planning based on how we have decided what the future will be?
Planning — what, how frequent, and for whom?
Planning is thinking before the action takes place. To maximize resource usage, we only make plans once. A few key people plan a big, consistent solution that answers questions about delivery time, delivery content, and cost. The plan must be failsafe. Risks are analyzed before planning is even started.
Everyone in the organization must be informed about the plan at the same time. It doesn’t really matter if they understand the whole; it’s more important that everyone knows exactly what they should deliver and who they report to. The plan should show how each resource is used to 100%. It’s a failure if a resource is ever unallocated. The consequence of this orientation is that we might work toward many goals simultaneously.
With flow optimization, decisions are deferred until they are needed — not too soon and not too late. We figure out what we need to plan right now
and leave other options open. We consider variation in…
- order frequency
- types of orders,
- and resource capacity.
Our assumptions are based on collected data. Planning for multitasking is bad, even if it means that resources may not be used to 100%. We trust Little’s law.
To increase our ability to adjust to unpredictable change, we embrace replanning when something happens that affects our plan. We observe what we are doing right now and, from there, plan what we should do more of
and what we should do less of. We can also decompose what used to be successful and then combine in new ways.
Instead of analyzing every risk in advance, we probe by planning many small experiments that can fail without causing too much damage. They are safe-to-fail rather than failsafe, opposed to how it is in resource-optimization planning. The hypotheses might contradict each other, and we run the experiments in parallel.
Since the departments are empowered and know our organization’s overarching purpose, they plan independently and at their own pace. We want broad perspectives when planning, so we bring together customers, users, designers, developers, and many other types of experts. We optimize to be able to change quickly, whatever happens.
None of these three approaches demands more or less planning than the others, by definition. Another thing we plan is how refined value travels through the organization.
“Why is it every time I ask for a pair of hands, they come with a brain attached?”
Henry Ford (1863–1947)
It is unclear if Henry Ford really said this. But he truly cherished the idea of following the plan, no questions asked.
Workflow — flow of the value?
The sequence of processes through which a piece of work passes from initiation to completion is very different in our three approaches. When we know exactly what to do, we have no variation, and we cannot possibly be surprised by new circumstances, then practicing push is king. The work starts when we have decided the who, the when, and the expected output — neither before nor after. The departments are specialized in their parts of the workflow. They receive raw material from the previous one and pass on their refinement to the next. In the end, the final result is shipped in a delivery!
When order frequency, resource capacity, or individual order customization varies, then push halts the system. As we saw earlier, Sir John Kingman explained this in the 1960s. The flow must be controlled by pull. Those with needs pull. No work starts before there is a need. The customer is thus an important part of the workflow. Using the metaphor of streams helps us take advantage of mathematics from queue theory, when we design our process flow.
In both resource optimization and flow optimization, we have an idea of the connection between cause and effect. One difference between them is that, with flow optimization, we struggle with variations that fluctuate in ways we can’t predict. But what does the workflow look like when the causality relationships can only be understood in hindsight? We have evidence of several contradictory theories, and we don’t have time to prove which of them is correct. In that case, the value is better refined in a network with a roadmap that cannot be designed upfront. The stream metaphor is therefore misleading here. The better the network you have, the more diversity you get among decision makers.
When we ignore evolution in our environment, we easily end up with false statements. Listen to the following which might sound reasonable.
“Insanity is doing the same thing over and over and expecting different results.”
This quote could be found in a 1981 publication from Narcotics Anonymous. It is often attributed to Albert Einstein. Not only is that attribution wrong, but when our environment evolves, the statement itself is wrong. Doing the same thing twice might give us two different outcomes.
So, what should we focus on?
Focus — where to put our attention?
Cutting off context from what we work on is sometimes suggested, but not always. When our overall goal is 100% resource utilization, then each department should focus on doing the task they are assigned, nothing else. And that task should be done as efficiently as possible. Someone else takes responsibility for the whole.
If we want the refinement journey from idea to value to be as fast as possible without quality trade-offs, then the whole flow is important, not just each subprocess. The whole is the sum of the parts. We need systems thinking.
But if we have open, intangible systems, then the whole becomes more than the sum of the parts. At each level, our system is affected by what happens in other systems. What seemed obvious yesterday is not true today. John Boyd’s OODA loop helps us here: We must constantly observe, orient, decide, and do. We want to do this loop as fast as possible in order not to base our forecasts on yesterday’s weather. The difference is between focusing on my activities, focusing on our customer’s value, or focusing on our environment’s evolution.
Focus is one thing, but what does control and follow-up look like with these three approaches?
Control and follow up — are we doing ok?
To optimize resource usage, a carefully selected group of people think, decide, and measure. The rest perform according to instructions. It is a hierarchical chain of command that clarifies who does what, when, and how. Goals are broken down into increasingly small components And are assigned to increasingly small fractions of the organization. Data are collected on compliance with our plan and compliance with industry standards. When we don’t get what we predicted, we consider replacing the resources that couldn’t meet the goals we planned.
In flow optimization, the whole is more important than the parts. We continuously ask ourselves if our customers get excellent quality in the shortest possible time. We control and measure processes, not people. Internal standards are developed from our own experiences. Employees at all levels are encouraged to frankly and continuously challenge these standards.
To be able to adapt when disruptive change comes, every part of the organization must continuously observe its environment. The entire organization is as transparent as possible. Curiosity is a word of honor,
especially when it comes to corner cases. This means that information can be retrieved by people through various paths instead of pushed from a single central point. We still deliberately impose universal constraints, however. The difference is that these constraints are unifying and guiding, not controlling. For example, they can indicate a direction rather than describe a wanted state.
The values of the organization are also of great importance. Rigid governance in uncertain contexts inevitably becomes bureaucracy, which fosters informal processes — sometimes even corruption — to bypass the bureaucracy. When we consciously try to fix this, informal processes become invisible obstacles and create even more uncertainty. A dysfunctional culture grows. Tougher rules in complex systems increase complexity, not decrease it. The three approaches are of great importance for control and follow-up. Having more than one alternative is also essential, however.
“Plan B. You’ve always got to have a Plan B.”
Sylvester Stallone (1946–)
That’s what Sylvester Stallone said in the movie Escape Plan from 2013. Having a plan B may mean working in parallel with two tracks, but it can also mean deferring decisions that terminate options.
Options — plan B?
An option is a thing that is or may be chosen. To maximize resource usage,
we never spend effort keeping multiple options open. We don’t even start until the best option is identified. Then we stick to it. Every initiative must be failsafe.
An interesting method in flow optimization is set-based development. Knowledge about what delights customers is lowest when we start. Rather than choosing a solution early, we focus on what constraints we must set to continue development. Based on these, we develop multiple alternative prototypes in parallel; these don’t need to be market quality, but they give us and our customers a sense of what we like and dislike. With help from customers, we progress by adding constraints upon new insights. Alternatives can thus gradually be skipped or merged with other alternatives.
To maximize the opportunity to change quickly, we don’t want to speculatively invest in prototypes or big parallel initiatives. Instead, we focus on delivering a small, viable solution that is easy to change. “Viable” means that it solves the problem, even if not in the best way. We create value as soon as possible and are not afraid of reworking when we acquire new knowledge. While we deliver, we continuously explore our environment. Sometimes we find evidence of contradictory theories. We test them in small, safe-to-fail experiments. What we discover affects our plans and deliveries.
We can’t do everything ourselves, of course. All organizations have providers.
Providers — how to collaborate with them?
Sometimes we need external support. The approaches differ here as well.
Resource-optimizing businesses select providers with open, maximally fair, comprehensive tendering procedures. We know what we want, so we can specify parameters that make a difference. By multiplying these parameters with proper coefficients of importance, we can calculate which provider is most cost-effective.
It is important that provider contracts contain all possible scenarios and the consequences they will eventually have. Specialized internal departments are alternatives to external providers, of course; the necessity of these detailed contracts also applies to them. For this reason,
we sometimes get win-lose games between two departments in an organization. The important question becomes: which party did not honor the contract?
Maintaining both good flow and excellent quality requires long-term win-win partnerships. We never, ever outsource core business. Instead, we insist on long-term growing experience.
Our adaptability benefits from cooperating as long as it’s win-win. We want small deliveries from our providers, and each delivery should add value. This way, we make sure we have value early. Agility requires that we travel light. When we notice we do something ourselves that can be purchased as a commodity, we stop doing it and get a provider.
The Toyota Production System is known, among other things, for Kaizen, a method that never stops refining our ways of working.
“And we are reducing that time line by removing the non-value-added wastes.”
Taiichi Ohno (1912–1990)
Taiichi Ohno is considered to be the father of the Toyota Production System, which inspired Lean Manufacturing in the U.S.
How does process improvement work with these three approaches?
Process improvement — how can we do better?
If everything is crystal clear and no evil changes or variations are hiding around the corner, then process improvement is mainly done by finding more cost-effective resources and implementing new industry standards. We must always ensure that everyone knows what to do. Another strategy is to copy ways of working from our most successful market competitors.
To minimize waiting times while still maintaining excellent quality, however, we ask ourselves other questions. Can we eliminate waste and inconsistency in our value stream? Can we automate but still retain intelligence? Can our process be more robust, coping with large variations within given limits? Does everyone in the organization understand the whole, the customer value? We improve with a combination of deductive and inductive logic, such as the PDCA wheel — plan, do, check act.
Agility rises when tools become more generic. To understand what a generic tool is, compare pen and paper to software. Software is great for quick processing, mass distribution, and long-term big data storage. But it is also constrained by the fact that the features were implemented in the past. Pen and paper can be used to solve an unlimited number of problems. Agility also rises when we reduce dependency on specific resources. The questions here are different. Is new information transferred smoothly in all parts of the organization? Are we traveling light? Do we have broad knowledge as well as empowered individuals and departments on all levels? Can we be better at exploring our environment?
Bringing it all together — we need a mix of all three
We could go on and on. For example, how is recruitment managed in these three approaches? Procurement? Training? Modern organizations need a combination of resource-utilization optimization, flow optimization, and agility.
Without resource optimization, we don’t fully exploit what we know is profitable. Without flow optimization, we can’t manage variation within given limits. And without agility, we soon get stuck in outdated assumptions.
For example, think about sprints in scrum.
They consist partly of methods for optimizing resource usage. Iterative work entails a recurring process. According to the scrum guide, the sprint should contain a number of roles, artifacts, and events, and it has a predetermined end point in time. But the sprint is also flow-optimized. For example, it is implemented by a cross-functional team with shared responsibility. At the end of each iteration, they review and discuss improvements to their process. The sprint is also designed to create agility. More important than all our tasks is the sprint goal. Each day, the team meets and plans how to best adapt to what they learned since yesterday to achieve the sprint goal.
Let me emphasize one important thing before I close this essay. You may have noticed that agility is not about change or change management. Planned change is actually resource utilization, sometimes in a good way. However, agility is about creating an ability. It’s an ability to adapt when our context changes disruptively in ways we did not expect upfront.
Three approaches
Let’s revisit the definitions one more time.
- Resource utilization: Using all resources to the maximum for what they are best at.
- Flow: Minimizing time from idea to cash without decreasing quality and despite variation.
- Agility: Ability to adapt to unexpected changes and wisely influencing the new situation to our advantage.
No enterprise can do with just one of these three. We need to find the balance
and, above all, understand in what contexts one approach is better than the others.
And that brings us to the end of this essay.
Thanks for reading.
