LEAN - the starter kit

The lean company

A lean company is a company where the foundation is the human respect and its goal is to provide an environment where capable people actively participate in running, designing and improving their work areas and are able to express and use their capabilities at their fullest. The root of lean development practices is the urge for continuous improvement.

It is a hard work to make a company successful, to implement improvements and even more difficult to maintain the commitments over time.
Bear in mind that replicating a process by the book and overlooking the principles that drive it will result in a half functional process where the people involved will act as sheeps.

"If you put fences around people, you get sheep. Give people the room they need."
"Encourage, don't nitpick. Let people run with an idea."
William McKnight

Adopting Lean

Lean thinking requires a change in culture and organizational habits that might be beyond the capability of some companies.

The requirements are:

• The idea must be proven to work operationally

• People who are considering adopting the change must understand why it works

• People doing the work should be committed to the change

Fear of change is subtle. It operates under the radar convincing us that it's there to protect us and keep us safe. In reality, fear of change is one of the most common reasons for resistance to change because it stops you taking any action at all.

Why change an already working system? Why invest time and resources for a new way to work if the current one is ok?

At every moment everything is changing. Moment by moment your business is either growing or shrinking, your customers are either getting happier with you or less happy. This is what happens. Everything changes every moment of every day. Most of the time the change is so small you can't feel it and sometimes big things do happen.
You can't be perfect and you can't win every battle, but you can be better. Adapt to the market to the need to the new ways by changing the way you work the way to approach things and solve them.

You can overcome this resistance to change by communicating effectively. Talk about why change is necessary, and share your vision of change with everyone. These early discussions can help you assess the various barriers to change, and then plan how to manage the people involved.

According to Beckhard and Harris's Change Equation, for people to be motivated to change, they must be dissatisfied with the current situation, and must think that the proposed solution is desirable and practical. Another way to go is simply demonstrate that the change works by making everyone involved understand the mechanics that make the change work and why is needed.

After passing through all the stages (status quo, disruption and exploration) the people involved finally commit to the change. And will get involved in the process with spirit and will change it and model it to overcome new problems. There is nothing you cannot do with a committed team.

Lean principles

1. Eliminate waste: Spend time only on what adds real customer value.
2. Amplify learning: When you have tough problems, increase feedback and sharpen your tools and skills.
3. Decide as late as possible: Keep your options open as long as practically possible for a better result, but no longer.
4. Deliver as fast as possible: Deliver value to customers as soon as they ask for it this will increase feedback.
5. Empower the team: Let the people who add value use their full potential.
6. Build integrity in: Make your system a smooth, cohesive whole both by design and by customer perception.
7. See the whole: Beware of the temptation to optimize parts at the expense of the whole.

Waste

#1 Partially done work often becomes obsolete long before it's ever finished. Until software code is released into production, you have no clue whether or not it solves the business problem at hand. Partially done work will always get in the way of other work. Any work that's not checked-in, integrated, tested, and deployed gets in the way of any other development efforts.

#2 Extra processes will tend to make your overall time to develop longer. Eliminate unneeded processes and lower the process times to achieve smoother information and work flux.

#3 Extra features actually drive an increase in all of the other wastes. Extra features have their own share of costs. According to the Poppendieck's [1], all extra code in our systems must be tracked, compiled, integrated, tested (every time the code is touched!), maintained (for the life of the system!), increases complexity, adds a potential failure point and likely will become obsolete before it's used.

#4 Task switching The cost of task switching in software development has been a recognized as a problem for many years now. Tom DeMarco and Timothy Lister highlighted it in 1987 in their influential book Peopleware: Productive Projects and Teams. Let’s put it simple if you have 4 tasks and each takes 1 week to deliver and you switch though all 4 tasks, by week 1 you will have around 20% (because depending on the complexity of the task some time for realignment/recalibration to the task is needed) of every task, at week’s 5 end you will have all 4 tasks completed. Now you must test all 4. So the tester waited 5 weeks for some piece of functionality that might or might not be correct. Considering the task switching in action, the tester will test and deliver the bugs in 2-3 weeks and so on.

#5 Waiting Unlike the wastes we've looked at previously (partially done work, extra features, relearning, handoffs), there isn't really a process improvement spectrum around delays. Delays either exist, or they don't. Your goal is simply to eliminate them.

#6 Motion is any time that you pass work from one role to another, essentially relinquishing responsibility for it. By doing this you are delaying the solution to the problem at hand.

#7 Defects consider a minor defect - perhaps an overly complex algorithm that causes certain business operations to take 1.5 times longer than necessary for users to complete. That 50% overhead quantifies real hours on the clock multiplied by real hourly wages that are lost to your company because of this defect.

Value Stream Mapping

The Value Stream Mapping method (VSM) is a visualization tool oriented to zeroing-in on problems. It helps to understand and streamline work processes using the tools and techniques of Lean Development. The goal of VSM is to identify, demonstrate and decrease waste in the process. Waste being any activity that does not add value to the final product, it is often used to demonstrate and decrease the amount of 'waste' in a development of a system. VSM can thus serve as a starting point to help management, schedulers, developers and customers recognize waste and identify its causes. As a result, Value Stream Mapping is primarily a communication tool, but is also used as a strategic planning tool, and a change management tool.

A value Stream Map is not something that can be created from the comfort and peace of your office; you have to observe the actual process, not what is written in the company manuals and other documentation. You need to start gathering the actual data that reflects the reality of what goes on in your company.

Go through the process, record your data meticulously as you go, identifying any obvious waste. Record information flows as well as the flow of material. Remember, we are recording actual process data not what is written in the procedure manuals.

Once we have the relevant data about what processes are involved and the delay between them and the actual time spent in a process then we can draw a chart and see where we can remove waste processes and diminish the process time.

If you read books or any literature that attempt to apply VSM on the entire iterative development for the entire product, it is utterly non-sense. But if you see VSM applied on features, stories, user case modules, graphical items, etc. then they make perfect sense.

Mapping out the activities in the development process with cycle times, down times, in-process inventory, material moves and information flow paths helps to visualize the current state of the process activities and guides towards the future desired state.

Amplify learning

Schools of thoughts

There are several schools of thoughts, let’s focus on the “Do it right the first time” and the “Try-it, test-it, fix-it” approach.

1) do it right the first time (DIRTFT)

- small knowledge generation from deliberation and review
- works great for well-structured problems
- small to medium implementation time (depending on the original problem size)

2) try-it, test-it, fix-it cycles (empirical cycles)
- great knowledge generation from empirical processes
- works for ill-structured problems
- long implementation (depending on the original problem size)

Empirical cycles imply the observation and creation of a hypothesis, devising a series of tests to verify the hypothesis and gather results. When the probability of failure is 50% the amount of generated results is at maximum. A correct hypothesis will only generate a small amount of information only proving it right.

"A piece of program logic often needs to be rewritten three or four times before it can be considered an elegant, professional piece of work." Why, he asks, do we object to revising programming logic when we are quite happy to rewrite prose three or four times to achieve a professional result?[2]

To achieve a good result you should balance the DIRTFT and the Empirical cyclesw ith deliberation and review but also with experimentations. Furthermore the scope of this is generating the most knowledge with the lowest cost/resources. Usually combining peer programming, peer review, experimentations and iterations will provide with the best practice for generating best results.

One of the most effective ways to learn is providing small duration 1-2 weeks hardening iterations where code refactoring and knowledge exchange take place

Increased Feedback = Increased quality and learning

Increasing feedback is the single most effective way to deal with troubled software development projects and environments.

• Instead of letting defects accumulate, run tests as soon as the code is written.

• Instead of adding more documentation or detailed planning, try checking out ideas by writing code.

• Instead of gathering more requirements from users, show them an assortment of potential user screens and get their input.

• Instead of studying more carefully which tool to use, bring the top three candidates in-house and test them.

• Instead of trying to figure out how to convert an entire system in a single massive effort, create a Web front end to the legacy system and try the new idea out.

Another thing you should consider is to increase the frequency of the feedback loops in the areas with problems. (see VSM)

Set-based Development

Develop a set of alternative solutions to a problem, see how well they actually work, and then merge the best features of the solutions or choose one of the alternatives. It might seem wasteful to develop multiple solutions to the same problem, but set-based development can lead to better solutions faster.

Set-based development does not replace iterative development—it adds a new dimension. During early iterations, multiple choices are developed for key features; in later iterations, they are merged or narrowed to a single choice.

As the group grapples with the problem, resist the temptation to jump to a solution; keep the constraints of the problem visible so that the team can discover the intersection of the design space that will work for all concerned.

Decide as late as possible

Broad and Depth approach

There are two types of approaches or practices for solving design problems, the broad approach can be viewed as a pipe that is broad at start but as it goes down (as time develops) is narrows down to a point; the depth approach can also be viewed as a straight pipe that goes down with tiny to no variations. The designers that are at a starting point tend to take the depth approach which might reduce the complexity of the problem and fix the path that the project will follow. The real problem is that the horizon is narrowed down from the very start and the leaves no room for options therefor making a future change in vision costly.

The depth approach is only feasible if there is a correct point to focus on. For that point to be correct we need a few things: a domain expert that can select the point with wisdom and for that point not to shift from the starting position.

If we cannot estimate nor grasp the correct point and also lack a domain expert then the broad solution is to be considered for best results.

The broad approach does exactly what is supposed to do: postpone the moment you fix a path for a portion of the project. Making a decision as late as possible entitles you to a more fair evaluation of the problem and might lead to a correct decision. I said might because if you fail to decide by the deadline on that feature you will only delay the project and furthermore the decision might not be up to you anymore as the project will tend to pick a path by its own and that path might not be the best one.

Both broad and depth approaches benefits from a domain expert. Having a domain expert can improve the decisions made on a project and make the approaches less rigid for the depth and less wide for the broad approach.

Deliver as Fast as Possible

Delivering the product as fast as possible leaves the customer to make decisions as late as possible, leading to greater business flexibility.

The pull

No planed and prescheduled assignments regarding a complex solution can be effective. Depending on an automated system to generate the schedule and depending on PMs to assign the work to the developers is not the best way to handle complex and changing solutions.

A lean system is defined by a pull system. The pull is simply a way that can lead to an increased production rate. The pull system empowers the sw developer to make his own decisions and pull work (work that is known to give back the best ROI) when he can (has the resources to accomplish that goal) from the queue. This approach leads to a more efficient way to work and handle changing solutions.

The developers estimate how much time the piece of work will take to accomplish and the direct customer or PM/ analyst gives the priority on that piece of work. The pieces of work now become kanban cards on a board that provide transparency to both the PM as of what work is done, progress and where is a blockage and also for the developer which knows what work needs to be worked on next.

The pull system is feed by a queue. For the pull system to work efficiently the cycle times must be as low as possible. The cycle time is the time from when the task was identified to the moment it is complete. The only reason you have queues is that the resources necessary to achieve the work goal are limited. The time a kanban spends in a queue is time wasted.

Provide a smooth steady pace for the system. This can be achieved by monitoring the queue more exactly monitoring the feeder and the processor for that queue. One way to improve the feeder is to break down the work in smaller parts. Working with smaller parts instead of batches will reduce queue time. The bigger the batch the bigger the queue will be for the next kanban. The smaller the work the quicker it gets through the system as the whole evaluation of the batch is not done but the evaluation of a small portion is and therefor is fit for processing.

Once we have dealt with the queue we can deal with the processing time. Processing small amounts of work leads to lower complexity and a lower number of things to go wrong.

Empower the team

In a lean company the developers have the authority to design new ways to do the work and also they have the decision-making responsibility; they are the energy core of the company. They are the point of focus for resources time and training.

Research has shown that intrinsic motivation requires a feeling of belonging, a feeling of safety, a sense of competence, and sense of progress.[3]

Thomas lists the building blocks of intrinsic motivation: choice (self-determination), meaningfulness (purpose), competence, and progress (p. 49). In Maslow, Frager, and Fadiman, Motivation and Personality, Abraham Maslow presents his classic hierarchy of human needs: physiological, safety, belonging, esteem, and self-actualization.

Here we introduce the concept of the Master developer. A Master developer is a developer that has acquired the skill needed to understand the customer. The Master developer plays a key role in a project/team because it knows the customers requirements of the customer and what kind of tradeoffs the programmers have to and can make to achieve the requirements. If the flow of information to the developer team is obstructed then the product is surely to have a low or even no perceived integrity.

Build integrity in

Perceived integrity means that the totality of the product achieves a balance of function, usability, reliability, and economy that delights customers.[4]

The perceived integrity is the integrity that the customer can grasp it is the perception on the end product through the eyes of the customer. The grater the integrity is the grater the customer satisfaction. Let’s us not consider that a great number of features is leading to a greater integrity. Quite often it is the opposite, by spanning the features the product seems larger and it might be perceived as more complex and also it might be prone to more bugs and more integrity failures.

The greatest perceived integrity is the one that is created from a simple product that will satisfy the customer in the exact way that leads the customer to a perception that the product it tailored for him.

Conceptual integrity [5] means that the system's central concepts work together as a smooth, cohesive whole. This is achieved by making a system flexible, maintainable, efficient, and responsive.

Clark and Fujimoto's research showed that conceptual integrity is a reflection of the integrity of the upstream and downstream technical information flow in the product development process.[6]

The conceptual integrity is the base of the perceived one. If the original design and implementation does not manifest integrity there is little chance that the product will work as a smooth whole.

Although the conceptual integrity is the base of the perceived one that does not mean it guarantees it.

Refactoring

Lean sw development is based on aggressive code refactoring. Refactoring is the key element in a lean development cycle. Without refactoring and adapting the new code to the new improvements achieved by the system, gaps are created in the conceptual integrity which will impact the customer in the end.

The need of refactoring is most visible as the system evolves and matures. New features are added every cycle but usually the features interact with each other. All beginners should resist the blunt approach and consider the architectural capability of the system. If you let blunt approaches pile up in the system then the conceptual integrity will start to degrade which can lead to a complex system that is hard to manage and maintain leading to more resources wasted and a possible poor perceived integrity.

Refactoring on a time base is what keeps the integrity of the system and makes it work as a smooth cohesive whole.

One of the myths about refactoring is that it is actually rework and is a waste. While this can be viewed as a waste in the eyes of the beginner, refactoring is actually the way of improving the design as the design evolves. Also as the system progresses and it lacks refactoring, the development process will start to take longer and longer as the code and system structure becomes more complex and ambiguous.

See the whole

As the product must be seen as a whole so should the company. Usually as the problems tend to get worst managers will often apply the same policies and practices more aggressively and that will only lead into causing more problems and worsening the current ones.

If a solution seems to solve the current problem it does not necessarily mean that it is the correct one. There is a thing called limits to growth [7].

Even if a process or practice produces a desired result, it creates a secondary effect that balances and eventually slows down the success. If you continue use the same process for increased success, you will amplify the secondary effect and start a downward spiral. Instead creating forced growth, find and remove the limits to growth.

Senge says, “Don’t push growth; remove the factors limiting growth.” Criticize less; find a solution, not a culprit. Identify the reinforcing spirals—both upward and downward spirals, then seek to understand and use the "structure” and its patterns of behavior.

“Beware the symptomatic solution. Solutions that address only the symptom of a problem, not the fundamental causes, tend to have short-term benefits at best,” Senge writes. Continuation of a symptomatic approach can also cause even our capability for “fundamental solutions” to atrophy.

High quality parts of a product do not make the product of high quality. The ability for the product to satisfy the needs of the customer depends on how well the parts interact with each other.

Local measurements of an overall performance will lead to a good impression of the product but a poor overall performance which is hidden by the local measurement. If you can't measure everything that is important, partial measurements will lead to sub-optimized measurements.

If you can't measure everything that is important and necessary to optimize the overall business goal, then you are better off without the sub-optimizing partial measurements. If you still use sub-optimized measurements, you are encouraging sub-optimized behavior which will lead to low productivity.

Bibliography

[1] Poppendieck, Mary and Tom. Implementing Lean Software Development: From Concept to Cash. Addison-Wesley, 2006.

[2] Yourdon, Classics in Software Engineering, 151.

[3] Thomas, Intrinsic Motivation at Work

[4] The definition of perceived and conceptual integrity is adapted from Clark and Fujimoto, Product Development Performance, 30.

[5] A term found in Brooks, Mythical Man Month, 42.

[6] Ibid., 30.

[7] Senge, The Fifth Discipline, 95.

Mike Cohn, Agile Game Development With Scrum

Mary Poppendieck, Tom Poppendieck, Lean Software Development: An Agile Toolkit

Curt Hibbs, Steve Jewett, and Mike Sullivan, The Art of Lean Software Development