I had a discussion recently with a colleague about the usefulness of the Last Responsible Moment (LRM) in the context of a complex project. The LRM is when the Cost of Delay overweights the Benefits of Delay. In other words, it's often useful to wait with a decision until a certain point - last responsible moment - before the cost of making a decision too late will be higher than the benefits. I want to come up with an example that may happen quite often in a project:
Lets assume you want to order server hardware that will host your web application. If you order it very early, you may not have enough information to buy the hardware properly sized. Later in the project you may have enough information to make the decision about the hardware sizing. But ordered too late there is a risks that the hardware will not be up and running before the release date and you will loose some expected income from your web application.
We came to the conclusion that the LRM makes practically no sense. The only thing we know is that not all decisions have to be made in the beginning. The nature of complexity is that we cannot plan all details in advance. Only after a project has finished, we know exactly which decisions have been made correctly. Although there exists a LRM, we cannot find it in advance (in complex software projects). By focusing too much to find the LRM in advance, we may actually risk that some business value is not delivered.
Of cause, if we can quantify the Cost of Benefit and Delay then we can find the LRM. It may also help us to know about the LRM while we are still making decisions trusting our guts. But agile coaches should teach the whole story, so people avoid gambling for the LRM.
Ken Schwaber explained the changes to Scrum in a Keynote at the Scrum Day in Germany. One major change is the renaming of the Sprint Commitment to Sprint Forecast. Ken said that some Product Owners were yelling to the team that had just missed to Sprint Commitment. I understand that this yelling is not something you want to have when you introduce Scrum. Scrum and agile methods in general rely on highly motivated teams. Choleric Product Owners wont help you. But is the reason for this behaviour the much more binding term "Commitment"?
If a team fails to meet the commitment, then this may be an indicator for two things:
change in the environment of the team is needed
the problems the team has to solve are to complex.
In both of these two circumstances the team or the organisation should learn and improve. The team can learn how to better cope with the complexity of the given problems and the organisation can learn how to transform to support the teams better to get things to done. IMO: Renaming the Sprint Commitment to Sprint Forecast will take away the urgency of the necessity to learn.
We find commitment in many cultures. The Germans for instance are well known for their punctuality. Everybody commits to be at a certain place on time. It reduces the time to wait and people can start working immediately. Every German who has worked in another country realizes how this behaviour helps to get things done and avoids wasting time. Commitments help us to better work together.
In the organisation we have commitments on various level. Shareholder, investors and owners give their capital and want commitment from the board that the investment will have a return. Customers pay for the products and services and want commitment from the vendor to keep the promises. Employees want a commitment that they get a monthly salary. There are commitments everywhere on the boundaries of the organisation. Inside the organisation we have managers who commit to certain goals, e.g. increase revenue, create innovative products or reduce costs.
Is a commitment by a team to deliver some value after two weeks not possible within a learning organisation?
Companies use nowadays more and more Commercial Off The Shelf (COTS) software products. I have seen many of these projects that launch a big software product within an larger IT organization. My feeling is that it not always helps the organization and that it is usually a very costly endeavour.
I'm looking for explanations, why these projects are causing so much effort. One explanation could be found in the number of dependencies that a software product has to other IT systems. The more interfaces a COTS product has, the more complex is the integration. But why is it more complex than integrating a software written from the scratch? There are two reasons:
We know less about the internals of a COTS product. Since it's not written by the developers themselves, if a problem during integration occurs, it will be more difficult to find the causes of the problem.
The COTS product implements more than required. A product must be adaptable in many organization. It come with features that one organization finds useful, others don't. But it contains all the features. It will also have some kind variability build in - to adapt it to all different situations.
Because of these two reasons, the variety of a COTS integration is higher. If a company that integrates a COTS software has to deal with a larger variety, we must have in the organization with a higher variety, to be able to control it. This is due to Ashby's Law, a very important law in cybernetics. If one system wants to control another system than it must have a higher variety.
So there are two ways of dealing with it:
Decrease the variety of the system that you want to control. We do that for instance in coding software by type checking or information hiding.
Increase the variety of the system that wants to control the other. We do that in software development by self-organized teams. Those teams are able to have a higher variety than a team lead by a command-and-control style manager.
Okay - so what we know is that COTS will have a higher variety - maybe. If a COTS product has less interfaces to other systems/components the integration will not amplify the uncertainty into the project, and thus may still be manageable.
Related to my last post on the need for documents during the design of software, I came across the topic of Design Thinking. Unlike analytical thinking, Design Thinking is aimed at building up ideas. It fits perfectly to agile methods, because it requires iterations and team work. I would say that agile methods include many parts of the theory of Design Thinking.
Here are some statements that I want to make related to Design Thinking:
A team should know the theory of Design Thinking in order to support best the generation of ideas. Agile coaches should explain to team, how ideas and solutions are created and what hurdles will probably in its way.
Our education has formed us to pick the first best idea and carry on with it. That may not be the very best idea to solve a problem. We have learnt that creating bad ideas is unacceptable. Children are very creative and can come up with lots of variations. We learn at school and from our parents what is right and get punished for what is wrong. Therefore we try to create right ideas and hesitate to communicate bad ideas. This blocks the idea generation within ourselves and within a team. If a team can overcome this issue, the creative output of the team will be higher. A team should be aware of this issue and create trustful relationships. This relates to "phychological safety" that is higher developed in a long lived team.
We also have another problem with intuition. If people bring up ideas, the ideas are discussed in a team to validate and merge them with other ideas. If someone has a intuitive idea, it will be difficult for him to explain it. He or she just knows that it's right. But without explaining the idea, the collaborative process to build a common understanding will fail. Either the idea will be lost - and maybe it is the very best solution - or someone else can explain it. This also relates to leadership. Leaders can have intuative ideas and others may simply follow without understanding it. They trust the leader.
Well, I think that a team that is aware of the principels of design thinking can use it to create better solutions. It may also help the team to create trustful relationships quicker.
It's common sense in the agile community that design is emerging during the development. Some extremist state that modelling before coding is not necessary at all. Well, that is not true. But is it necessary to create design documents to capture the models?
Every problem solving for even a little complex problem is done by an evolutionary process. Evolutionary means that variations are created and the best variant is selected. If design is a solution for a problem, then the process to get to a design will look like this:
The developer creates variations of possible solutions in his brain. By playing virtually with it, he finds and selects the best solution.
Then the developer will collaborate with other developers and compare the possible solution. They select again the best solution or may even find a better one. This step creates a common understanding.
The final step is that the solution is put into the code. That really proves it and the common understanding from the second step will help if multiple developers implement the solution. During this the solution may even be amended iteratively by going back to step 1 and 2.
First of all it's important to note, that the inital step is to create a possible solution in the brain of the developer. That requires creativity and experience. Good developers will come up with good solutions. But nobody is perfect and other developer with different experience may come up with other solutions.
In the second step the solutions can be merged together. That works best with synchronous collaboration and a modelling technique. An asynchronous review process (send review comments via email) is inefficient. Also, developers can come up with solutions that wont be selected - they are wasted. It's a nature of the evolutionary process that variants are wasted. The question is, how much effort has be invested into the variant and is it possible to create a variant with less effort.
The third step will transfer the solution - at this point it is only an idea of a solution - into the code. Only with the code the solution becomes real.
In which part of this process do we need design documents? While a developer thinks about the solution, he can write it down. That can be done with a word processor, modelling tool or paper and pen. He is maybe so bright that he actually don't need to write it down. Given that in the second step the solution variant may not be selected, he should invest not much effort. But he needs something to show the solution to others. In the ideal scenario, he has it in his head and draws it on a whiteboard by talking to others. After the second step the developers may want to fix the common understanding, because it may take some days to implement it. You could take a white board photo and store it on the wiki. After the third step the solution is in the code. The common understanding may have changed a little.
The light weight documentation (paper/pen, whiteboard, photo) may not be useful when you have large systems and bigger changes. But that would require a more high level design, aka architecture. An architecture is more long term, may change but not quickly. Also a developer cannot implement the architecture in a few days. It is more a guidance for lower level designs. Architecture and high level design documents can be reasonable. And - documentation for later maintainance is reasonable as well. It depends on the effort to what level starting from the highest level you want to create the documentation.