Monday, November 28, 2016

On discipline, agile, lean and kitchens

Discipline in planning and execution is not a culture war.

I used to see method and process as core execution discliplines, as background activities that one just took for granted on any project. For the past few years, I have witnessed an increasing swell of support for making them a matter of style and team culture.

I would place the shift starting a couple of years after the Lean Startup book came out and inadvertently normalized the notion of outcomes over processes even outside the startup world.

In fairness, that is not what the book and underlying concepts propose, but there were large swaths of the software development field who were ready for a well-articulated message that advocated for less formalism, less planning, less checking, less verification, get the idea.

By Mark Morgan
The ensuing phylosophical battles can be distilled to basic questions such as "how much time should one spend writing down the processes to be followed by a team?" or "How much time should one spend planning activities and estimating their costs before deciding on a course of action?"

The "it-is-boring" camp argues for less discipline in favor of faster execution, which allows for more iterations towards success.  For this camp, the air is sucked out of the room the minute someone asks for  agreement on processes and plans.

The "it-is-boorish" camp argues that unmanaged chaos can seize up execution progress and force part of the team into silently picking up the slack.

I can see a strong correlation between individual style and the choice of camps, which is why it is so easy to dismiss the entire unpleasantness of the debate as a matter of personal style or culture war. 

About "Let us choose"...

Sprinkle some conflict aversion, superficial analysis and we soon find the discussion abandoned behind the wall of false compromises: "let each team choose what works best for them".

I find that reasoning particularly disingenuous in that it implies that the only alternative is an arbirtrary decision forcing freedom fighters into submission. Given time and the right audience, you may even see an Austrian economist or two being quoted in the discussion.

To be clear, teams should absolutely decide what works best for them, as long as the selection of "best" is made against a (preferably long) background of good and bad experiences.

And it goes without saying that the discussion should stay away from the extremes of Go-Horse programming*, with virtually no time assigned for planning, and of the waterfall model where nothing meaningful ever hits the market (and no one has tried pure watefall for at least two decades nor would in their right minds argue for its return) .

This is the point where I confess to lean towards the "it-is-boorish" camp and my reasons are simple:

When no discipline is actually a lot of it

Hidden in the anedoctes about improved results due to less planning and less process, there are invariably teams with extensive practice with planning and processes. Behind each IPO-wonder, you will find leaders with well-lived experiences ahead of other similar initiatives (if you absolutely must bring up Facebook, that is a different animal, leave a comment and I will respond to it) .

Teams do not succeed because they have less discipline, they succeed because they have people who know enough about discipline and processes to hand pick the correct approach for the circumstance at hand. Moving from process to actual results, the parallel is that behind each story of frequent and short iterations leading to a winning design you will find people who have produced winning designs in the past and who have had the benefit of internalizing what worked, what failed, and why.

Many victories seemingly stemming from agility are the consequence of solid experiences and discipline unhindered by minutia. And yet, many of these victories may be short-lived if the  technical debt incurred while executing with less discipline is not managed properly.

Tragedy of the cooks

The analogy here derives from the Tragedy of the Commons, with "order" being the shared resource. In a completely unregulated environment, either by intention or natural pressure, the participants tend to exhaust the shared resource for two reasons: (1) the assumption that the resource is infinite and (2) the expectation that other parties will consume or hoard the resources faster than everyone else competing for it.

Absent some notion of externally mandated order, you end up with an ecosystem where the participants are lifted to the same level of access to the resources and neglect tending to the shared resources along a spectrum of obliviousness and forced sociopathy.

The obliviousness comes in the form of people internalizing an experience where lack of discipline simply works, unaware of other efforts happening in parallel to restore the original order to the system. You know the drill: that coworker who was asked to maintain, improve and share a few guidelines here and there to ensure some bad customer situation was avoided in the future. Then came the point where people realized that the "few guidelines" were several pages long and everyone needed to be mandated to read and follow the guidelines because bad customer situations kept on happening. Poof! Fun is over and the productivity-sapping scapegoat is standing by to take the fall.

An even more insidious side-effect is the internalization of these tragedy-of-the-commons experiences at an earlier stage of someone's the career, where they become the lenses through which the beginners will see work relationships.

By Roberian Borges
On the sociopathy extreme of the spectrum, my analogy is simple: If your team is asked to prepare a four-course meal and there are no rules about who should clean up the kitchen, there is always someone who will be bothered about the mess before the others, and that person is usually someone who has dealt with dried up batter on the counter before (give it a try) . The boorish cycle is completed when the other cooks pat the cleaner on the back, proclaim he has a natural vocation for cleaning, and mentally excuse themselves from the task from that point on.

Sometimes your most productive cooks are simply the ones who can ignore a dirty sink the longest.

The age of WIT (Winging IT)

Though I currently lean towards the "It-is-boorish" camp, I can see the allure of reduced planning and have the feeling that some of the discomfort experienced at the hands of the "it-is-boring" camp are growing pains in a generational shift. We just have to find ways to make it work at the scale it must work, then deal with the new state of things.

In this age of freemium web applications, where everyone expects everything to be free, the lines between outright market dumping and viable business models are becoming blurrier by the day - Uber and its driver's incentive program come to mind. This sort of expectation has become so ingrained in society that large swaths of the workforce simply accept the notion that products should be created under the successful (?) umbrella of freedom.

Where that camp loses me is in the expectation that (1) wonder startup efforts can be created out of thin air without something as basic as market research and (2) established organizations can be morphed into startups. The Lean Startup crowd is onto something that is very specific for the high-failure rate model expected of actual startups developing as-a-Service offerings, but that is a topic for a different posting.

At some point, when you realize most people don't like cleaning the kitchen, chastising them into doing the chore may just drag down morale and push people out. And here is the moment where I acknowledge the lost battle while still staring at the prospect of dealing with a messy kitchen.

Planned chaos

For established organizations, the solution is not to chastise the workforce into doing chores, but finding ways of avoiding the mess in the first place. One can despair and give in to chaos, give up on creating new products and going down the route of acquiring whichever small company survives the Darwinian grinder of the startup world, but that is hardly a system that is scalable or inclusive enough to support the industry as a whole.  Even them, without a solution for the cultural aspects and the right balance of discipline and freedom, these acquisitions will be doomed from the start.

By Nicole Quevillon
Learning fast and adaptability are a powerful combination of success factors, but ignoring past lessons baked into existing processes is a dangerous mix of  irresponsibility and innovation.

The acceptable compromise between camps seems to require a bit of discipline and planning upfront on how much chaos (technical debt) is survivable, how it will be measured, and how it will be remediated. As a concrete example, if a team decides on not commiting to a service level agreement in its initial offering period, will the team agree on implementing enough monitoring to at least keep track of service levels? If the team does not want to have a mandatory training program for reuse of open source software (and dragons be there) , should it spend a few hours publishing a list of licenses that are accepted?

Ultimately, a good conversation should pass through an examination and adaptation of processes.

And the discipline part? It just better be there.


* It amuses me to no end that cowboys and horses are used as common-place characters in analogies about poor practices.

No comments:

Featured Post

Crowds in the clouds, a brave old world