Thursday, November 30, 2006

Ford Taurus, and innovation turned disaster

Since I took exception with the construction industry on my previous posting, I thought it would be fair to take a stab at my second favorite counterpoint to software quality problems: the auto industry.

To make the point, I resorted to the news of October 27th, where Ford Motor Company announced the end of the line for its (once) beloved Ford Taurus. After almost 7 million units sold, the automaker called it quits on one of the least known cases of success turned failure in modern industry.

The Taurus history begins at the end of 1985, with a rounded profile that blasted the boxy gaz-guzzling competition. After six years in production and facing mounting competition from increasingly better Japanese cars, the new cosmetic modifications to the 1992 model fueled a 5-year streak as the best selling car in USA.

And then tragedy. On the trail of that enormous success, Ford introduced the 2nd generation of the car, on 1996. In the less endearing terms used by Ford's engineering teams, it became known as "the car that could not be built". Its impossible combination of curved oval shapes colluded with the free falling quality control standards in the US auto industry to deliver the most severe blow to the company fortunes in decades.

The complicated assembly process forced the prices to go up and the body panels to go sideways (in opposite directions). On 1997, the car lost the top honors as the nations' best seller and forced Ford into the death spiral of pushing the cars at a loss. The idea was to keep the assembly plants running and bet on the sales of large trucks and SUVs. Meanwhile, its Japanese competitors worked overtime to meet the demand for their own best sellers. The sales chart hides the loss of favor amongst US consumers, and incorporates fleet-rental sales that accounted for the bulk of the Taurus sales towards the end of its life.

What happened during that 11 year span? Essentially Ford executives missed a market transition to where customers had overtaken product control from the companies. They decided to innovate from a shell, transplanting its oval emblem to everything in its best-selling model, from door handles to dashboard shapes. Deciding for the customers had worked in the past and lifted the company from a financial tail spin on 1986. Ten years later, the same recipe (go rounder than the competition) resumed the tail spin.

You can read more in this insightful book by Mary Walton: "Car: A Drama of the American Workplace".

Oh yes, what does it have to do with technology companies?

I think we all know. Of course, 20 years is an eternity for a technology product, but specialists in the industry all agree with the combination of increased complexity and lost quality as the source of Ford's impending demise. Insult added to injury, the Taurus '96s design was widely perceived as ugly by the press and the market. That ill-received design was somewhat rectified 4 years later, but its competitors countered it with even better all-new models packing engineering years ahead of what the cash-strapped Ford engineering teams could muster.

Tuesday, November 28, 2006

Road building gone awry

For those living in the RTP area, the gigantic failure of a recent resurfacing job in the I-40 interstate is already old news.

In the past I have cited the construction and auto industries as examples of more mature industries. In my view, these industries have better established practices and are more likely to achieve consistent results. Nevertheless, this example shows that they are not immune to problems when those practices are abandoned in favor of untested "innovative" techniques.

In this particular episode, the Department of Transportation decided to go with a concrete overlay procedure that had never been tested with such big traffic volume (100.000 cars a day). The DOT engineers lifted the procedure from a small road in the Raleigh area, made a few assumptions, and sold the idea to the state officials.

At a cost of U$18.6 million dollars, the reconstruction effort will involve the removal of 50.000 tons of concrete and subsequent resurfacing with regular asphalt. For the innovative employees, the consequences are expected to be just as dire, in the words of the Secretary of Transportation:
"My investigation revealed that a number of state DOT employees were at fault from 1999 through the completion of the project in 2004," he said. "Many of these employees have since retired or otherwise left the department. Others who still remain with the department will be subject to suspension, demotion or firing. "
Interesting and alien aspects to software engineers; ranging from a failure that cannot be explained away in terms of unrealistic expectations (the asphalt should last 30 years and is crumbling after only 2) to documented links between the failures and the people responsible for it.

In the presence of real liabilities, it seems that innovation should be restricted to the labs until proven. At least if one is building cars or roads.

Tuesday, November 21, 2006

The Hive Mind: gravity, magnetic fields and electricity

"I postulate the theory of the "Hive Mind", where individual or collective thought carries over great distances..."
A few weeks ago I explored the parallel between physics and corporate behavior, applying the concept of hysteresis to explain the mechanics underpinning the adoption of new technologies and processes within a large corporation. This week I wanted to ponder this interesting article mentioned on CNN, about a successful experiment involving people communicating through phones and e-mail.

The experiment showed that some people could actually predict the identity of a phone caller or email sender with a greater accuracy than simple statistics would dictate. The experiment lacked a more controlled environment to be considered scientific proof, but I somehow share the scientist's belief that there is an interconnectedness of all minds within a social grouping. If anything, there is a higher probability that we spend more time thinking about the same subjects as people in our social grouping than thinking about subjects that are not interesting to that group.

The shared interest on the same topics is part of the glue that holds a social grouping together, including the ones at the workplace. That shared interest is possibly a stronger bond in our diverse corporate environment because we are brought together around a shared goal and often lack the other demographic background that unites a social grouping outside of the work environment.

Without getting into the telepathic effects explored in the experiment reported by CNN, it comes as no surprise that sometimes we find ourselves thinking about writing an email to someone and are suddenly interrupted by that same person calling on the phone or IM. I am willing to go beyond the inference that such phenomenon is the result of diligent statistics at work; offering a hypothesis that explains the (unverified) results observed by Rupert Sheldrake, the British scientist behind the experiment.

I postulate the theory of the "Hive Mind", where individual or collective thought carries over great distances and establishes a communication mechanism with other people beyond the currently known media; an effect that will be completely explained by (then) common physics years from now. Furthermore, I postulate that such magnetic field is also a component in the organizational realignment example introduced in my previous analogy between magnetism and corporations.

Our brains generate an elaborate wave pattern while working through any technical challenge; it is somewhat expected that the average human brain activates the same regions of the brain while solving similar problems. Now, those brain waves are formed by massive (for our body masses anyway) electrical currents traveling over thin conduits, which is bound to produce a coaxial magnetic field around these conduits. Magnetism, electricity and gravity are all related forces according to the Unified Theory for Electricity, Magnetism, and Gravity, which offers a more complete model for the physics underlying the "Hive Mind" hypothesis.

Magnetic and gravimetric fields travel unimaginable distances through a mechanism that has eluded the most brilliant scientific minds of all times and will demonstrably reach people across the globe. I could even entertain the idea of other kinds of unknown fields beyond these two but would do so at the peril of a weaker hypothesis. The effect on the people reached by these fields would largely depend on the magnitude of the field and the susceptibility of the receptor to the field. Newtonian physics show that the effect of the field diminishes significantly with distance (to the tune of distance3) but do not account for the repeating effects of intermediary brains resonating and amplifying the waves along the way.

Another interesting thought about the Hive Mind comes from the effects of mass in the laws of gravity. What is "mass" in the hive mind? The strength of the transmitter, or the intensity of the thoughts around a certain subject, are certainly a factor; but the resonating effects of the retransmitters (other people's brains) in the hive mind are bound to offset and surpass the strength of the original thought. In other words, the "mass" parameter in the intensity and reach of a particular line of thought is dictated by the number of people attuned to that line of thought.

The tribulations of our daily lives and the strength of our individual thoughts introduce a lot of background noise to the resonant waves of the Hive Mind; creating a somewhat healthy obstacle to a Borg-like conscience breeding out of the work place. The magnetic properties of corporate direction covered in the posting about hysteresis also explain some of the impediments to a freer flow of waves in the hive mind.

At any rate, the hypothesis of the Hive Mind is worth a thought, or many.

And if you were thinking about the exact same thing but never cared to write it down, do not be spooked; maybe I picked the idea from you ;-)

Angry monkeys and cargo-cults

A good friend of mine shared his experiences at the annual "No Fluff Just Stuff" conference. This event features presentations on the themes of Java and Open Source and is open for external participation. It spans several US states and in Raleigh is hosted at the SAS institute.

He was ecstatic about the keynote presentation by Dave Thomas, co-author of the Agile Manifesto and the best-selling "Pragmatic Programmer".

In his talk, Dave turned to the story of "cargo cults", the aboriginal religions grown in the South Pacific during WWII. The significance to software development was obvious even before Dave started to address it more directly on the topic of "cargo cult programming".

Dave also turned to the famous anecdote of "Angry Monkeys", where scientists supposedly started an experiment with five monkeys in an enclosed environment. One of the central elements in the environment was a banana-tree, which the monkeys immediately attempted to climb. Whenever a monkey attempted to climb the tree, the scientists hosed all of them with water until the offending monkey climbed down.

Over time, whenever a monkey attempted to climb up the tree again, even before the scientists reached for the water-hose, the other monkeys would readily beat the upwardly primate. After a few days, none of the monkeys attempted the banana-gathering stunt anymore.

Then the second part of the experiment starts: the scientists replaced, one at at time, the water-savvy monkeys with monkeys that had not been water-hosed before. The scientists replaced a watter-savvy monkey as soon as the new monkey ceased his attempts for banana sprints through the reassuring beating of the other monkeys.

In the end, once all the original monkeys were replaced, none of the new monkeys would attempt to reach for the bananas anymore, even though they had never been interrupted by a stream of water while trying to do so. They only knew that they should beat any monkey who attempted, no questions asked.

Now, I don't know about you, but how many of the original people that worked on your project are still around?

Dave's points in his speech were about how software developers will learn to accept self-evident truths and not attempt something new or beneficial because something has not worked before. The conditions in our industry change so quickly and technologies may have reached the absent maturity that made them fail in the past.

Thursday, November 16, 2006

Up on the wall: when your career decisions are not yours

Last week I attended a two-day class titled "Problem Solving-Decision Making-Critical Thinking". The course was divided in 5 modules, one of them about decision making. One of the exercises was about choosing a new project lead amongst 4 candidates. We were split into 3 different teams of 5 people to make that decision.

Each team had a large whiteboard to list the decision objectives, alternatives, benefits and risks. The three teams eventually arrived at the same conclusion by applying a formal decision process starting from the performance evaluations and availability data provided for each candidate. As we ruled out one candidate after another and assigned weights to their strengths and weaknesses, something dawned on all participants as revealed later during a lunch break conversation:

"It could have been any of us"

The thought of having your name up on the wall and your skills and traits (including personal ones) dissected by an objective decision process can be frightening. For one, you are not there to explain past incidents or how some perceptions may be inaccurate. Many of the evaluators misread the availability of one of the candidates and ruled him out in the first minutes of the session. Additionally, the majority of the participants assumed that the strongest technical person would not be a good influencer for the rest of team. For this last assumption, everyone went along with the second assumption that the chosen candidate could always consult with the runner up.

With more than 30 minutes, these makeshift "decision boards" probably could have requested more information and arrived at different conclusions. One of the participants argued that the people making the decision would probably be the managers of the candidates. Others dismissed that assumption in favor of another hypothesis, where the first-line managers would present their assessment to the mid-level managers actually responsible for the final call.

Regardless of how someone's name reaches the board, fair decision makers will slap numbers in front of pros and cons, tally up the scores and pick a name. If all the right data and evaluations are considered, the right person will get the job. The others? Well, they may never known that they have been up on the wall.

Sunday, November 12, 2006

DNA Hardware and the final frontier

I did not mean it literally when I wrote about Software DNA, but DNA hardware seems to be at hand: 'Alive and clicking, the computer made from life itself".

The notion of an organic computer reminded me of a Time Magazine article titled "What will replace silicon". It also points us eerily past the reality of an intriguing article featured the Fortune Magazine, titled "Quantum leap", where headbands enable the interfacing between the human brain and powerful and pervasive computers.

For some reason, the possibility of organic computers that can be injected and integrated into the human body has darkened my thoughts about the future of mankind. During WWII, the battles between Axis and Allies were fought over world domination as much as they were fought over the unwarranted genetic superiority of a race.

Decades later, Jared Diamond, in his excellent "Guns, Germs, and Steel", further challenged the myth of genetic superiority in favor of geographic and societal factors.

What darkened my perception of the future was the thought of what could happen when genetic superiority becomes tangible and measurable. Ethics, science, law, even religion, as we know them will be challenged in ways that will dwarf the controversy over the cloning of human beings or stem cell research.

Far from the romantic belief in the superiority of men over machine, hidden in the inscrutable complexity of our brains and the scientifically unproven existence of an omniscient God-like entity; the possibility of an organic computer completely sidesteps the discussion. Humanity does not have to be defeated, just subverted. It is a subtle argument, but once we can no longer separate between what we wear and what we are, then Jared Diamond's book will have ceased to be a treaty about the triumph of human potential and have become a book of history about the time when all men could evolve equally given the same resources and conditions.

Some could argue that the innocent usage of the technology should preserve our humanity, but then again, humanity is not known for moderation. No matter how much I try, I cannot avoid imagining the 'what if' scenarios explored in John Mnemonic (1995) and Gattaca (1997) .

As a fellow blogger once said, "I don't have to like the future to know it is coming". And it is coming fast.

Monday, November 06, 2006

Software DNA

On the trail of Creationism, evolutionism and intelligent design, I wanted to introduce the notion of software DNA, which can be used to measure the quality of a project. There are several somewhat expected conclusions in the following posting but it is interesting to see Mother Nature at work in what is otherwise an aseptic activity.

We can all accept the notion of a "DNA" signature for a product, a sophisticated mesh of everyone's contributions to the final outcome. The interesting part of the analogy starts from how these contributions are distributed over time. Product development is a somewhat pipelined series of activities, with each stage requiring the intense participation from different sets of people. As the assembly moves on the pipeline belt, those sets of people donate their "DNA" to what is going to be the final product, compressing several generations of evolution in a matter of a few months.

In biology, a diverse pool of genes is a crucial factor for a successful species. The diverse DNA combination increases the odds that one will survive the harshness of the environment, while nature callously weeds out the less adaptable variants. Back to the software analogy, that diversity is equally important, originated from contributions from people from different backgrounds, with different expertise and perspectives on how the product should be shaped. Before I continue, think of it the next time you feel your participation is too small to make a difference.

From a customer perspective, that evolutionary diversity is beneficial as it results in better products. Even the powerful "species" coming out of the Microsoft pipeline can have their survival threatened by random mutations on foxes, owls, and penguins.

Although companies will try to influence the environment to improve the odds toward their offspring, environmental control is become increasingly difficult with long-tail markets emerging everyday. When environmental control becomes difficult to manage, the only hope for a company is to revert back to its own resources, carve the best product out of its gene pool and hope that the environment will be kind to it.

I already explored the importance of diversity in the early paragraphs; now let's take a look at mutations. Skunkworks, research, and an emboldened community of innovators are a fantastic start because they represent the random mutations in the gene pool. Some mutations are more beneficial than others and the environment can be harsh with seemingly superior species (can you say Betamax?)

But there is one gene we want to preserve in large and healthy doses in the final species: the customer genes. They are (or should be) injected on the first cells of a product and preserved in the final species at all costs. Skipping the business modeling phase for a project is an almost fatal handicap to any species. The lack of those genes makes the species unattractive and as soon as the parents exhaust their capacity to incubate it, it becomes extinct.

The lack of formal requirement analysis will also push those genes through the cracks in the pipeline. Here is a less obvious point. The new software is already in contact with the environment while it is being created. Once the customer genes are injected into the initial species, they are visible in the pipeline and some may be unpleasant to the DNA contributors. Without proper enforcement, an ill-controlled environment may become hostile to certain desired characteristics.

I do not see bodily rejections of the customer DNA in the software creation process as a sign of corruption or shortsightedness, but rather as a natural occurrence. While we do want the final product to preserve the customer genes, we need to acknowledge that they make the creation process harder. Without good safeguards for those genes, they become easy prey to a mismatched set of localized liabilities and greater goals.

As examples, even with the foreseeable disastrous results, assuming what a customer wants instead of actually asking the customer takes much less time and effort. In a large organization with experienced architects, those kinds of assumptions tend to fall on the creationist side of my previous posting titled "Creationism, evolutionism and intelligent design". If you ever take time to read, you will agree that software creationism is extremely hard to pull off.

Conversely, assuming that certain conditions will not be present during real deployment can save development time by avoiding more costly test injection techniques. The rugged reality of customer environments can be an exacting master to people making those kinds of assumptions. And if it was hard to debug it in the lab, try and imagine debugging it in the wild.

In the end, one can try to rely on common sense and good will, but nature is a mighty adversary. Creation of software is still an engineering practice, even when seen through biological lenses.

Featured Post

Crowds in the clouds, a brave old world