Monday, October 30, 2006

Creationism, evolutionism and intelligent design

This article is an exploration into the software development industry practices and an attempt at understanding the dominant role played by human nature into what customers end up installing on their machines.

I may not get this right on my first write up and will skip altogether the discussions around the scientific and religious arguments over the origin of life. My intention is to jump straight into the realm of discussions around the origin of software products. Note that my eventual opinions on the software analogy have no bearing on my beliefs and opinions on the mainstream theories about the origin of life.

Creationism

In terms of software development, this form of software design is supported by a super intelligent, omniscient engineering body. The engineering body is formed by architects, designers and market managers that have an outstanding knowledge of the work flows that will assimilate the resulting product. Such capable engineering body is thus able to craft the software features to perfectly match and improve those work flows. The team can anticipate every single adverse affect of a product feature on the users' daily routine and produce a superior creation that is easy to use, free of defects and capable of solving all stated and unstated customer problems.

The perceived perfection of the resulting product does not warrant modifications or corrections to fulfill the original customer requirements. If anything, changes are only required in response to significant new requirements; requirements so significant that would possibly entail the creation of a different product.

I will not outright dismiss the possibility of software creationism; in fact I contend that engineering bodies should have more specialists in their field and not generalists trained primarily in the art of software development. Ideally, these specialists should emerge from the ranks of the user community and land a leadership role in the creation of the products for that same community. This perspective is somewhat of a counterpoint to the "other" creationist theory in that the creator of the perfect product is forged by its experiences with imperfect ones.

I am afraid that the software industry tends to be incredibly short on those kinds of engineering bodies, given the abyss-like disparity between the skill set of the users and the skill set of a software engineer. In lieu of transforming a customer into a software engineer, one could always contend that we could listen to them carefully. In some more mature fields, that is a perfectly good solution because the customers know their business work flows inside-out and simply expect the software products to automate some of those work flows. Conversely, emerging areas (Web 2.0 anyone?) are not a good breeding ground for creationist software design in the sense that customers often do not understand the effects of the technology on their business models and, in many cases, may not even have a use for the technology. It is not uncommon for customers to establish pilot projects to study the technology out of a herd-like fear of being left behind by the next big wave.

Evolutionism

In this theory, software evolves by chance and probabilities. A multitude of companies throughout the world creates initial releases out of some limited understanding of the marketplace or simply to explore new concepts and ideas. The environment, represented by the market, causes transformations on the subsequent releases and the natural selection of consumer choices ensures the survival of the fittest.

Note that the quality of being fit for the environment is as impermanent as the environment itself, which translates to a very dynamic succession of killer applications vying for their very survival. As a particular branch of the environment matures, the fittest application eventually prevails and becomes an incumbent that reaches some level of symbiose with that same environment. In more precise terms, the users get used to the solution and the solution matures to a point where new releases are virtually unnecessary.

Decades-old systems such as SABRE are examples of systems reaching those levels of symbiosis with the environment, addressing their goals in a pure and efficient manner. Their survival is assured for as long as people have to reserve flight tickets, lock horns with software problems or have to correct software bugs.

Some evidences of the software evolutionism theory are recognized in form of a myriad of startups biting the dust everyday whereas an infinitesimal number of other ventures thrive against reason. I can't really draw a parallel to acquisitions of these overachieving companies as I have yet to see evidence in nature of smaller organisms being fully incorporated into larger ones :-)

Intelligent Design

In this case, the perceived natural evolution of a software product through natural selection is explained in terms of an external intelligent being driving the evolution to fit the environment characteristics. In other words, the evolution of a software product or its prevalence on the market is not caused by chance, but by intelligent interference from the engineering body behind it.

The engineering body considers the relationship between the product and the environment at several points in time and adapts the product to the changing conditions of that environment. The omniscience of this intelligent being is represented not by a superhuman awareness of reality, but by a relentless effort to understand the customer needs. Not a single effort is made to modify or design the product until its effects are fully understood under reasonably real scenarios.

It is in that context that intelligent software design is the better theory, acknowledging the gap between the expectations from the user base (the environment) and the technical bias built into products (the creation). If simply left to chance, product evolution results in tremendous waste realized in the form of throngs of projects fallen before the chosen (acquired?) one.

This is where formal software development processes such as RUP induce positive change by structuring the way the knowledge from customer operations is collected, recorded and reflected into the final product. The tight engineering linkage from software artifacts from inception to transition ensures a more deterministic intervention in the product evolution that stands a better chance of achieving customer goals.

Tuesday, October 24, 2006

Using magnets to understand the corporate culture

Magnet Grabbing Businessman
I have always been fascinated with the how human nature mirrors several aspects of Mother Nature. A natural extension, some of those patterns also repeat themselves in organizational systems created by humans.

A good example comes from magnets and organizational culture. A magnet made from an iron core has millions of microscopic metal particles polarized in the same direction. In the absence of that cohesive field orientation across all particles, you just have a regular piece of metal.

When you apply an orthogonal magnetic field, those metal particles tend to reorient themselves along the axis of the external field. When the external field is removed, the metal particles tend to return to their original alignment, but not entirely. This effect is known as hysteresis. In simple terms, magnets have a memory.

And now to the analogy to organizational structure. In a large company, employees are the little metal particles united in a common body. With the proper external force, the particles will align in the same direction; remove that force and the metal particles tend to go back to their old orientation. Apply a strong enough field for a long enough time and these particles tend to stay permanently aligned in the same direction, thus creating a magnet.

Another interesting part of the magnetization phenomena is that the reorientation happens in individual clusters rather than in a uniform way throughout the metal body. In other words, some portions of the metal reorient themselves along the new field before others do. At some critical point, there is an avalanche effect where all particles fall in line under the influence of the external field and its neighboring particles. There is a nice animated illustration for the effect here.

Sounds familiar?

I just liked the thought and its implications to the challenge faced by executives to reorient a company in a new direction, keeping the external field in the desired direction long enough for the employees to start maintaining that direction on their own.

Can you do your part in a situation where change depends primarily on external factors? Sure, as long as you are aligning yourself with the external field, you help exert field pressure on your peers. Typically, that is the role of early adopters, evangelists and technology champions; helping the top-brass move the company towards the intended direction.

Welcome to The RTP Scrolls

"The RTP Scrolls" is a blog dedicated to my observations about corporate culture and product development. The Internet is crowded with many good resources on both topics but I always wanted to write about the influence of the first over the second.

I believe human nature plays a large role in the way corporate culture is shaped. I also believe that human nature and corporate culture tend to affect the judgment of product development teams in ways that they cannot understand, or at least acknowledge at a conscious level.

The intent of "The RTP Scrolls" is to draw analogies from other fields of the industry (automotive and construction are my favorites) with the way software is created. I will also have some fun drawing the analogies from unexpected sources, borrowing examples from Mother Nature, mathematics, physics, biology, politics and on occasion, metaphysics.

PS: RTP stands for Research Triangle Park, a public/private, planned research park, created in North Carolina in 1959 by leaders from business, academia and industry. You can read more about RTP in http://www.rtp.org.

I hope you will enjoy the content as much as I have fun writing it.

LinkedIn