Monday, October 30, 2006

Creationism, evolutionism and intelligent design

This article is an exploration into the software development industry practices and an attempt at understanding the dominant role played by human nature into what customers end up installing on their machines.

I may not get this right on my first write up and will skip altogether the discussions around the scientific and religious arguments over the origin of life. My intention is to jump straight into the realm of discussions around the origin of software products. Note that my eventual opinions on the software analogy have no bearing on my beliefs and opinions on the mainstream theories about the origin of life.


In terms of software development, this form of software design is supported by a super intelligent, omniscient engineering body. The engineering body is formed by architects, designers and market managers that have an outstanding knowledge of the work flows that will assimilate the resulting product. Such capable engineering body is thus able to craft the software features to perfectly match and improve those work flows. The team can anticipate every single adverse affect of a product feature on the users' daily routine and produce a superior creation that is easy to use, free of defects and capable of solving all stated and unstated customer problems.

The perceived perfection of the resulting product does not warrant modifications or corrections to fulfill the original customer requirements. If anything, changes are only required in response to significant new requirements; requirements so significant that would possibly entail the creation of a different product.

I will not outright dismiss the possibility of software creationism; in fact I contend that engineering bodies should have more specialists in their field and not generalists trained primarily in the art of software development. Ideally, these specialists should emerge from the ranks of the user community and land a leadership role in the creation of the products for that same community. This perspective is somewhat of a counterpoint to the "other" creationist theory in that the creator of the perfect product is forged by its experiences with imperfect ones.

I am afraid that the software industry tends to be incredibly short on those kinds of engineering bodies, given the abyss-like disparity between the skill set of the users and the skill set of a software engineer. In lieu of transforming a customer into a software engineer, one could always contend that we could listen to them carefully. In some more mature fields, that is a perfectly good solution because the customers know their business work flows inside-out and simply expect the software products to automate some of those work flows. Conversely, emerging areas (Web 2.0 anyone?) are not a good breeding ground for creationist software design in the sense that customers often do not understand the effects of the technology on their business models and, in many cases, may not even have a use for the technology. It is not uncommon for customers to establish pilot projects to study the technology out of a herd-like fear of being left behind by the next big wave.


In this theory, software evolves by chance and probabilities. A multitude of companies throughout the world creates initial releases out of some limited understanding of the marketplace or simply to explore new concepts and ideas. The environment, represented by the market, causes transformations on the subsequent releases and the natural selection of consumer choices ensures the survival of the fittest.

Note that the quality of being fit for the environment is as impermanent as the environment itself, which translates to a very dynamic succession of killer applications vying for their very survival. As a particular branch of the environment matures, the fittest application eventually prevails and becomes an incumbent that reaches some level of symbiose with that same environment. In more precise terms, the users get used to the solution and the solution matures to a point where new releases are virtually unnecessary.

Decades-old systems such as SABRE are examples of systems reaching those levels of symbiosis with the environment, addressing their goals in a pure and efficient manner. Their survival is assured for as long as people have to reserve flight tickets, lock horns with software problems or have to correct software bugs.

Some evidences of the software evolutionism theory are recognized in form of a myriad of startups biting the dust everyday whereas an infinitesimal number of other ventures thrive against reason. I can't really draw a parallel to acquisitions of these overachieving companies as I have yet to see evidence in nature of smaller organisms being fully incorporated into larger ones :-)

Intelligent Design

In this case, the perceived natural evolution of a software product through natural selection is explained in terms of an external intelligent being driving the evolution to fit the environment characteristics. In other words, the evolution of a software product or its prevalence on the market is not caused by chance, but by intelligent interference from the engineering body behind it.

The engineering body considers the relationship between the product and the environment at several points in time and adapts the product to the changing conditions of that environment. The omniscience of this intelligent being is represented not by a superhuman awareness of reality, but by a relentless effort to understand the customer needs. Not a single effort is made to modify or design the product until its effects are fully understood under reasonably real scenarios.

It is in that context that intelligent software design is the better theory, acknowledging the gap between the expectations from the user base (the environment) and the technical bias built into products (the creation). If simply left to chance, product evolution results in tremendous waste realized in the form of throngs of projects fallen before the chosen (acquired?) one.

This is where formal software development processes such as RUP induce positive change by structuring the way the knowledge from customer operations is collected, recorded and reflected into the final product. The tight engineering linkage from software artifacts from inception to transition ensures a more deterministic intervention in the product evolution that stands a better chance of achieving customer goals.

No comments: