Monday, November 25, 2013

What is your problem? – Part 2: Real versus imaginary problems

To ask "What is my contribution?" means moving from knowledge to action.
The question is not: "What do I want to contribute?"
It is not: "What I am told to contribute?"
It is: "What should I contribute?"
Peter F. Drucker

In the first part I covered the general aspects of identifying and reporting problems. Now it is time to apply those concepts to my domain of choice: software engineering (minor apologies to the software gardeners out there, I will come back for you in a future posting) .

I use the Agile method as the backdrop because it has completely overtaken the field to the point of erasing debate on alternatives (defeated waterfall proponents are still called through the backdoor to fill the gaps with valuable contributions, but I digress) .

As a short Agile method recap, work is delivered in small iterations called stories and executed over relatively short intervals, called sprints. If you can tell a story while on the run (sprints, running, see what I did there?) , you know sprint durations can range from ear-bleeding single-week sprints to waterfall-bordering 8 weeks. Beyond 8 weeks, there be dragons and T-virus, walk backwards slowly towards the nearest door and avoid eye contact at all costs.

The drudgery of tasks…

A fundamental tenet of Agile method is that stories be written in the form of “As a [role name] I need [a feature] so that [I can achieve a goal]”.

I personally prefer to replace “achieve a goal” with “solve [one of my] problems”. There are far more people in this world facing immediate problems than people who have goals, and even for the people who have both, the immediate problems tend to grind one’s will and resources to execute on long term vision.

That is not to mean Agile should be cast as a reactionary method that can only tackle situations after they have become a problem, but that any complex project can be mapped to a mind-map of tasks, where each node can be represented as a problem to be solved.

… is no match for the challenge of a problem

The payoff for such mental gymnastics is that a problem statement engages, whereas a task dehumanizes, and success in software development hinges on engagement: on engagement between developer and customer, on engagement between user and solution, and as a more recent phenomenon, engagement amongst users.

It is part of many professions to walk into an engagement where the customer knows exactly what they need, are willing to pay for those services, watch you walk out of the door after completion and then deal with the next task in a master project plan.

That is invariably not true of software development for two main reasons: (1) our largely INTJ subtlety-loving personality is prone to invent dozens of different ways of achieving the same goal with dozens of distinct advantages and disadvantages for each choice, (2) we collectively get bored of those solutions more often than we should, reinventing the field every couple of years in a way that is utterly incompatible with what we once thought to be a good idea.

With all that said, as a prospective customer to a solution requiring software, whenever you engage a developer or a development shop, the first part of your homework is to be absolutely sure the problem you want solved is one of your top-most problems, lest you (or your company) may not sustain the motivation to see it through. As the person (or company) experiencing the problem, and this may seem outrageous since you are about to pay for contracted services, you will be integral part of the solution: there will be hard questions to be answered at some cost of time, intermediary solutions to be attempted, final validations to sign off, all activities that will require your attention and resources.

Know your problems…

As a prospective software provider, accepting a task at face value is a recipe for disaster.
A good software architect must know how to artfully act as a devil’s advocate while engaging a customer, not to blindly question motivations, but to understand why the customer needs a solution. In other words, a good architect will ask the customer “what is your [his] problem?”
I often hear from peers disgruntled with the fact a great idea was not accepted by a potential customer, at the same time failing to recognize the problem solved was not all that important to the customer.

… know thyself

In a previous project I joined a team which had developed an internal tool for analyzing log data from hundreds of products. At the time I was the enablement lead for the technology and really got behind it. We persevered for a long while to make our world-wide support team adopt the(internal) product. After a relatively extended period of … err…lukewarm responses, we changed our approach, meeting frequently with these support teams and also with another internal development team who was already successfully supplying tools to the support organization.

It became clear that our log analyzer tool, however sophisticated in what it could do with log files, required memory capacity that, although readily available to our development team, was unthinkable to a support engineer. This tool also had the ability, developed at great expense, to shift the memory requirements to a relational database to cut down on memory usage, but deploying and maintaining a relational database was equally unthinkable for an audience which had no expertise in managing such systems.

The question no one asked…

At the same time, meeting with the more successful internal development team revealed their key selling point to the support organization: their solution was based on a SaaS model and the support teams could access most of its function through web interfaces, avoiding the need for high-end systems and the costs of installing and maintaining new tools. Their tooling also integrated with another SaaS offering where customers submitted all the supporting information, including the all important log files, for any problem reported in the field.

In the end, building (or selling) a log analyzer to a support team which routinely performed log analysis seemed like a success story in the making, but it failed to recognize two key aspects:
  1. Their most common activity related to log analysis was to isolate error entries in log files, then use snippets of the log entry on Internet searches (incredibly effective) , a feature absent in our tool.
  2. All the information used by the support teams resided on virtual services, requiring only a web browser on their machines, which side-stepped the need for high-end systems.
At the time, and this was a remedial approach to not having asked the “what is your problem” question first, we decided to harvest the analysis internals from the tool, put it under the web-based interface already being used by the support team and surface the analysis results through a page that only contained warning and error messages, with a quick link to an Internet search based on the error message.

…is the problem no one had

Had we asked the harder questions first, the answer would have been a SaaS version of the patent we filed a few months later, where log files submitted to customers were automatically analyzed, cross-searched on websites, and results ranked according to their rate of incidence in results. At that point I left the team for other reasons, but I am told they kept on delivering on that vision.

Something fantastically useful eliminates a problem that is really at the center of someone’s attention, and not something you set out to improve, however successfully. Success also involves far more than technique and technology. In fact, too much technology may just put the solution out of reach for the target audience, in requiring system upgrades, training, and adaptation.

In closing, I end with a quote that symbolizes the consequences of offering a solution on the basis of vision without sufficient understanding of the problem, resulting in one of the costliest mistakes of the kind in recorded history:

We could hardly dream of building a kind of Great Wall of France, which would in any case be far too costly. Instead we have foreseen powerful but flexible means of organizing defense, based on the dual principle of taking full advantage of the terrain and establishing a continuous line of fire everywhere.André Maginot
December, 1929

Wednesday, October 30, 2013

Critical thinking and cows in the field

imageI heard this fictitious story as a joke some 20 years ago from my college-time great friend, Walfred Tedeschi, and it has stuck with me for all these years, first for the good joke that it is, then as a profound lesson in critical thinking.
In a gathering at the University, three students see a cow in a nearby field. One of them is a Mathematics student, the other a Physics student, and the other a Software Engineering student. Note: The fields and order of participation change depending on who tells the story, and I can say that as a Physics student at the time, Fred would not agree with my recollection, but it goes like this:
The Math student hears a mooing sound to his left, about 100 yards away, turns around and says: “Look, a cow” .
The Physics student turns around in the same direction, then adds: “A spotted cow”, with a slightly overbearing emphasis on the word “spotted”.

The Software Engineering student analyses the situation for a few moments, then concludes:  “At least on the side we can see.”
That punch-line is the one aspect that makes or breaks a great professional, in that it surfaces how different people may perceive the exact same situation (there is a cow in the field) with varying degrees of details and how those details may be factual (the visible side of the cow is spotted) or inferred (a cow spotted on one of its sides is likely to spotted on the other side) .

I don’t think this is the basis of a new philosophy graduation course, but it is worth telling people starting in their careers, both in the leadership and exact science fields, to guide the way in which we reach conclusions and how we trust information while making decisions or establishing a new hypothesis.

Later in life people may question why a cow in the field was even worth noticing, or whether the students were so distracted by the verbal sparring that they missed the sight of a golden unicorn a few hundred yards to the other side, but those are stories for a different time.

Friday, March 08, 2013

Crowds in the clouds, a brave old world

Sunrise at King penguin colony, Salisbury plain
Sometimes, I like crowds.

I will soon be flying home, back from Las Vegas, where I had the privilege of attending IBM Pulse's conference.

The opportunity to meet in person many colleagues and friends from all over the world is equal only to the opportunity of listening to some of the most prominent voices in the technology field, from some of our brightest colleagues, from analysts, from business partners, and specially from our customers.

These interactions are an anchor in reality that cannot be taken lightly and at the same time are an anchor of a scale and relevance that are almost impossible to comprehend. This conference has become the equivalent of a small city of over 10.000 people, which is created and torn apart in the span of a few days. I wrote what I could during the conference, but Twitter only goes so far to convey the sense responsibility that comes with working in the information technology field.

In the view of CEOs, information technology is now the most important aspect of the their companies' future. It is also a critical aspect to the future of the entire world. Connectivity and smart devices are shaping an entire new dimension of interactivity between people, governments and the enterprise.

The democratization of technology... 
If you lived your career through the 90s, for a while it seemed technology would only get faster, until it became totally interconnected and different altogether.

For the first time in modern history (and I use 'modern history' very judiciously here) , self-organization, information sharing, and merit-based leadership have allowed crowds to emerge and galvanize quickly, and somewhat effectively, around subjects ranging from designing a new product, to funding new ideas, to fighting against tyranny and oppression.

The world is small again, and technology has rescued fundamental aspects of human nature from the incomprehensible and ever increasing size of our world population. Easier access to means of production enable people to become direct producers once more, weakening the hold of wage-to-capital relations; pervasive access to means of communication eliminates barriers between individual producers and individual consumers, it reconnects people with their power to decide their destinies and ultimately reconnects individuals with the rest of the world in more meaningful and direct ways. rewriting the books on capital ownership...

The power of the masses puts tremendous positive pressure on leaders from public and private spheres in ways that even the most jaded of citizens cannot refute. Crowd-sourced projects like Domino's Ultimate Delivery Vehicle or movements like the Arab Spring are evidence that the mere existence of crowds can make executives and governments see and engage people in a whole different light.

Open and free platforms for supporting virtual communities are already part of daily activities for a large portion of the world population, 3D printing is maturing at a rapid place, new materials and more efficient recycling technologies will further reduce the importance of capital in human initiative.

In the not so distant future, you will be able to ship a toy to a shop across the country and get it "restructured" in a recycling unit that can reprint the raw materials in the format of another more interesting toy for your growing child

Imagining the gadgets of the future, however exciting, is not nearly as interesting as imagining the dramatic implications to economic and social relations of the future. Micro-financing is a fantastic example, closing the gap in democratizing the access to means of production, a reality for many small businesses in less developed areas of the globe, with rates of default that are smaller or equal to those of traditional bank financing.

Fantastic initiatives like the one from Marcin Jakubowski, outlined in his TED talk titled "Open-sourced blueprints for civilization", also point the way at closing the gap on intellectual capital. Ever increasingly, people are having more access to knowledge, means of financing, communication, and production, than ever before.

...and on social relations

Cities are learning how to integrate immediate feedback from the general population into their own management systems. Researchers are also exploring with how to understand the sentiment of a city. Technology is not only reaching the masses, it is starting to understand the masses, and that is only the beginning.
Regardless of how one may question the motives of companies and governments, it is important to realize that what we are seeing now, however unprecedented, is also a very small step in enabling a different future for humanity.

For those skeptical and suspicious of technology, the only message is to accept it without fear, because the only other option is to become part of a lost generation. The advancements are not here to rob us of our identity and individuality, they are here to restore these qualities in ways modern civilization has long forgotten.

Just like when technology evolution surprised everyone for not being just about making things go faster, and while everyone is grappling with the evolution of the new small world, technology will change the very nature of society and the economy itself.

The new companies and the new governments

Companies will be challenged by customers and employees to achieve a sustainable model that is not based on the ownership of capital, of intellectual property, or of distribution channels. Governments will be forced to adapt to collective participation far beyond general elections. Failure to invest in education for creativity and to adapt the education curriculum for a new merit-based economy will land millions of people in a fairly uncompetitive heap. Allegiances will be formed to communities, not to companies or to countries.

Successful economies and business models will thrive on unlocking the power and creativity of individuals, in forming communities of individuals with large overlaps between personal goals and business goals. Being competitive will be more about doing things that others cannot do than about doing things cheaply and more efficiently.

Being in the technology field for a relatively short (or long) 16 years, I cannot contain my enthusiasm for the things to come in the next 5 years, let alone in the next 16 years.

Wednesday, January 16, 2013

The Agile Enterprise - Communication, collaboration, and cooperation

I received a link to an article titled "Team Collaboration the 2.0 Way", extolling the virtues of Web 2.0 collaboration over regular communication and wanted to register a point I often make in association with the notion of an Agile enterprise:
Collaboration does not foster cooperation, collaboration is premised on the need for cooperation.

Featured Post

Crowds in the clouds, a brave old world