Lost Potential

Much of this software, in turn, has no proprietary value for the enterprises that develop it. So why isn’t the world deluged with enterprise-written open-source software? Why do so many CIOs gladly use open source but not contribute to it? Because open source is hard. Really, really hard.

So says Matt Asay in his Register column, and while I don’t disagree I also think it’s a lot like everything; the more you practice the easier it gets, and there are plenty of companies, large and small, that do push projects out as open source and over time they create a processes and policy to support that.

In fact the real problem is that Software is hard. Really, really hard – governance, code provenance, peer review, modularity, source code control, tracking bugs, responding to user/developer queries, etc are all things you want to do or issues you need to tackle in any well run software project regardless of the project being open or closed. Keeping a project in-house simply offers the possibility (the temptation?) to skimp or short cut on these things. Making a project open is a commitment to a wider community to do things right and be held accountable.

Aside: Kent Beck once commented at OSCON (in 2004 IIRC) that open source developers kind of have Maslov’s hierachy upside down, ie they start with self-actualisation (creating a project to scratch their itch), which can the lead to strong relationships with like minded people, respect from peers, and finally (if you’re lucky) a job to provide for food and shelter.

Matt links to Jim Whitehurst who laments “ Think how much software is written out there that is behind proprietary walls“. Again I like the sentiment but don’t regret the lost code or lack of economic efficiency as much lament the lost opportunity for personal satisfaction and growth, the missed opportunity to retain and nuture talented, creative people.

How many companies struggle with staff motivation and retention when experts from Dan Pink to Pete Carroll will tell you that the secret to happy and fulfilled employees is having autonomy and mastery over your work, to have a purpose in what you’re doing. Nothing beats this kind of retention policy or motivation, not even money (if fact simply giving people more money can, in fact, be de-motivating).

Maybe I’m just finding another scapegoat, but it seems that the real problem is not so much that doing open source is hard, or that lawyers get in the way, but that accounting for the benefits and costs of doing open source is too hard, so the value can be ‘proved’. Where the value is clear (or someone is willing to trust their gut) getting over the organisational hurdles will happen

“According to Whitehurst, JP Morgan’s CIO realized that support costs could be reduced by contributing the source code to the Linux community. Other Linux users would benefit, which would be nice… but more important to JP Morgan, the company wouldn’t have to invest its own resources in maintaining an internal application. The Merge code would now be updated and enhanced by Linux developers at large, in addition to any committers on its own staff.” From cio.com

This is presented as a pretty cut and dry case of not wasting resources (and therefore money) but actually the accounting is incomplete because various things are not factored in;

  • maintaining an internal fork is boring, fiddly, unrewarding work compared to working on the real Merge functionality – what of the affect on team morale, productivity, and retention
  • external usage can provide feedback and alternate deployment scenarios and testing which make Merge more robust (if you manage it correctly)
  • if your project becomes a standard open source component in it’s class, other parts of the stack will be written to integrate, interface, or compliment so ultimately your systems become easier and cheaper to develop

Suppose in the JP Morgan case making Merge open source actually required more, not less, resource – how do you factor in the above (and other negative factors) so you can calculate the break even point. Unfortunately I don’t know but I would wager that there is a strong correlation between teams efficiently delivering great products and/or services and management that creates opportunities for the their teams to grow and to build software the right way (which may or may not include making it open source – that’s actually a secondary detail).

Making the worker achieving implies consideration of the human being as an organism having peculiar physiological and psychological properties, abilities, and limitations, and a distinct mode of action. It implies consideration of the human resource as human beings and not as things, and as having—unlike any other resource—personality, citizenship, control over whether they work, how much and how well, and thus requiring responsibility, motivation, participation, satisfaction, incentives and rewards, leadership, status, and function. — Management by Peter F. Drucker [Kindle Edition]

I think this is the toughest challenge in any organisation – it’s a complex, inter-related, non-linear system, where the smallest, apparently insignificant, changes can have butterfly effects through the system, where small costs saving in one area can lead to huge deficit in others, where a small investment in something not “business critical” can indirectly reap huge dividends. Enlightened accountants are welcome to tell me how to do a cost benefit analysis of this upfront.

However I suspect it’s not possible otherwise we could just run everything through a spreadsheet and every company, charity, government dept, would just run like clockwork. Instead I’m reminded of possibly the greatest ever company about page of Ludicorp (the company that created Flickr) which quotes from “Disclosing New Worlds: Entrepreneurship, Democratic Action and the Cultivation of Solidarity by Charles Spinosa, Fernando Flores & Hubert Dreyfus (MIT Press 1997)” of which I will cut a snippet

One might think that they view their businesses as nothing more than machines to produce profits, since they do closely monitor their accounts to keep tabs on those profits.

But this way of thinking replaces the point of the machine’s activity with a diagnostic test of how well it is performing. Normally, one senses whether one is performing skillfully. A basketball player does not need to count baskets to know whether the team as a whole is in flow. Saying that the point of business is to produce profit is like saying that the whole point of playing basketball is to make as many baskets as possible. One could make many more baskets by having no opponent.

The game and styles of playing the game are what matter because they produce identities people care about.

Management is hard. Really, really hard. Man Up.