The Buy versus Build Shift: Issues With Buying Software

In part 1 of this series I discussed traditional reasons people have for buying software, which turned out mostly to be based on perceived or real issues with building software.  I discuss in this post, which contains parts 2 and 3, issues with buying software.

Part 2: Traditional issues with buying

Motivating the decision to buy software by pointing out the problems with building software can be effective, but it implies an important assumption, namely that there are no issues with buying software or that these issues are smaller than the issues with building software. It should be pretty obvious that there are no approaches without issues, especially not in IT.

So, what are the problems that organisations have encountered when buying software products? And how big are these problems?

Software packages deliver features that are not needed

Companies that write and sell software packages face an interesting challenge. Usually each of the organisations they are selling to have somewhat different needs and processes. The overlap can be sizeable but even in the best case there will be features that are only required by a subset of the potential customers. At the same time it is normally too costly and complex for the software vendor to produce individual packages for each of their customers. Their business model is based on the fact that the software can be written once and sold many times. Responding to this challenge most vendors choose to implement all features that all of the potential customers may need.

A variation of this theme is software that is split into modules. Large ERP systems for example have modules such as payroll and logistics. In these cases, however, the same challenge repeats at the module level, not every company’s payroll requirements are the same. To make matters worse, the customer now has to choose, install and maintain several pieces of software.

In addition to responding to known needs of their customers the software vendors, especially in competitive markets, exert force on each other. They must implement ever more features to differentiate themselves from the competition. At this point features required across most of the customer base are considered tickets to the game, features that just have to be in the product to be considered. The differentiating features are often aimed at smaller segments of the audience or more speculative in nature, which normally leads to a very large feature set, and to users that only need a fraction of that feature set. An extreme case is Microsoft Word. A blog post on Microsoft’s developer site reports that out of hundreds of commands in Word “five commands account for around 32% of the total command use in Word 2003.” There are varying estimates for business and enterprise software but often the number of features used by a specific customer or user is estimated to be below 10% of the overall feature set.

When all the features needed by the users at a given customer are available one could consider this a success. The users have their requirements met and the software vendor has a model that works for them. Unfortunately, there is a problem: All those features that are present in the software but that are not needed by a given user add overhead for the customer and their users. First and foremost, the features are visible and at best may just take up space in menus, toolbars, etc but at worst may confuse the users or make use of required features more complicated. This often translates to lost productivity and increased training effort. The additional features are also likely to result in a higher resource usage, such as memory and storage space as well as network utilisation, which translates into higher capital expenditure for hardware and increased operational costs.

Software packages require customisation

One way to address the issues outlined in the previous section is to customise the software so that unneeded features are not visible or not even installed. This way, users are not burdened with these features and the features do not use additional resources. However, any customisation of packaged software brings with it the problems discussed in the context of developing software. Even if it is to a smaller degree, customisation brings risk in that unforeseen issues can cause delays or increase cost, they take time to carry out and test, thus increasing the time to introduce the package, and customisations require competencies that are not core to the business that is acquiring the package.

In most cases customisations are needed beyond removing unneeded features. Business, even those that operate in the same market, providing the same services to their customers, usually employ somewhat different internal processes and practices, and obviously the software package needs to support these. Some packages basically store data and make it accessible at a later stage. In such cases customisation is limited to adjusting the structure of the data and the screens to access it, which is normally relatively painless. More and more often, though, software packages actively support the workflow of an organisation and in such cases customisations quickly become more complex.

Adjusting workflows and processes in a software package exposes the customer of the software package to several equally unappealing options: They can change the actual source code of the package, thus effectively forking the codebase. Apart from the upgrade issues discussed further down, this brings buying software even closer to developing it in the first place, with all the issues discussed before.

Another option is to choose a software package that allows customisation through rules engines and workflow engines. The hope is that following a formal business process modelling exercise, producing a clear description of the business processes in a standard notation such as BPMN, the workflow engine can simply use that model without the need to change any code. In practice, however, it turns out that formally modelling the business processes is actually hard. In fact, many of the problems and risks associated with developing software are not caused by the actual programming but by getting the requirements right, and that is closely tied to understanding the business processes at a detailed level. Furthermore, capturing complex processes in a graphical notation adds its own set of problems. Even with sophisticated tooling navigating and manipulating large diagrams can be more cumbersome that working with a textual representation such as code. Features that facilitate collaboration between several modellers, such as version control involving merges and conflict resolution, are nowhere near as mature in workflow tools than in those tools available to manage source code.

A middle ground between source code and graphical notations are textual configurations and domain specific languages. In the end, they address some of the concerns, but still do not resolve the fundamental issue that any customisation adds risk and complexity.

Now the key question is: How much risk and complexity do customisations add? Unsurprisingly, there is no simple answer.

Read the rest of Part 2 and all of Part 3 on Erik's blog for more information on buying software.