The Excel Trap

If your only tool is a hammer, you tend to see every problem as a nail. – Abraham H. Maslow

The pressure on the team is growing every day. The introduction of the process area “Requirements Management” was defined as one of the measures during the process optimization. The entire CMMI project is firmly planned and must be finished by the end of this quarter. Previously, the organization was examined; efficiency gaps were found and systematically analyzed. The resulting work packages were defined and planned. All tasks were budgeted: the CMMI consultants, the additional project members, and the expert working groups within the customer’s organization. The final version of the process model was recently delivered, reviewed, and released. All requirements of the 2nd CMMI Maturity Level were solved cleanly, most of all the traceability. In the process area of Requirements Management, the CMMI clearly demands a well-defined and maintained traceability between different work results. The procedure was arranged straightforwardly: when the requirements manager receives a new feature, he registers it in a prepared Excel sheet and passes the requirements on to project planners, system analysts, and architects. They report the updated plan data and component IDs back to the requirement manager, who registers them in the appropriate columns of the Excel sheet and updates all relevant requirement statuses. The Excel sheet is placed on a shared drive, where everyone can see the current status of the requirements. Only a few users have write access, e.g., the requirements manager.

„Tools are unimportant; only the process counts. “

How beautiful is this simple solution? Since an ordinary Excel sheet was used, expensive tools and all efforts associated with their introduction could be spared. “It is not important what tools you use; it’s only the process that counts,” says the chief consultant. And he surely knows what he’s talking about. How this concept works in practice in a pilot project is to be examined. The pilot project is under full steam, just like all other teams in this organization. The pilot project is well on its way, and the first phase of the requirement analysis is just taking off.  Two hundred and thirty new requirements flow into the Excel sheet. They are systematically analyzed, estimated, and planned. The requirements manager has plenty to do; each day he processes all the data that pours in via email from his “customers:” Comments on the requirements, status changes, plan data, and system component IDs from the Impact Analysis. The smart-looking bar and pie charts look magnificent. Everything makes a very correct and professional impression. The pilot project testing the process area “Requirements Management” picks up the pace. The workload of all team members reaches its natural limit. In the rush of everyday project work, it turns out that some requirements were forgotten. These must catch up really quickly. In addition, some requirements were recognized as doublets and must be split. During this activity, it frequently turns out that the impact analysis of the alleged doublets was based on different assumptions. Thus, different results were collected. These requirements still need to be divided, but they must remain interconnected. This is quickly fixed by using predefined ID ranges.

„Why don’t they just let us do our work? Things were so easy before… “

Soon it becomes clear that the Excel sheet is not error-free. The requirements manager must now handle 350 entries and 35 columns. That makes 12250 single entries. To err is human, but the process does not allow it. It shows that simple self-control is insufficient to ensure the correctness of the quickly growing amount of requirements data. More and more resources are busy with the quality assurance of the various versions of the Excel sheets. Weekly reviews, complex checking scripts, smart macros, and many formulas are used to stabilize the concept. Still, the number of inconsistencies is growing. The requirements manager must simultaneously play the roles of a quality assurance person, a script developer, and an Excel expert. He is increasingly working overtime. It seems that the requirements manager has failed and the resource planning was incorrect. It becomes obvious that at least two requirement managers are needed to handle the workload. The management is now under increasing pressure to make this sub-project successful. Another requirement manager is hired. From now on, two persons are working simultaneously on a single Excel sheet. What happens now is what was foreseeable: Discrepancies, ad hoc parallel activities, and mutually overwriting changes become increasingly frequent. Excel sheets are not made to be multi-user friendly. The errors are corrected through even more reviews and expert questioning. The project workers begin to mutter: “Why don’t you just let us do our jobs? We are drowning in this bureaucracy. It was all so easy and straightforward before … “ The story – taken from the everyday IT project life – moves down its predestined track. The project planners, architects, analysts, testers, and customers are increasingly fed up with frequent questioning and lengthy interviews. Frustration and the chaos in the requirements management are reaching staggering levels. The CMMI project is finally put on hold; the consultants are sent home. Without functioning requirements management, the CMMI maturity level remains unreachable. The optimization project has ultimately failed.

Tools cost money. Missing tools cost a fortune.

Excel sheets are sufficient for provisional solutions or one-time actions. They are a terrible idea as a sole tool for systematic professional requirements management. The constant pressure to reduce costs and the widespread belief that Excel would be a solution for complex management tasks is, in fact, a frequent reason for the failure of many process improvement projects. In the discipline of requirements management, more than a simple spreadsheet analysis is needed. Many things must be available, such as a configurable state machine for the workflow, access rights concept, multi-user ability, a baselining function, a minimum of ergonomics, integrated consistency checking mechanisms, etc. Interestingly, nobody would use a simple drawing program instead of a CASE Tool in software design, even if the drawing tool is much cheaper and produces really nice pictures. In other process areas, however, this is frequently the case. Using cute spreadsheet templates, projects are planned, risks, defects, and requirements are managed, large numbers of test cases are written, etc. In most cases, at the beginning of a project, the spreadsheet-driven solution seems to be a perfectly right choice. Eventually, it ends up in more or less complete chaos. It hits those organizations the hardest that manage their entire process models with text processing and spreadsheet applications. Such artifacts are often called “write-only documentation.” A “write-only“ process definition is pointless and even dangerous since it creates the deceitful impression that the organization is well-managed and stable. The money that was saved on the tool is eventually spent on extensive manual data administration and intense attempts to obtain the necessary process acceptance within the staff. However, process models that are not supported by proper tools and instead include repressions and penalties for making mistakes will hardly find any acceptance.

Tools must be introduced, deployed, and integrated.

Various tools are needed for different tasks: Project planning, requirements management, architecture and design, quality assurance, etc. In large-scale projects, dozens of varying expert tools are used. In order to ensure the correct data exchange between these tools, e.g., consistent traceability between requirements and the project plan, those tools need to be integrated appropriately. Otherwise, the data exchange quickly becomes error-prone and expensive manual work. This tool integration is a complex venture; each tool must potentially interact with all other tools, the systems have incompatible data interfaces, and various needs in every process area must be carefully considered and implemented. It is essential to realize that without reasonably functioning integrated tooling, no organization can really properly function. Important is also the insight that a tool introduction costs a considerable amount of money. An introduction of new tools is in fact usually just another regular (sub) project. Resources and time must be paid, organized, and planned systematically. However, ROI (Return on Investment) analysis is difficult in this case. It turns out that an optimization project without proper tools shows a substantially higher probability of a total loss of the investment if the tooling question is ignored.

Summary and advice

We briefly summarize our observations below:

• Without suitable tools, process optimization projects will almost certainly fail.
• Proper tools significantly increase efficiency and are vital for the new process acceptance issue within the organization.
• Tools rarely work “out of the box.” They must be customized, scripted, and wisely introduced.
• Tools must be integrated so that efficient data exchange is possible.
• Properly configured tools are a better way to assure process conformity in the organization than process definitions, punishment threats, and frequent audits. Process conformity assurance that is based on any kind of “penalty clause” will not work.
• All tools must fulfill CMMI requirements, particularly regarding quality assurance and configuration management.
• Excel is no substitution for specialized tools.

Reaching any CMMI maturity level requires a good tooling concept. There is no alternative to this rule. Therefore, we recommend the following:

• Distrust anyone who tells you that tools are irrelevant and that the good process model will fix all problems instead.
• Define a sane budget for the tooling sub-project. Unfortunately, it’s a common mistake to underestimate the complexity of this task. If you do not have the money for your tooling, it’s actually better not to start the process optimization project at all and save all the vain expenses.
• Choose a tool specialist team that is also familiar with CMMI or is at least led by an experienced CMMI consultant.
• Generally, I distrust all Excel-based “solutions.”
• Choose tools that can be adapted to all of your processes. A team should customize at (hourly) rates agreed upon before the project starts.
• While reviewing your new process model, demand explicit references to tools and information on how those tools are integrated. At least on a low (detailed) process level, this must be sufficiently described.
• Listen carefully to your team. If the new process is said to be “too complex,” then you probably have an urgent tooling issue.

We have omitted a number of further details, as it would overextend the given space in this short article. Besides, all of our recommendations must be further refined: Each project has its own specific aspects with different tools that are used differently. However, the right tools are the core prerequisite for any successful CMMI project. It is essential not to ignore it and to believe the puristic slogans that suggest the “right” process definition would save you from having to choose, customize, integrate and deploy the proper tooling.

---

Let’s start a conversation on LinkedIn or X.com (formerly Twitter).