Random Curry

A new blog posting from your humble narrator…

A Plea to the Engineers

Successful Software Projects are Miraculous

It occurs to me that if you can hit a baseball successfully 1 time in 4 and can play the field effectively, you can maintain a career as a major league baseball player (I’m talking to you, Ryan Howard). “Success” depends entirely on what criteria you apply to the results.

Your thoughts?

We complete our journey by discussing the effect of metadata on project and program management…

The Metadata ROI Series: Episode 4: Program Management

Ever grateful for the opportunity provided by my company to write, I give you the latest installment…

Process is Half the Story

My latest contribution to the Baseline Consulting blogs…

The Project Manager as News Reporter

I’d love to know where they get those pictures…

Enjoy!

A great post from Nick Malik today…

Understanding the root causes of poor software requirements

This is a pretty long post, but it illustrates that this issue is bigger than pretty much any breadbox you can think of. But, it’s also illustrative of how important this issue is to the overall craft of software development.

Incidentally, I may take a whack at discussing the assumption…

Assumption: Improving the quality of software requirements will have a net positive effect on the quality, reliability, applicability, usability, and value of custom software as perceived by the business users who use it.

In Part 2 of this article, we examined the requirements for introducing Actuarial concepts into IT project management. We conclude by describing how you might go about implementing these concepts.

——–

The Solution

It is clear that better measurement of costs associated with large IT initiatives is needed to manage these projects more effectively. It is also clear that there is little incentive to capture the data necessary to perform the statistical analysis required for effective risk management on an operational level. This contradiction contributes to information technology’s poor reputation on delivering solutions that satisfy business requirements.

Can this situation be remedied? I believe so, but it will take some new technology and a different perspective from both the people doing the work and the people managing it. The new technology is being developed right now – a great example is the ability of the Tivo corporation to “predict” the outcome of the weekly voting on the “American Idol” television show by recording the usage patterns of their subscribers. Capture techniques such as this will be necessary to collect the data necessary for detailed risk assessments and usage patterns of spreadmarts within an enterprise.

Management personnel will need to shift risk management topics to the forefront of their philosophies and remove risk management from an anecdotal focus to one of metric analysis. Part of this shift will include contacting and retaining actuarial skills for statistical assessment or risk parameters, reducing the uncertainty in the current level of analysis. This will make decisions more effective and save the organization resources and money.

There are examples of corporate executives requiring metrics-based analysis in support of decisions, as described in Jessica Tsai’s article on predictive analytics. Unfortunately, these examples are rare enough to be called out in vendor presentations, which is a poor commentary on the presence of these measures. The presence of an actuarial staff would improve this situation greatly.

Operational personnel will need to understand their role in reducing overall risk to the organization. The easiest way to do this is to demonstrate the cost of activities such as spreadmarts and place appropriate sanctions in place for continued spreadmart use. Often, the easiest way to correct behavior is to associate the activity with a tangible financial cost. Once the habit is broken, it is unlikely to reassert itself in the organization, finally realizing the cost savings desired.

Organizational change is rarely quick or easy, but the movement toward a better way of analyzing costs will bear many benefits over the long term for enterprises looking for more effective information technology management techniques.

In Part 1 of this article, we explored the issues surrounding risk management in IT organizations. Part 2 of the article describes the role of the Actuary in business and how this concept could apply to issues facing IT.

—–

The Actuary
The dictionary defines an actuary as “someone versed in the collection and interpretation of numerical data (especially someone who uses statistics to calculate insurance premiums).” The
actuarial function is very important to the insurance industry since the accurate prediction of risk allows the accurate calculation of premium rates and loss exposure, which demonstrates the soundness of the business. Actuaries have access to decades of “experience” data to base their calculations, and a well-established certification process to ensure competence.

The application of this skill set would pay great dividends to an IT department that is interested in quantifying the risk associated with the continuance of “unapproved” behavior of business uers, such as shadow data analysis systems. For example, the statistical analysis of hours spent by business staff on gathering data for these systems and the reconciliation of numbers produced by similar systems could predict the financial effect of the continuation of the practice. From this data, decision-makers can weigh the costs of lost productivity with the benefits of the more local control and flexibility that these systems provide and make an intelligent decision on the continuance of the practice, as opposed to today’s practice of anecdotal decisions.

The Requirements
Why haven’t more enterprises embraced this sort of detailed analysis in their IT departments? Why have initiatives such as Enterprise Risk Management failed to gain much traction outside of regulatory compliance? The answer lies in data itself. Actuaries have decades of experiential information at their disposal to do their jobs, mainly collected by public agencies such as the United States Census, hospital records, accident records, and the like. This data is collected and made available as part of the normal operation of organizations that are external to the actuarial team in the enterprise, so the business incurs little if any cost in obtaining the data.

Business users and IT departments do not generally track staff utilization to the point where it would be useful for statistical analysis, and they would need to establish policies and infrastructure to collect the data within the organization. The cost of obtaining this infrastructure and training staff in its use would need to be added to any project plan implementing a risk management program. The fact that the cost would be borne entirely by the enterprise is a strong disincentive to undertake such an effort.

There is a more significant hurdle to cross than cost, however – the definition of the costs themselves. How difficult would it be to collect this data? Here are some examples:

• When a business user creates a “spreadmart*,” how much time is devoted to obtaining
  and reconciling data, and how much time is devoted to the analysis itself?

• What is the cost of retrieving quality data in a spreadmart and in the enterprise data
  warehouse?

• What is the cost in time and materials for a local hard drive crash where spreadmart data
  is lost?

• What is the reconciliation cost of data calculated by formulas that deviate from the
  corporate standard in a spreadmart?

• What is the potential cost to the enterprise if a spreadmart metric is incorrect?

Very few, if any, enterprises collect data on staff utilization and costs at this level of detail, mainly because there are few automated methods for data collection, and the testimony of staff members in status reports or time sheets is fairly unreliable for statistical purposes. Yet, it is precisely this level of detail that is required to perform the analysis necessary to quantify risk.

Another requirement for actuarial analysis is scholarship regarding risk management standards in
the IT profession. Here, insurance actuaries have a huge advantage due to the relative age of the professions: information technology has been practiced for just over 50 years, where insurance has been practiced for hundreds of years, and entire university curricula is devoted to the subject.

However, this should not be a great impediment to the adoption of actuarial philosophy to information technology, provided that IT is viewed as simply another business process. The statistical measures of risk should be similar, although the specific situations may be different. The analytical techniques should be similar enough to encourage adoption.

—–

In the final installment of this article, we will bring the issues and definitions together to propose a potential solution to risk management issues in IT organizations.

* The term “Spreadmart” was coined by Wayne Eckerson, Director of Research for TDWI, to
describe the primary implementation of a shadow data analysis system as a spreadsheet with
data obtained from enterprise systems that functions as a data mart.