The Business Benefits of Measuring Return on Investment for Business Intelligence Implementations

1

The Business Benefits of Measuring Return on Investment for Business Intelligence Implementations Prepared by: Elliot King, Analyst, Unisphere Research

Abstract Although measuring return on investment is a key information technology (IT) management discipline, many enterprises do not regularly conduct return on investment (ROI ) studies, particularly for business intelligence (BI) implementations. Respondents to a series of research studies exploring ROI indicated that both costs and benefits are often difficult to measure, particularly for BI. However, if companies take the time to truly evaluate their ROI after their BI applications are implemented and focus on certain key measures, many of the obstacles that appear to be challenges for conducting appropriate ROI analysis can be overcome. The companies that have taken on effective ROI studies uncover significant tangible and intangible business benefits and launch an important cycle through which they can realize the full value of their BI investment.

Introduction Measuring the return on investment for technology investments is generally seen as a critically important tool in effective IT management yet not all deployments are easily quantifiable. The logic behind conducting ROI studies is obvious. Faced with an accelerating rate of technological change, companies must constantly invest to upgrade and improve their information technology assets. As data is increasingly understood to be a company's most valuable asset, the stakes for technology investments in data management and reporting have never been higher. Prudent, cost-effective, and appropriate investment in information technology can give a company an important competitive advantage. Spending on failed projects results in huge cost overruns, performance problems, and a general failure to live up to expectations which seriously wounds an IT organization's credibility and long term effectiveness. The basic formula for calculating ROI is simple enough. Calculate the overall cost of specific IT projects and compare that to either how much money a company saves or the incremental revenue it generates over a period of time. The difference is the ROI-the higher the return, the better the investment. But ROI studies are harder to implement than the basic formula suggests. Calculating the direct and indirect costs of technology projects, either as reduced costs or increased revenue, can also be complex to measure. The difficulty in calculating ROI for IT projects is the single most significant reason that these studies are frequently tossed aside. In fact, a recent survey of 415 IT professionals conducted by Unisphere Research in conjunction with Noetix Corp., called "Key Metrics for Measuring ROI for Business Intelligence Implementations," found that only a little more than one quarter of all respondents conducted ROI studies for significant IT projects. Moreover, almost 11 percent never conducted ROI studies at all within their organizations.

The Business Benefits of Measuring Return on Investment for Business Intelligence Implementations

Noetix - 2008

2

Figure 1: How frequently does your company conduct ROI analysis for significant IT projects? Never 10.91%

From time to time 28.43%

For every project 26.9%

For selected projects 33.76%

Since business intelligence is generally deployed in multiple departments across large enterprises, it could be expected that BI initiatives would be subjected to sophisticated fiscal management similar to other large IT projects. Interestingly, this is not the case. In the Unisphere Research survey, more than 30 percent of the respondents reported that they did not perform a cost justification before investing in BI. Nearly 55 percent said that they've never calculated the return on investment for their business intelligence implementations. And 77 percent said that they have never calculated their total cost of ownership for the BI implementations. This statistic seems staggering as BI initiatives are seen today as critical components of an organization's overall business strategy (see note below).

Figure 2: Have You Conducted Cost Justification or ROI Studies for Your BI Implementations? Analysis Conducted for BI Implementation

Cost Justification Prior to Investment

ROI after Implementation

Total Cost of Ownership

Yes

68 percent

45 percent

23 percent

No

32 percent

55 percent

77 percent

Companies do not perform ROI studies for their BI implementations for several reasons. Nearly 60 percent of the survey respondents said that the most significant challenge to conducting ROI is that real costs and specific benefits are hard to quantify. Assessing the overall impact of an IT investment on corporate performance is also hard. Calculating ROI can be complicated and can take considerable time, which of course always seems to be in short supply.

The Business Benefits of Measuring Return on Investment for Business Intelligence Implementations

Noetix - 2008

3

Figure 3: What are the most significant challenges to developing a meaningful ROI analysis? (Multiple responses allowed)

Real costs are hard to estimate

59.94%

Specific benefits are hard to quantify

57.49%

Impact on overall corporate performance is hard to determine

47.09%

Process is too complicated

35.47%

Process takes too long

30.89%

No single person is responsible for calculating ROI

27.52%

There is no time to conduct post hoc ROI calculations 24.77% Calculating ROI has no real value

3.67% 0

20

40

60

80

100

Finally, in many organizations, it is not clear who is even responsible for conducting ROI studies. Approximately 30 percent of survey respondents said that ROI studies were conducted by a team with participants from both the IT staff and the business side of the organization. History has shown that it is extremely difficult to assemble, manage, and coordinate joint IT-business teams. Moreover, a follow-up study conducted by Unisphere Research in conjunction with Noetix Corp., titled "Challenges and Alternatives in Measuring ROI and TCO for BI Implementations," found that most respondents felt that IT staff personnel were not trained well enough to conduct ROI analysis, while the financial staff did not have the right tools to measure ROI for BI implementations.

FiguFigure 4: How many people regularly access your business intelligence applications?

500 or more 17.46%

100-499 20%

0-9 13.97%

10-49 29.21%

50-99 19.37% Charts do not add up to 100 percent due to rounding.

Despite the significant hurdles involved in conducting ROI studies, companies still believe in their inherent value. Less than five percent of survey respondents felt that calculating ROI had no real value.

The Business Benefits of Measuring Return on Investment for Business Intelligence Implementations

Noetix - 2008

4

Business Benefits Clearly, to date, the difficulty and complexity in conducting meaningful ROI studies for BI implementations has discouraged many enterprises. Indeed, in the face of other pressing priorities, why bother investing precious resources to calculate the return on investment when the project is already completed? The reason is simple. Conducting ROI studies for business intelligence delivers many business benefits. Nearly 90 percent of the respondents said that effective ROI analysis would enable them to invest more in business intelligence and therefore support improved decision making and overall corporate performance. Close to 85 percent said that knowing the ROI for their technology investments would make it easier to justify purchasing additional hardware and software. And 75 percent of people surveyed said that it would be easier to forecast technology needs if they knew the ROI for their current BI implementations. Moreover, if ROI studies can be used to justify additional investment in business intelligence, companies can launch what can be seen as an important cycle within their overall BI strategy. Most companies are not realizing the full value of their investment in business intelligence. Indeed, 71 percent of the study respondents said that they believed most organizations do not make good use of their BI applications. The majority said that the key to getting more value from BI is enabling end users to create more reports on their own, and the greatest impediments to getting more value from BI is a lack of training and limited staff resources. Clearly those obstacles can be overcome and more easily achieved if the return on the initial investment is better defined from the start of the BI initiative.

Determining Key Metrics The primary reason companies do not commonly conduct ROI studies for their BI implementations, even though they recognize the value in doing so, is that it is difficult to calculate the real costs and benefits. The first step is to identify which costs and benefits should be assessed and how to measure them. A BI implementation involves three different types of costs-direct costs, indirect costs, and training costs. According to the survey, the most significant direct costs include the initial cost of the software licenses; the initial cost of any new servers required to deploy the solution; and the initial cost of the consulting services to successfully implement the application. The most important indirect costs to calculate are the costs for upgrading existing software and servers. Finally, the most significant training costs to include in ROI calculations are the costs to train the actual end users. The survey revealed that respondents felt the direct costs were generally more important to calculate than the indirect costs. The cost of training end users was seen as more significant than the impact on existing servers and network traffic. Calculating the cost of training the IT staff was seen as less important than measuring the top indirect costs of the project. In fact, survey respondents indicated that the greatest difficulty in calculating indirect costs is determining which indirect costs to assign to the BI implementation and how to measure them.

The Business Benefits of Measuring Return on Investment for Business Intelligence Implementations

Noetix - 2008

5

Figure 5: Ranking of Costs Associated with BI Implementations

Initial cost of software licenses

4.14

Initial cost of new servers

4.03

Initial cost of consulting services for implementation

3.94

Upgrading related software

3.88

Time for training end users

3.73

Impact on existing servers

3.69

Impact on network traffic

3.67

Time for training IT staff

3.51 0

1

2

(5 is most important)

3

4

5

Cost is only one variable in the BI ROI equation. Calculating benefits is equally necessary. In practice, different stakeholders derive different benefits from BI implementations. That said, there is significant overlap in the key benefits for business staff and senior management. In both cases, access to data for decision-making is seen as a key benefit that needs to be measured. Determining the time spent on analysis is a more important calculation for senior management than for staff. Increased use of business intelligence by senior management and the ability for staff to create meaningful reports are also key benefits. The picture is different for IT personnel. The key measurements include the time it takes for IT staff to develop reports; the time spent supporting end users; and the number of reports requested from IT staff.

Figure 6: Top Benefits of BI Implementations for Different User Communities Community

Senior Management

Business Staff

IT Staff

Top Benefit

Access to data for decision making

Access to data for decision making

Time to create reports

Second Benefit

Time spent on analysis

Number of reports users can create

Staff time responding to user requests

Third Benefit

Increased use of BI

Time spent on analysis

Number of requests to IT to create reports

In general, the most important benefit that BI implementations deliver is higher quality information for decision making, both for senior management and front-line business personnel. The bottom line impact of decisions and the speed at which decisions are made are also important benefits but less important than access to higher quality information.

The Business Benefits of Measuring Return on Investment for Business Intelligence Implementations

Noetix - 2008

6

A Strategy for Conducting ROI Analysis To conduct an effective ROI analysis, enterprises should go beyond projecting their costs and benefits in the initial planning process and analyze the impact of their BI implementation at a specific interval following the "go live" date. At that point, direct costs and training costs should be evident and relatively straightforward to calculate, although indirect cost estimates may still require additional analysis. By conducting ROI on a post-hoc basis, key benefits can be more easily documented and calculated. The easiest benefits to measure are: Number of reports end users create Time it takes IT staff to create reports

Getting Started on ROI Studies Although ROI studies are an element of sound fiscal management, in essence, they are a type of evaluation research as well. Fundamentally, the goal of an ROI study is to determine if a project achieved its specific aims. The cost/benefit analysis associated with ROI is primarily a way to quantify the answer. Conceptualizing ROI studies as evaluation research has two implications. First, the methodology used should allow for ongoing, periodic assessment during the lifecycle of the implementation. Secondly, the benefits generated by the project should be understood within the context of the enterprise's overall goals. Useful ROI studies can be completed using the following six steps. Step One:

Establish the overall goals of the project. What problems do you want to solve and what advantages do you want to gain by implementing BI technology? These questions have to be answered for all the stakeholders in a BI project-senior management, analysts, front line personnel, and IT personnel.

Step Two:

Determine the status quo. Developing a baseline is a key step to all evaluation research. How are you currently using BI technology? How many reports are you generating? How long does it take to create a new report, etc? Assign a value to each of the activities associated with the use of BI technology. Many of these values can be approximated through an analysis of personnel costs to complete each task.

Step Three:

Calculate the costs of implementing BI technology. The costs include hardware, software licenses, maintenance, training, and support.

Step Four:

Estimate the probable return on investment. Based on the goals, assigned value of BI activities, and anticipated costs, project the possible ROI for the implementation.

Step Five:

Monitor activity. After the project goes live, survey each stakeholder community periodically to determine changes from the baseline and if the usage goals are being met. If usage goals are not being achieved, identify why and take corrective action.

Step Six:

Determine actual ROI. The specific dollar value either in revenue generated or in costs saved is often less important than the knowledge that the goals of the implementation are being met or not.

IT staff time required to support end users Number of requests to IT to create reports Increased use of BI tools by senior management

While these are not the top benefits associated with BI implementations, they are very significant and tangible calculations. According to the study, despite a concerted effort to drive analytical tools and business intelligence functionality to a wider community, approximately half of the respondents stated that the majority of the BI reports are still produced by IT staff, significantly increasing the costs to support end users. Therefore, measuring the impact of a BI implementation on IT staff time can produce enough of a benefit to better understand the real return on investment. Moreover, when explored after the fact, other significant benefits become much easier to measure. When asked what would make it easier to calculate ROI, survey respondents listed five priorities. They would like to be able to document: Increased use of BI tools in the organization Time saved in developing custom reports Fewer IT resources consumed when creating BI reports Time saved discovering data Reduction in training time for end users Looked at retrospectively, many of those benefits are not extraordinarily difficult to determine.

Approached in this way, virtually all enterprises can gain insight into the ROI for their BI implementations. More importantly, they will know if their overall goals are being met and be able to make the necessary adjustments if needed.

The Business Benefits of Measuring Return on Investment for Business Intelligence Implementations

Noetix - 2008

7

Time Frame for Analysis Identifying the appropriate costs and benefits is essential for conducting a meaningful ROI study. However, the time frame in which the study is conducted and the proposed period for which the return on investment should be reported is also critical. The follow up study, "Challenges and Alternatives in Measuring ROI and TCO for BI Implementations," in which 280 IT professionals were surveyed, showed that 54 percent felt ROI should be calculated in the initial planning process for a BI implementation. Only 4 percent felt that ROI should be calculated three years after an implementation has gone live. At the same time, nearly half of the respondents felt that the measurable benefits of a BI investment should exceed the costs within three years. In other words, the ROI should turn positive after three years. Fifteen percent felt that the measurable benefits should exceed costs within five years and nine percent felt that the measurable benefits should exceed costs within one year.

Conclusion Conducting meaningful ROI studies is not a trivial task. Additionally, when faced with the day-to-day pressures of operational pain points, bypassing the ROI process can greatly expedite the delivery of a potentially beneficial solution. However, companies that do conduct ROI studies can realize significant business benefits and can gear their investments to those technologies that will have the largest positive impact on their operations and ultimately, on their corporate performance. A certain level of analysis is always better than none, particularly if it can assist in more accurately directing valuable investment resources and avoiding sub-optimized expenditures.

Corporate Headquarters 5010 148th Ave. NE, Suite 100 Redmond, WA 98052-5119 USA Tel 425-372-2699 Toll-free 866-4NOETIX Fax 866-436-0406 www.noetix.com

The Business Benefits of Measuring Return on Investment for Business Intelligence Implementations

Noetix - 2008