Measuring the ROI of Enterprise Search – Making the Case with Numbers

Far too often, the Enterprise Search experience is woefully inadequate and falls noticeably short of the carefully tuned and nurtured experience that internet users have come to expect, courtesy of the heated contest between Google, Bing and Yahoo. One factor that’s not helping to close this innovation lag is the complexity involved in building the clean-cut, formal ROI case that often drives project charters within the Enterprise.

NOTE: recognizing that “Enterprise Search” can include a vast spectrum of content sources and content types, each with unique characteristics and limitations, we will focus the discussion here on web-centric content that’s typically served through standard HTML, in both internal and external scenarios.

So how exactly do you build the business case for overhauling your search implementation? A leading Canadian IT consulting firm, NLC, recently tackled the question head on in their ‘Building the Business Case for Enterprise Search’ whitepaper [1] last week. While maintaining that a formal analysis ‘misses’ the real value of Enterprise Search, the paper does go on to provide a formal approach based on the IDC report, The Hidden Costs of Information Work (May 2009).

The proposed formula boils down to measuring:

• Search Cost Today = # of knowledge workers X avg. salary (incl. benefits) X % of time spent searching
• Potential Annual Benefit = Search Cost Today X The Search Lift Ratio (%)

Once deployed, estimates around ‘The Search Lift Ratio’ can easily be quantified through a combination of A/B testing, measurable and correlated call-to-actions (phone numbers present a challenge here, in some cases), ‘accurate’ time-on-page measurements [5] and traditional survey/feedback polls [4].

While the guidance is sound, it is largely based on estimates and, like similar reports before it [6], focuses primarily on the productivity gain; it does, however, offer a compelling, low-cost approach to building the case without relying on metrics that may be available in other areas. 

The “saves you money” argument, by all accounts, is wildly overshadowed by the broader “makes you money” advantages [2], yes; but, recognizing that corporate culture thrives on the the power of hard numbers, can we do  better upfront in the ‘reasonably measurable’ areas by relying on other metrics?

Assuming that the business has realized the value of matured analytics, even around legacy search implementations, can we explore these to make a stronger case?

While some of these are domain-specific, here are other areas that may yield useful ROI nuggets:

• Reduced Database Volume => License/Hardware Savings: for high-volume applications that rely on a traditional RDBMS alone for ad-hoc queries or search-based functionality, a well-designed search implementation can significantly reduce the load on those database servers, resulting in both hardware and licensing savings. Through standard profiling tools, the volume of traffic that stands be offloaded to a search infrastructure can be quantified beforehand. Similar hardware/licensing savings (and training/staffing can also be realized in consolidating a fragmented search environment.
• Measuring Publication Reach => Minimizing Sunk Costs: in scenarios where content was specifically authored and published for an intended audience, you should be able to measure the ‘publication cost’ and compare this to the ‘reach’ of the content (again, based on call-to-actions or accurate ‘time on page’ measurements). In scenarios where the reach is below a certain threshold, there is a clear and measureable ‘sunk cost’ that can be avoided in the future.
• Reducing Call Center Volume => Headcount Reductions: assuming that source of call-center calls (both internal and external) are clearly classified (e.g. “I can’t find any information on…”), it should be possible to attribute a portion of call-center volume (by extension, head counts) to inadequate search implementations.

Can you think of others that affect your organization?

References:

[1] – Building the Business Case for Enterprise Search
http://www.nonlinearcreations.com/whitepaper/index.asp?id=23

[2] – In Search of ROI | Search Done Right
http://searchdoneright.com/2008/08/in-search-of-roi/

[3] – How search can add real ROI to eCommerce websites
http://googleenterprise.blogspot.com/2010/01/how-search-can-add-real-roi-to.html

[4] – KISS Insights - It’s so quick to respond that many of your customers will!
http://www.kissinsights.com/tour

[5] – The Problem With Measuring Time Spent on Page
http://blog.webanalyticsdemystified.com/weblog/2007/07/on-netratings-and-time-spent-on-site.html
http://www.marketingpilgrim.com/2007/07/the-problem-with-measuring-time-spent-on-a-web-site.html

[6] – Return on Information:  adding to your ROI with Google Enterprise Search
http://static.googleusercontent.com/external_content/untrusted_dlcp/www.google.com/en//enterprise/pdf/internal_search_roi.pdf