Eric Weiss, The Worcester Group
Many small and mid-sized businesses have not invested in Business Intelligence systems because owner/manager perceptions are that it is too expensive and complex on the one hand, and not a true business necessity for their organization. In reality, costs of BI and database products have come down enough to fit into nearly any budget, but the returns from a properly designed initiative will pay for the investment many times over. The trick is to start, not with a product that impresses you, but by identifying the business need to be met first, and then making a technology decision to fill the need.
What is Business Intelligence?
“Business Intelligence” (or BI) is the current umbrella term for what used to be called “Decision Support” (DSS, for Decision Support Systems). The purpose in both cases was to make data culled from all around the business easily accessible in a form that supported analysis for decision making. Large quantities of transaction data need to be summarized and collapsed into a format that can be presented on a spreadsheet, from which a decision maker can quickly judge the best action to take from among several alternatives. The major difference between DSS and BI was that DSS tended to be self-contained, usually supported by a proprietary OLAP database, whereas BI usually works with a relational data warehouse that can be shared by multiple applications. Because of this, BI is also seen as scalable and extensible, which DSS was not.
BI is perceived as a technology, which is an understandable mistake. Technology vendors have hammered the marketplace with products that claim to be the Business Intelligence solution. While the products are impressive, they are not solutions. They are tools. Like any tool, the wrong one in the wrong hands does more harm than good.
Because of the perceived association between technology and Business Intelligence, most BI initiatives originate in corporate IT departments. While this is not necessarily a bad thing, it is a weaker starting point than a project that originates in a business function. IT departments promoting technology initiatives are often perceived (if unfairly) as being heavy handed in pushing disruptive technology onto the workplace. And make no mistake, the technology is disruptive. It induces changes in the way business is conducted that alter the roles and relationships of the people who must use it. And such change is often not welcome.
If the people who are expected to use the technology resent it and do not support it, a BI initiative will fail. Failure is expensive, because the investment will never be recovered. At least half of all BI initiatives fail. Those that are successful are the ones that achieved early buy-in and support from the user audience. How do you do that?
How to Begin
The key first step is to divorce the idea of Business Intelligence from technology. The fact is, every organization already has some form of Business Intelligence process. Information is pulled together from standard reports by individuals in functional departments in order to make daily management decisions. It’s done on Excel spreadsheets. Many people are probably doing the same thing at the same time, and not necessarily getting the same answers, so when decision makers need answers from two or more of these individuals, they spend a fair amount of time reconciling the numbers to make sure everyone is on the same page.
A well designed Business Intelligence system eliminates the duplication of effort by performing all the data collection and summarizing from which all analysis is developed. It includes a catalog of business definitions that everyone agrees on, so that if two analyses are based on inventory turnover, for example, they both start with the accepted definition and a common set of data. That is the foundation. The technology choice should not be made until after the foundation is built.
What is the most effective way to launch a BI initiative, to insure that it addresses an urgent need and gets the buy-in and support of everyone who should be involved in it? One word sums it up: irritation. Every manager in the organization, from the CEO on down, should take a couple of days to think about what has happened in the recent past to irritate him. What has disrupted routine, delayed activity, increased uncertainty and made your head or your stomach hurt? How much of this was due to a lack of information, or a breakdown in communication? What caused this condition?
Create a laundry list of issues and their causes, and assign priorities based on the severity of their impact on achieving departmental and company goals. Then compare your list with the other managers’. You will see exactly where the information pipeline is weakest, and that will tell you where a BI initiative should be directed.
This is a bootstrap approach. You can also bring in an outside consultant to facilitate the process. It is generally a good idea, but be careful in selecting a consultant. Most of the big consultancies have VAR agreements with software vendors, and while they may represent all vendors equally, they want to sell you software, because they make money on it. And they will try to influence your choice of software. It is best to try to avoid that and work with a consultant who is not a VAR.
The consultant’s job is to facilitate the process of self-examination required to identify the business need, then to guide the organization through the next steps of determining what the solution for that need must deliver to all levels of the organization. Who will be direct users? Who will be indirect consumers? What are the inputs? What are the outputs? What is the delivery method?
These questions are raised and answered in what are called Joint Application Development workshops (JAD). The purpose of JAD is to bring together subject matter experts from all parts of the company to discuss the need that has been identified, how it relates to their own business functions, and how the solution should work to meet their individual needs. The consultant then organizes the results of the workshop to create the system specification document, which all workshop participants should then review and sign off on. This is very important for securing buy-in and support of the project throughout the company.
From the specification document the consultant can then derive a project plan, spelling out exactly what steps will be completed in what order. Essentially, any plan should start with the building of the data warehouse or other data integration process. This includes the processes and procedures to cleanse data of errors, to reformat it for common access, and to validate it against its sources. This is a fairly intensive process, and it may be too much to tackle all the data at once. The consultant can break this down into modules based on business function. The JAD participants, who remain on call as project advisors, should determine which module should be implemented first. The development effort expended on the first module will subsequently be leveraged into faster turnaround for the remaining modules.
When the first data module has been created, it is time to call in the vendors. Invite all the vendors you like, but they all must be able to demonstrate their product using your data, whether from a data warehouse or a real time aggregation appliance. They should also be required to demonstrate that they can deliver the capabilities that were listed in your specification document. When vendors put on a canned demo, it is based on the most impressive-looking features of the product. But you and your people will be using the twenty percent of product features that are less impressive but necessary for eighty percent of your work. This is where you kick the tires. The vendor should be able to come in, set up the demo on your servers with your data, and let you work with it for a day or so.
All the JAD participants should have the opportunity to play with all the products being demonstrated. Then the whole group should spend a part of a day deciding which product to buy. Here, again, making this decision by consensus is crucial to the project’s overall success. The role of the consultant in this case is to coordinate the vendor demos and organize the IT resources required, and then to facilitate the selection of the tool of choice.
Once the tool has been selected, the consultant can help guide you through the implementation and rollout, including application-specific documentation.
How long should a project like this take? Realistically, from the day you launch, it will be about three months before you have a data repository build and populated. It will probably take another two weeks to install and evaluate all the vendor demos. Once you have selected a tool, you will spend some time on its deployment and on creating the first outputs from the system. A total of six months is likely for the first module to go live. After that, you should find subsequent extensions to planned modules take a matter of weeks, or even days.
How Much Does it Cost?
And the cost? That’s a little harder to pin down. Big name software products charge license fees based on the number of users or the number of processors that will run the software. Eighty thousand dollars is a rough guess. There are no-cost open source alternatives as well. Consulting costs are likely to run around $100,000 and this is non-repeating. In fact, the consultant should insure that knowledge transfer is sufficiently comprehensive that your own IT people can manage all future extensions of the system. The internal costs of your managers and subject matter experts should probably not be included in the calculation, since that expense is there anyway.
But if you did the ground work, you already know what you expect to recover as well. You are doing this, after all, to address a problem, and you should have assigned priorities so that the problems addressed first are the ones that pay back the most.
Consider the case of one company that had both wholesale and retail businesses. Collecting and aggregating retail data for monthly buying decisions was done mostly with spreadsheets and took weeks of late night sessions. The company built a data warehouse and developed its own end user tool, eliminated the late nights and was able to make buying decisions daily, with better accuracy and cost-effectiveness.
The wholesale area had a problem with orders being returned by customers who had no room to receive shipments. The warehouse was overflowing with returned merchandise, and there were mounting reverse logistics charges and customer charge-backs. It was a multimillion dollar headache. The problem was traced to a breakdown in communication between Sales and Customer Service. Hundreds of messages were exchanged each day by phone and by email regarding customer requests not to ship or when to resume shipments, and many of them were just getting lost. The solution was to use the company’s BI system to collect and display messages between Sales and Customer Service. Anyone authorized to deal with a customer order could see the entire communications history since the order was placed. Furthermore, the application tracked and color-coded every shipment from the time it left the factory to the time it was delivered to the customer. Delays in factory shipments were highlighted so that Sales could immediately inform the customer. Shipments that were being held in the warehouse were visually identified and could be excluded from scheduling until released.
Within a month of going live, the returns and charge-backs had been eliminated, the inventory backlog had been reduced, and Customer Service overall workload had been reduced by almost 20%.
Eric Weiss has been a business intelligence consultant for over twenty five years, providing service to well-known brands in a wide range of industries. The Worcester Group, Inc., is a Business Intelligence Systems consulting firm, established in 1995.