We’ve covered a lot of topics in this series, and they are all meant to serve as reminders of important things those of us who work in business intelligence and data warehousing tend to forget as we get busy with our day-to-day work. The final recommendation we’ll make may not be particularly exciting, but it’s hugely important: make time for regular check-ups of your EDW/BI solution or over time the overall health and user adoption will suffer. Call it refactoring, call it continuous improvement–whatever you call it, you have to invest in long-term health if you want to stay competitive with the next product somebody comes back from a conference wanting to buy.
The “regular check-ups” we’ll address fall into 3 categories: performance, usability, and approach.
Let’s start with performance. When you start out with brand new hardware and tools, not (yet) a lot of data, and not (yet) many users, it doesn’t take genius architecture and tuning skills to have a high-performing data warehouse that delights end users with sub-second query response time. And it doesn’t make a lot of sense to toil over database optimization when things are performing well. However as the user base and volume of data grows, a neglected system will become bogged down over time. It should be a practice of the EDW/BI team to monitor performance of the data warehouse and regularly set aside time to address whatever is the most pressing bottleneck. Staying ahead of emerging issues will ensure the system continues to perform well. I’m convinced that too often we jump to new hardware or hugely expensive data warehouse appliances to overcome performance issues that could have been mitigated through consistent, targeted tuning efforts.
The next category of check-ups you need to plan for are usability check-ups. Similar to our earlier post about marketing, this one involves getting out of your cubicle and taking field trips to understand how your users are utilizing your EDW/BI offerings. Along with monitoring usage via automated tools, this should give you ample ideas on what existing things need improvement and where you should focus your new development efforts. We all like the idea of running our department like an independent business in concept, but too often in practice we lose sight of the customer and focus on keeping the machines running as if our purpose is just that: keeping the machines running. But if there’s only one concept you take away from this blog series I beg you to make it this: the tools, platform, data, and even our very jobs exist for our end users, and in healthcare, ultimately for our patients! So we must be proactive in exceeding the expectations of our end users and not becoming complacent with simply keeping the machines running.
The last category of check-ups we’ll address is revisiting your standards, guiding principles, and overall approach to developing and maintaining your BI/EDW solution. It’s extremely important early on in the process to develop your philosophy of development along with the “rules of the road” for how you will deliver your product. This should be a team exercise, as the whole team will need to be on the same page in holding each other accountable so the data warehouse doesn’t become the Wild West.
However this doesn’t mean that once the standards and guiding principles have been set, they can never be changed. The things that should govern a brand new development effort are not all the same as what should govern a more mature EDW/BI environment. For instance when beginning development of a new data warehouse, the emphases may be experimentation of different approaches and rapid content development so end users see progress and have (albeit imperfect) data to begin testing out. As a result you may have relaxed controls over the production environment because there is little risk of major organizational impact if things break. On the other hand in a more mature data warehouse effort, you will need stricter controls over the production environment since you will have gained greater dependencies on your data structures and therefore a higher impact to the organization if something breaks.
Just like with your own health, regular check-ups of your EDW/BI environment are essential in maintaining a healthy, high-performing system. Follow these tips and you will be sure to exceed the expectations of your end users for years to come.
