In the previous post, we talked about how you can in a sense “crowdsource” the development of your enterprise data warehouse (EDW) by having the EDW development team relinquish some of the control over every aspect of the EDW and allow data analysts who know the subject matter of the data best to develop the business logic for metrics and data elements. The data analyst actually develops the sql code (the database query code) for the metric/data element and “owns” the logic for how the metric/data element is defined going forward. The data architect then takes that code developed by the analyst, ensures it complies with EDW-wide standards, and tweaks it if necessary to make it run faster.
However for this to work well, it assumes data analysts will be willing to learn sql programming. As I mentioned in the last post, we have seen strong sql holdouts become equally strong sql advocates, but this doesn’t happen overnight. Here’s how it generally works out in the real world (after the data analyst emerges from the Flat-Out-Refusal stage of course):
- The data analyst will sit down with the data architect and the data architect will write the sql code for the data element (like the readmission flag) or metric (like readmission rate)
- The architect will help the analyst become comfortable over time with the sql statement, such as simply running it and viewing results and eventually understanding where the filter criteria can be found (“you can see here it shows the discharge statuses that are excluded”)
- The analyst will start to make changes to the sql statement and run it past the data architect to validate
- The analyst will start creating new data elements and measures by using the existing sql statements as a template
- The data analyst will eventually gain an understanding of sql and will come to appreciate the flexibility and scope of analysis not possible inside a BI tool
- Peace and harmony will prevail*
*results may vary
Adopting a process like this for ownership and development of key metrics and data elements does not in any way mean data architects are less important. This approach simply allows data architects do what they do best: design the EDW to be fast and easy-to-use, and allows data analysts to do what they do best: understand and analyze the intricate details of the data they are responsible for. The result is a reduction in the number of handoffs and less key information getting lost in translation, making the whole process more efficient and allowing your organization to do more with less.
I hope this series has been useful to you, and it no doubt raises many questions that can’t be covered well within the scope of a blog. Reach out to us to talk with a member of our team about the challenges you are facing!
