Data Virtulization or near real time data for reporting with Data Warehouse

By: Milind Zodge

Business requirement

Need to report near real time data segment along with consolidated data

Details

Data virtualization is getting lot of attention now-a-days. In current world business need of a data is changing. Previously Data warehouse used to support DSS applications and reporting tool like Dashboard and Scorecards which preliminary need summarized snapshot of a data.

However now the trend looks like going towards having a mix of consolidated data and near-real time data. There are few EII techniques and tools available for this however if you have to deliver this without spending a fortune you can leverage user database layer.

You can create a Data warehouse either top down or bottom up modeled and can have an ODS tables/schema to hold the ODS data without transformation. Various change data capture techniques can be used to keep ODS data in sync with the source, like in Oracle you can use Change data capture or Streams method.

Now since we are not transforming the ODS data it need to be transformed virtually. We can create a view layer combining these two layers to deliver data for some operational reporting and near real time data needs. Key is to transform the ODS data and fit it together with the data mart or data warehouse data.