Warehousing Your Data in the Cloud with ETL

DWH on the CloudThe process of taking data from different systems and putting it into a data warehouse for business analysis can be a complicated affair. In this article, we look at what is involved and how the cloud has made matters potentially trickier.

A data warehouse is a relational database that is designed for query and analysis rather than for transaction processing. It usually contains historical data derived from transaction data, but it can include data from other sources..

Real Time Data Integration - CDC

There is an ever greater need in the Business Intelligence environment to have the information in the shortest time possible, data generation cycles getting shorter and updating of data in near real time. There is talk of 'Operational Business Intelligence (OBI)' and 'Real Time Decision Support'.

It is critical to reaching operational data analytic environments in shortest possible time. There is a need for a 'Real Time Data Integration'.

In the optimization of these data integration processes, we must consider both the usual source data (ERP's, CRM's, operating systems, databases, flat files, Excel, XML, etc. ..) and other from more immediate nature such as messaging queues and on-line information accessed via Web services or RSS.

Aspects to evaluate the selection of an ETL tool

Addressing a business intelligence project is important to proper assessment of the ETL tool that we will use. The tool based on which we will implement our procurement processes of the Datamart, Datawarehouse or storage structure based on which further exploit the data. It is a cornerstone for the design, construction and subsequent evolution of our BI system. We will analyze technical issues only, without entering into the economic aspects or otherwise (licenses, agreements, technical support, tool changes, etc. ..) . Note that the ETL processes, are closely linked to the processes of data profiling and data quality, here we will not consider.

Do your BI platform support the data source you want to analyze?

Although the latest versions of most BI platforms support a wide range of data sources, this is a common question which involved the version of your BI tool, the version of the database file format or ERP that acts as a data source and the operating system.

 

In the case of SAS can solve these questions by referring to the SAS / ACCESS Validation Matrix, selecting the version of SAS, the database, the operating system and you have the answer.

SSIS: Workaround to two errors for no apparent reason when we insert data in MySql

SSIS: Workaround to two errors for no apparent reason when we insert data in MySqlFor whatever reason we may have to develop a Microsoft Integration Services package we move data from any source to a table that is in a MySQL database.

The first aim will do so through an ADO.NET Destination and MySQL provider for the connection. If we do well to insert the data directly, to create the target, select the connection and then select the table appears an error like the following check everything with the preview or try and do the assignments...