Iterative data warehousing

Like iterative software development, there is lot of value to be gained by delivering data warehousing projects using iterative techniques. Iterative data warehousing techniques allow the end users of the data warehouse to determine what reports they want and the ETL developers and data modelers to deliver those features without wasting time with data modeling and ETL jobs that do not provide immediate value to the business.

The industry has seen significant changes to the way we use and store data over the past few years. Agile development practices have lead to greater emphasis on evolutionary database design, requiring new tools that support migration of schemas in line with changes to the domain model of an application. As storage space consistently becomes cheaper and data access speeds increase, many organizations are investigating the use of multiple schemas to hold data for different purposes, e.g. transactional and analysis schemas. Incremental data warehousing is becoming increasingly popular as the cost of moving data between a transactional data store and an analysis environment is less than the value of having access to near real-time reporting of critical business data.