A good understanding of what a Data Warehouse appliance is, would help understand when to think of getting one.
A data warehouse appliance is specifically tuned and built for a data warehouse, which consists of hardware (Processor, Memory, Storage, Network Array etc.) and software (DBMS – DataBase Management System, Application Interfaces, Administration Interfaces etc.)
Now how is that different from what you may already have – dedicated hardware (servers) and software (database such as Oracle or MySQL)?
Difference is, these commodity hardware and database software are meant for general purpose, which could be used and tweaked to build and maintain a data warehouse as other type of applications. But they are not exclusively meant to host a data warehouse.
The point above begs the question – what would be the need to have a dedicated hardware and software and what could be expected out of it?
If you are a CIO, please look back into the IT organization. Look deep into existing data infrastructure and take stock of the tools and technologies used. Mainly look deep into the challenges you face today.
Most often you would hear from your data architects and administrators that while the applications are tweaked to their maximum and hardware resources have been added (both proactively and on-demand), it just is unable to cope with the ‘demand’.
So what is the ‘demand’ ?
There are two dimensions to ‘demand’. One, what end user expects out of an analytical application and two, how much of data need to stored and analyzed, to cope with the first dimension of the demand.
Organizations today, wants to collect as much data as possible. Equally storage prices [cost per byte (gigabyte, terabyte, petabyte etc.)] are crashing. On the other hands, ends user requirements and expectations demands to analyze as much as that is stored.
I would even argue that storage price and end user requirements/expectations demands that we collect more. And this cycle continues.
So if you are a CIO today, you know that ‘exponential’ data growth is sure thing.
Coming back to the general purpose hardware and software – these machines as software systems were mostly meant for common demands – Web Applications, Light-weight transactional systems, Portals, Small Business registers etc. They do work well but again their goal is different. So the performance expectations don’t often match with what you would expect out of a high-volume data warehouse.
A data warehouse stores data, lot of it and in different forms – Common Identifiers, Long strings, Free form text, Transactional Information, Event driven data etc. etc.
When you have them all, and in big volume, not only efficient storage is important, but being able to analyze them to get actionable intelligence and in an shortest possible time-frame is paramount.
And we realize that existing infrastructure and architecture is not an ideal solution. This is where a Data warehouse Appliance comes to our rescue.
We will see more on this in the next part.