Hadoop 2.0

Hadoop's initial architecture was based on the paradigm of scaling data horizontally and metadata vertically. While data storage and processing were handled by the slave nodes reasonably well, the masters that managed metadata were a single point of failure and limiting for web scale usage. Hadoop 2.0 has significantly re-architected both HDFS and the Map Reduce framework to address these issues. The HDFS namespace can be federated now using multiple name nodes on the same cluster and deployed in a HA mode. MapReduce has been replaced with YARN, which decouples cluster resource management from job state management and eliminates the scale/performance issues with the JobTracker. Most importantly, this change encourages deploying new distributed programming paradigms in addition to MapReduce on Hadoop clusters.