In this blog, we will discuss about Hadoop federation, Hadoop architecture vs Hadoop Federated architecture and will talk about various issues solved by hdfs federation.
So let us first see why it is gaining so much popularity. To address this question we must know the problems in the existing architecture of Hadoop which led to the creation of Hadoop federation:
1) Availability: If we have a single namenode(master) managing all datanodes slaves) and it fails then the entire cluster goes down.
2) Scalability: With the Hadoop architecture we used to have single namenode and multiple datanodes. We were able to scale datanodes both horizontally and vertically but this was not possible with namenodes which could be scaled only vertically and not horizontally.
3) Isolation: A single NameNode offers no isolation in the multi-user environment. A single sample application can overload the single NameNode and slow down the critical production applications.
View original post 405 more words
