Understanding HDFS Federation

Knoldus Blogs

In this blog, we will discuss about Hadoop federation, Hadoop architecture vs Hadoop Federated architecture and will talk about various issues solved by hdfs federation.

So let us first see why it is gaining so much popularity. To address this question we must know the problems in the existing architecture of Hadoop which led to the creation of Hadoop federation:

1) Availability: If we have a single namenode(master) managing all datanodes slaves) and it fails then the entire cluster goes down.

2) Scalability: With the Hadoop architecture we used to have single namenode and multiple datanodes. We were able to scale datanodes both horizontally and vertically but this was not possible with namenodes which could be scaled only vertically and not horizontally.

3) Isolation: A single NameNode offers no isolation in the multi-user environment. A single sample application can overload the single NameNode and slow down the critical production applications.

View original post 405 more words


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s