Big Data Hadoop & Spark

Horizontal vs Vertical Scaling in Hadoop

Introduction

In this tutorial, We will be discussing  about the need for Hadoop and we also will be discussing how Horizontal Scaling architecture is better compared to Vertical scaling and Traditional distributed architecture to store and process the Big Data .

If Big Data is implemented in traditional distributed architecture like DBMS, which is suitable only for working on structured and semi-structured data (partially) like structured documents, finances, stock records, personnel files and since the traditional data analysis software’s are not so equipped to handle Big Data as it may take months to process it.

100% Free Course On Big Data Essentials

Subscribe to our blog and get access to this course ABSOLUTELY FREE.

But, as in today’s world, organizations are concentrating more on a framework which will help to store and process structured, semi-structured and unstructured data like 3D models, location data, photographs, audio video files etc on a single platform which are complex to process in traditional distributed architecture. Hence, Hadoop is the solution which has the capability to store and process structured, semi-structured and unstructured data in a distributed architecture.

Vertical Scaling

We all know that more than petabytes of data are generated every day, from various sources around the world via social media, search engine, Stock market, transport sectors, banking sectors, e-commerce and many other resources. Since the flow of data continue to grow and shows no sign of slowing down, it is almost impossible to run the so-called Big Data on a single machine.

Moreover, because of several limitations on a single machine where a user cannot add multiple hardware (like ram, hard disk) to a single system and will be very complex to maintain the hardware when they are added to a single system. This procedure of adding multiple hardware to a single system is known as Vertical Scaling.

Horizontal Scaling

So, in order to increase the storage capacity and the processing power, the best and suitable solution to handle Big Data is to store the data in a distributed architecture, where users can add multiple systems or nodes when there is an increase in data. Since the systems are connected in a distributed architecture, the performance of the processing data will be very high compared to those running in a single system (Vertical Scaling). This procedure of storing and processing data in a distributed architecture is known as Horizontal Scaling.

Hadoop
For more insight on the concepts of Big Data and Hadoop you can refer to our below blogs

Understanding Big Data

Big Data Terminologies You Must Know

Step-by-Step Guide to Become a Big Data Developer

Thus, we hope this blog has helped you understand Need of Hadoop and Use of Horizontal Scaling architecture.

Keep visiting our site www.acadgild.com for more updates on Bigdata and other technologies.

Hadoop

Manjunath

is working with AcadGild as Big Data Engineer and is a Big Data enthusiast with 2+ years of experience in Hadoop Development. He is passionate about coding in Hive, Spark, Scala. Feel free to contact him at [email protected] for any further queries.

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Related Articles

Close
Close