Big Data Hadoop & Spark

Configuring Rack Awareness on Hadoop Using Centos

This blog gives you information about Rack Awareness in Apache Hadoop. HDFS block placement uses rack awareness for fault tolerance by placing one block replica on a different rack. This ensures data availability during network switch failure or a partition within the cluster. Rack awareness is very helpful in making an appropriate replication factor. Configuring rack awareness provides the information to Hadoop as to which Datanode is on which rack.
Note: For rack awareness configuration, all  the changes/modifications have to be made on NameNode (Masternode) only.

You can configure rack awareness in 3 steps:

1. Create a topology data file anywhere in Master node (NameNode)


Next, mention your slave nodes ( DataNodes) relative to their rack into

100% Free Course On Big Data Essentials

Subscribe to our blog and get access to this course ABSOLUTELY FREE. /rack1 /rack2 /rack2

2. Create a  script file (Also called as rack awareness script file)

HADOOP_CONF=/home/hadoop/address of
while [ $# -gt 0 ] ; do
exec< ${HADOOP_CONF}/
while read line ; do
ar=( $line )
if [ "${ar[0]}" = "$nodeArg" ] ; then
if [ -z "$result" ] ; then
echo -n "/default/rack "
echo -n "$result "

3. Add this property into core-site.xml of Master node only



Next, start your cluster.

Check the Hadoop admin report to see if the cluster is aware of the rack.

hadoop dfsadmin -report

hadoop dfsadmin -report
hadoop dfsadmin -report
hadoop dfsadmin -report
hadoop dfsadmin -report

This cluster is now rack aware !

Hope this post was helpful in understanding about the Commissioning and Decommissioning of the datanodes in Hadoop.
In case of any queries, feel free to write to us at [email protected] or comment below, and we will get back to you at the earliest. Keep visiting our website Acadgild for more updates on Big Data and other technologies. Click here to learn Big Data Hadoop Development.



One Comment

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Related Articles