Big Data Hadoop & Spark

Installing Mysql and Sqoop in Hadoop

Hadoop environment has made our work simpler and we can now insert and work with data effortlessly and quickly.

How about inserting table in Hadoop which already exists in MySQL?

In this blog, we will discuss the following:

  • Installing MySQL
  • Creating a table and inserting values
  • Installing Sqoop
  • Fetching the table within MySQL using Sqoop

Prerequisite

There’s only one prerequisite:

Running instance of Hadoop in any Linux OS.

Installing MySQL

Steps to install MySQL are as follows:

  1. Open terminal and follow the command below:
sudo yum install mysql-server

figure:1

     2.  If it asks for permission, enter yes.

figure:2

3. Let the download complete.

figure:3

    4. Once the download is completed, we will start MySQL services using the command:

sudo service mysql start

figure:4

   5.Enter the mysql shell as root user with the following command:

mysql -u root

figure:5

Congratulations!!! MySQL is installed on your Linux machine.

Creating a Table and Inserting Values

  1. Create a database to work with. The syntax is as follows:
Create database <database name>;
use <database name>;
  1. To create table and insert values inside table, follow the syntax shown below:
create table <table name>(column name1, column name 2);
insert into <table name> values(column1 value1, column2 value1);
insert into <table name> values(column1 value2, column2 value2);

figure:6

These commands are compulsory for a local user to have the privileges.

grant all on *.* to '<username>@<hostname>' with grant option;
flush privilages;
commit;
exit;

figure:7

Installing Sqoop

Steps to install Sqoop are as follows:

  1. Download Sqoop (1.99.6) from the link:

https://drive.google.com/open?id=0B1QaXx7tpw3SM1ZJUzdJemJjdDQ

  1. Download the highlighted tar file. Extract it and place it in a secure location.

Note: Location should not be altered.

figure:8

3. Set HOME and PATH inside .bashrc file.

Also, run the following command in the terminal for the changes to be affected.

source .bashrc

4.Search for the mysql-connector-java-5.1.21 in Google. Then download and save it within sqoop/lib directory.

figure:9

5. Come to the terminal to operate Sqoop.

6.Make the changes in .bashrc file with export path for Sqoop home. The path as shown in the image below.

figure:10

7. This command will confirm that sqoop is recognized by the system.

which sqoop

figure:11

Congratulations!!! Sqoop is now installed and running on your system.

Fetching the Table Within MySQL Using Sqoop

  1. The following command will fetch the table created inside MySQL named company:

figure:12

2. To see the result, use the following command which will allow access to HDFS.

figure:13

Hope this blog was useful in giving you an insight on Integrating MySQL with Sqoop. Click Here to learn more.

Hadoop

Tags

One Comment

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Related Articles

Close