Week7: Sqoop installation and sqoop workouts
Step 1: Installing Sqoop
The following commands are used to extract the Sqoop tar ball and move it to lusr/lib/sqoop"
directory.
O cd install
tar xvzf sqoop-1.4.5.bin_ [Link]
sudo mv [Link] hadoop-2.0.4-alpha /usrlocal/'sqoop
Step 2: Configuring bashre
You have to set up the Sqoop environment by appending the following lines to
.bashrc file:yi Hbashrc
echo 'export SQ0OP HOME=lusr/local/sqoop' >>
[Link]'export
PATH=$PATH:$SQ0OP HOME/bin' >>.bashrc
The following command is used to execute .bashrc
[Link] .bashrc
Step 3: Configuring Sqoop
To configureSqoop with Hadoop, you need to edit the [Link] file, which is placed in the
$SQOOP HOME/conf directory. First of all, Redirect to Sqoop config directory and copy the
template file using the following command:
cd $ZQ0OP HOME/conf
mv [Link] [Link]
Open [Link] and edit the following lines: vi [Link])
echo 'export HADOOP COMMON HOME=/usr/local/hadoop' >>
[Link]
echo 'export HADOOP MAPRED HOME-lusr/local/hadoop' >>
[Link]
Step 4: Download and Configuremysql-connector-java
cd install
p-p mysql-connector-java,jar /usr/local/sqoop
Step 5: Verifying Sqoop
The following command is used to verify the Sqoop
version. source /.bashrc
(sqoop-version
MYSQL (Preparation of Source) :
Login as mysql and start the Service:
su root
password:
hdusersu
mysql
service mysqld
restart mysql
Select the test database:
Use test:
CREATE TABLE customer (custid INT,firstname VARCHAR(20),lastname
VARCHAR(20),city varchar(50),age int,createdt date,transactamt int );
insert into customer
values(1,Arun'
20',100000); insert into customer ,'Kumar'chennai',33,201S-09
values(2,srini','vasan''chennai,33,2015-09-21,10000); insert into
customer values(3,vasu','devan,"'banglore',39,;2015-09-231,90000);
insert into customer
values(4,mohamed'
09-24,1000); insert into customer ,'imran',hyderabad,33,2015
values(5,arun' 'basker,chennai',23,2015-09-20",200000); insert into
customer values(6,'ramesh, 'babu''manglore,39,/2015-09-21,100000);
select from customer;
SQ00P WORKOUTS(Open a separate linux terminal)
To List Databases which are in MySql
Vsqoop list-databases --connect jdbc:mysql:/Mocalhost --username root --password root;
To List Tables from test database
sqoop list-tables --connect jdbc:mysql://localhostitest --username root -password root;
Import Table from SQL toHDFS:
sqoop import --connectjdbc:mysql://localhost/test --username root --password root -table
customer -m 1;hadoop fs -rm -r /user/hduser/customer
sqoop import --connect jdbc:mysql://localhost/test --uSername root --password root -table
cçstQmer -m 1
direct;
Import with -split-by option
Sqoop imnport --connect jabc:mysq1:/ocalhost/test --username root --password root
customer -m 10 --split-by custid; -table
hadoop fs -rm -r /user/hduserlcustomer
sqoop import --connect jdbe:mysql://localhost/test --username root --password root -table
Export from HDFS to SQL:(Create table in MYSQL
before running this commmand)
CREATE TABLE customerl (custid INT,firstname
VARCHAR(20),lastname
VARCHAR(20),city varchar(50),age int,createdt date,transactamt int );
sqoop export --connect
customerl -export-dir jdbc:mysql://localhost/test
savedjobl --username root --password root --table
