Friday, 19 June 2015

Steps to Install and Configure SQOOP


SQOOP is used to import and export data between structured environment (RDBMS) and Hadoop cluster. It internally uses JDBC driver in order to establish connection between source and destination.

Let us see steps to install and configure SQOOP on Hadoop environment. 

Step-1 [Download and extract the SQOOP binaries from the below link for the stable version]


>> tar -xvf sqoop-1.4.6.bin_hadoop-2.0.4-alpha.tar.gz
>> su
>> mv sqoop-1.4.6.bin_hadoop-2.0.4-alpha /usr/lib/sqoop

Step-2 [Configure bashrc]

Append the below lines to ~/.bashrc file.

export SQOOP_HOME=/usr/lib/sqoop
export PATH=$PATH:$SQOOP_HOME/bin

Execute the ~/.bashrc file

>> source ~/.bashrc

Step:3 [Configure sqoop-env.sh]

>> cd $SQOOP_HOME/conf
>> mv sqoop-env-template.sh sqoop-env.sh

>> gedit sqoop-env.sh

export HADOOP_COMMON_HOME=/usr/lib/hadoop
export HADOOP_MAPRED_HOME=/usr/lib/hadoop

Step-4 [Download and configure mysql-connector-java]

>> tar -zxf mysql-connector-java-5.1.30.tar.gz
>> su
>> cd mysql-connector-java-5.1.30
>> mv mysql-connector-java-5.1.30-bin.jar /usr/lib/sqoop/lib

Step-5 [Download and configure MSSQL-connector-java]


>> wget http://www.microsoft.com/en-us/download/confirmation.aspx?id=21599

>> tar -zxf sqljdbc_3.0.1301.101_enu.tar.gz
>> su
>> cd sqljdbc_3.0.1301.101_enu\sqljdbc_3.0\enu
>> mv sqljdbc4.jar /usr/lib/sqoop/lib

Step-6 [Verify the SQOOP]

>> cd $SQOOP_HOME/bin
>> sqoop-version