Sqoop tutorial 1 : Introduction

What is Sqoop?

Sqoop is a tool designed to transfer data between Hadoop and relational databases. You can use Sqoop to import data from a relational database management system (RDBMS) such as MySQL or Oracle into the Hadoop Distributed File System (HDFS), transform the data in Hadoop MapReduce, and then export the data back into an RDBMS.

Sqoop automates most of this process, relying on the database to describe the schema for the data to be imported. Sqoop uses MapReduce to import and export the data, which provides parallel operation as well as fault tolerance.


Comments

Post a Comment

Popular posts from this blog

Unix Tutorial 14 : Text Processing Commands 1

Unix Filter Commands grep:  Find lines in stdin that match a pattern and print them to stdout. sort:  Sort the lines in stdin, and print the result to stdout. uniq:  Read from stdin and print unique (that are different from the adjacent line) to stdout. cat:  Read lines from stdin (and more files), and concatenate them to stdout. more:  Read lines from stdin, and provide a paginated view to stdout. cut:  Cut specified byte, character or field from each line of stdin and print to stdout. paste:  Read lines from stdin (and more files), and paste them together line-by-line to stdout. head:  Read the first few lines from stdin (and more files) and print them to stdout. tail:  Read the last few lines from stdin (and more files) and print them to stdout. wc:  Read from stdin, and print the number of newlines, words, and bytes to stdout. tr:  Translate or delete characters read from stdin and print to stdout. Command grep...
Read more

Unix Tutorial 11 : Conditional Statements

Unix Conditional Statements The if-elif-fi Unix provides a number of relational operators in addition to the logical operators mentioned earlier. These can be used to compare numeric values. -lt less than -le less than or equal to -gt greater than -ge greater than or equal to -eq equal to -ne not equal to Unix provides a number of ways for conditionally executing the other commands. These are covered below: #1) The if statements Example: if <control command> then <statements> fi #2) The if…else statements Example: if <control command> then <statements> else <statements> fi #3) The if…elif…else…fi statement Example: if <control command> then <statements> elif then <statements> else <statements fi Given below are some example programs that illustrate these conditional statements: #1) Check if an input number is positive: $ echo “Enter a number” $ read num $ if [ $num -gt 0 ] $ t...
Read more

Unix Tutorial 12 : Loop Types

Loops in Unix You may use different loops based on the situation. They are: #1) Unix For loop statement Example:  This program will add 1+2+3+4+5 and result will be 15 for i in 1 2 3 4 5 do sum=`expr $sum + $i` done echo $sum #2) Unix While loop statement Example:  This program will print the value of ‘a’ five times, from 1 to 5. a=1 while [ $a -le 5 ] do echo “value of a=” $a a=`expr $a + 1` done #3) Unix Until loop statement This program will print the value of ‘a’ two times from 1 to 2. a=1 until [ $a -ge 3 ] do echo “value of a=” $a a=`expr $a + 1` done While running these loops, there may be a need to break out of the loop in some condition before completing all the iterations or to restart the loop before completing the remaining statements. This can be achieved with the ‘break’ and ‘continue’ statements. The following program illustrates the ‘break’ operation: num=1 while [ $num -le 5 ] do read var if [ $var -lt 0 ] then ...
Read more