Sqoop tutorial 3 : Import arguments


argumentDescription
--appendAppend data to an existing dataset in HDFS
--as-avrodatafileImports data to Avro Data Files
--as-sequencefileImports data to SequenceFiles
--as-textfileImports data as plain text (default)
--boundary-query <statement>Boundary query to use for creating splits
--columns <col,col,col…>Columns to import from table
--directUse direct import fast path
--direct-split-size <n>Split the input stream every n bytes when importing in direct mode
--inline-lob-limit <n>Set the maximum size for an inline LOB
-m,--num-mappers <n>Use n map tasks to import in parallel
-e,--query <statement>Import the results of statement.
--split-by <column-name>Column of the table used to split work units
--table <table-name>Table to read
--target-dir <dir>HDFS destination dir
--warehouse-dir <dir>HDFS parent for table destination
--where <where clause>WHERE clause to use during import
-z,--compressEnable compression
--compression-codec <c>Use Hadoop codec (default gzip)
--null-string <null-string>The string to be written for a null value for string columns
--null-non-string <null-string>The string to be written for a null value for non-string columns

Comments

Post a Comment

Popular posts from this blog

Unix Tutorial 11 : Conditional Statements

Unix Conditional Statements The if-elif-fi Unix provides a number of relational operators in addition to the logical operators mentioned earlier. These can be used to compare numeric values. -lt less than -le less than or equal to -gt greater than -ge greater than or equal to -eq equal to -ne not equal to Unix provides a number of ways for conditionally executing the other commands. These are covered below: #1) The if statements Example: if <control command> then <statements> fi #2) The if…else statements Example: if <control command> then <statements> else <statements> fi #3) The if…elif…else…fi statement Example: if <control command> then <statements> elif then <statements> else <statements fi Given below are some example programs that illustrate these conditional statements: #1) Check if an input number is positive: $ echo “Enter a number” $ read num $ if [ $num -gt 0 ] $ t...
Read more

Unix Tutorial 12 : Loop Types

Loops in Unix You may use different loops based on the situation. They are: #1) Unix For loop statement Example:  This program will add 1+2+3+4+5 and result will be 15 for i in 1 2 3 4 5 do sum=`expr $sum + $i` done echo $sum #2) Unix While loop statement Example:  This program will print the value of ‘a’ five times, from 1 to 5. a=1 while [ $a -le 5 ] do echo “value of a=” $a a=`expr $a + 1` done #3) Unix Until loop statement This program will print the value of ‘a’ two times from 1 to 2. a=1 until [ $a -ge 3 ] do echo “value of a=” $a a=`expr $a + 1` done While running these loops, there may be a need to break out of the loop in some condition before completing all the iterations or to restart the loop before completing the remaining statements. This can be achieved with the ‘break’ and ‘continue’ statements. The following program illustrates the ‘break’ operation: num=1 while [ $num -le 5 ] do read var if [ $var -lt 0 ] then ...
Read more

Unix Tutorial 17 : Advance Unix

Unix includes commands for: Testing various conditions associated with specified files. Testing various conditions associated with specified strings. Performing file read/write operations. Arrays Arrays are used to store a series of values in an indexed list. Items in an array are stored and retrieved using an index. Note that Arrays are not supported by the original Bourne Shell, but are supported by bash and other newer shells. File Test Operators Shell scripts often need to check various properties of files as a part of the control flow. Unix provides a number of options for this purpose. File existence checks: -f file True if the file exists and is an ordinary file. -d file True if the file exists and is a directory. -s file True if the file exists and is not empty. -c file True if the file exists and is a character device file. -b file True if the file exists and is a block devise file. File access checks: -r file True if the file exists and has read perm...
Read more