环境准备

　　Java

　　Hadoop(Hdfs/Yarn)

Hadoop2.6.0（单节点）下安装Sqoop

　　第一步：上传sqoop的安装包，这里不多赘述。

hadoop2.6.0（单节点）下Sqoop-1.4.6安装与配置（数据读取涉及hadoop、hbase和hive）

[[email protected] sqoop]$ pwd
/usr/local/sqoop
[[email protected] sqoop]$ ls
sqoop-1.4.6.bin__hadoop-2.0.4-alpha
[[email protected] sqoop]$ mv sqoop-1.4.6.bin__hadoop-2.0.4-alpha/ sqoop-1.4.6
[[email protected] sqoop]$ ls
sqoop-1.4.6
[[email protected] sqoop]$ cd sqoop-1.4.6/
[[email protected] sqoop-1.4.6]$ pwd
/usr/local/sqoop/sqoop-1.4.6
[[email protected] sqoop-1.4.6]$

hadoop2.6.0（单节点）下Sqoop-1.4.6安装与配置（数据读取涉及hadoop、hbase和hive）

[[email protected] sqoop-1.4.6]$ ls
bin CHANGELOG.txt conf ivy lib NOTICE.txt README.txt sqoop-patch-review.py src
build.xml COMPILING.txt docs ivy.xml LICENSE.txt pom-old.xml sqoop-1.4.6.jar sqoop-test-1.4.6.jar testdata
[[email protected] sqoop-1.4.6]$ cd conf/
[[email protected] conf]$ pwd
/usr/local/sqoop/sqoop-1.4.6/conf
[[email protected] conf]$ ls
oraoop-site-template.xml sqoop-env-template.cmd sqoop-env-template.sh sqoop-site-template.xml sqoop-site.xml
[[email protected] conf]$ cp sqoop-env-template.sh sqoop-env.sh
[[email protected] conf]$ vim sqoop-env.sh

　　第二步：配置文件

hadoop2.6.0（单节点）下Sqoop-1.4.6安装与配置（数据读取涉及hadoop、hbase和hive）

[[email protected] conf]$ vim sqoop-env.sh

# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements. See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

# included in all the hadoop scripts with source command
# should not be executable directly
# also should not be passed any arguments, since we need original $*

# Set Hadoop-specific environment variables here.

#Set path to where bin/hadoop is available
#export HADOOP_COMMON_HOME=　　　　（建议都安装上）

#Set path to where hadoop-*-core.jar is available
#export HADOOP_MAPRED_HOME=　　　　 （建议都安装上）

#set the path to where bin/hbase is available　　
#export HBASE_HOME=　　　　　　　　　　（建议都安装上）

#Set the path to where bin/hive is available
#export HIVE_HOME=　　　　　　　　　　　（建议都安装上）

#Set the path for where zookeper config dir is
#export ZOOCFGDIR=　　　　　　　　　　 （因为，我这里是，hadoop-2.6.0的单节点分布，所以就没必要去配置Zookeeper了）

如果数据读取不涉及hbase和hive，那么相关hbase和hive的配置可以不加；如果集群有独立的zookeeper集群，那么配置zookeeper，反之，不用配置。

在这里，我就全部配置吧，为了大家的方便！

hadoop2.6.0（单节点）下Sqoop-1.4.6安装与配置（数据读取涉及hadoop、hbase和hive）

　　所以，就没配置Zookeeper了。

export HADOOP_COMMON_HOME=/usr/local/hadoop/hadoop-2.6.0

export HADOOP_MAPRED_HOME=/usr/local/hadoop/hadoop-2.6.0

export HBASE_HOME=/usr/local/hbase/hbase-1.2.3

export HIVE_HOME=/usr/local/hive/hive-1.0.0

　　第三：配置环境变量

hadoop2.6.0（单节点）下Sqoop-1.4.6安装与配置（数据读取涉及hadoop、hbase和hive）

#sqoop
export SQOOP_HOME=/usr/local/sqoop/sqoop-1.4.6
export PATH=$PATH:$SQOOP_HOME/bin

　　第四步：生效环境变量

hadoop2.6.0（单节点）下Sqoop-1.4.6安装与配置（数据读取涉及hadoop、hbase和hive）

source /etc/profile

　　第五步：这里大家，记得要给sqoop安装目录，授予权限给hadoop用户

chown -R hadoop:hadoop sqoop-1.4.6

hadoop2.6.0（单节点）下Sqoop-1.4.6安装与配置（数据读取涉及hadoop、hbase和hive）

　　第六步：将相关的驱动 jar 包拷贝到 sqoop/lib 目录下。

hadoop2.6.0（单节点）下Sqoop-1.4.6安装与配置（数据读取涉及hadoop、hbase和hive）

　　这里，省略了，很多，包括。hadoo的相关核心jar包、hive的相关核心jar包和hbase的相关核心jar包（补补）

测试

　　比如，我这里打开下，数据库

hadoop2.6.0（单节点）下Sqoop-1.4.6安装与配置（数据读取涉及hadoop、hbase和hive）

Navicat for MySQL之MySQL客户端的下载、安装和使用

个人推荐，比较好的MySQL客户端工具

　　得，先启动之前安装好的数据库。

hadoop2.6.0（单节点）下Sqoop-1.4.6安装与配置（数据读取涉及hadoop、hbase和hive）

[[email protected] ~]$ su root
Password: 
[[email protected] hadoop]# cd /usr/local/
[[email protected] local]# pwd
/usr/local
[[email protected] local]# service mysqld start
Starting mysqld:                                           [  OK  ]
[[email protected] local]#

　　然后，这边，选择连接。

hadoop2.6.0（单节点）下Sqoop-1.4.6安装与配置（数据读取涉及hadoop、hbase和hive）

[[email protected] sqoop-1.4.6]$ sqoop list-databases --connect jdbc:mysql://djt002/ --username hive --password hive
Warning: /usr/local/sqoop/sqoop-1.4.6/../hcatalog does not exist! HCatalog jobs will fail.
Please set $HCAT_HOME to the root of your HCatalog installation.
Warning: /usr/local/sqoop/sqoop-1.4.6/../accumulo does not exist! Accumulo imports will fail.
Please set $ACCUMULO_HOME to the root of your Accumulo installation.
Warning: /usr/local/sqoop/sqoop-1.4.6/../zookeeper does not exist! Accumulo imports will fail.
Please set $ZOOKEEPER_HOME to the root of your Zookeeper installation.
17/03/17 20:30:25 INFO sqoop.Sqoop: Running Sqoop version: 1.4.6
17/03/17 20:30:25 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead.
17/03/17 20:30:27 INFO manager.MySQLManager: Preparing to use a MySQL streaming resultset.
information_schema
hive
mysql
test
[[email protected] sqoop-1.4.6]$ sqoop list-tables --connect jdbc:mysql://djt002/hive --username hive --password hive
Warning: /usr/local/sqoop/sqoop-1.4.6/../hcatalog does not exist! HCatalog jobs will fail.
Please set $HCAT_HOME to the root of your HCatalog installation.
Warning: /usr/local/sqoop/sqoop-1.4.6/../accumulo does not exist! Accumulo imports will fail.
Please set $ACCUMULO_HOME to the root of your Accumulo installation.
Warning: /usr/local/sqoop/sqoop-1.4.6/../zookeeper does not exist! Accumulo imports will fail.
Please set $ZOOKEEPER_HOME to the root of your Zookeeper installation.
17/03/17 20:30:48 INFO sqoop.Sqoop: Running Sqoop version: 1.4.6
17/03/17 20:30:48 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead.
17/03/17 20:30:50 INFO manager.MySQLManager: Preparing to use a MySQL streaming resultset.
BUCKETING_COLS
CDS
COLUMNS_V2
DATABASE_PARAMS
DBS
FUNCS
FUNC_RU
GLOBAL_PRIVS
IDXS
INDEX_PARAMS
PARTITIONS
PARTITION_KEYS
PARTITION_KEY_VALS
PARTITION_PARAMS
PART_COL_PRIVS
PART_COL_STATS
PART_PRIVS
ROLES
SDS
SD_PARAMS
SEQUENCE_TABLE
SERDES
SERDE_PARAMS
SKEWED_COL_NAMES
SKEWED_COL_VALUE_LOC_MAP
SKEWED_STRING_LIST
SKEWED_STRING_LIST_VALUES
SKEWED_VALUES
SORT_COLS
TABLE_PARAMS
TAB_COL_STATS
TBLS
TBL_COL_PRIVS
TBL_PRIVS
VERSION
[[email protected] sqoop-1.4.6]$ pwd
/usr/local/sqoop/sqoop-1.4.6
[[email protected] sqoop-1.4.6]$

　　然后，继续，还没达到我们想要的目的效果。

继续，怎么做呢？

补充Sqoop命令怎么用？

[[email protected] sqoop-1.4.6]$ sqoop help
Warning: /usr/local/sqoop/sqoop-1.4.6/../hcatalog does not exist! HCatalog jobs will fail.
Please set $HCAT_HOME to the root of your HCatalog installation.
Warning: /usr/local/sqoop/sqoop-1.4.6/../accumulo does not exist! Accumulo imports will fail.
Please set $ACCUMULO_HOME to the root of your Accumulo installation.
Warning: /usr/local/sqoop/sqoop-1.4.6/../zookeeper does not exist! Accumulo imports will fail.
Please set $ZOOKEEPER_HOME to the root of your Zookeeper installation.
17/03/17 20:03:21 INFO sqoop.Sqoop: Running Sqoop version: 1.4.6
usage: sqoop COMMAND [ARGS] 即sqoop 命令 参数

Available commands:
codegen Generate code to interact with database records
create-hive-table Import a table definition into Hive 跟hive有关
eval Evaluate a SQL statement and display the results
export Export an HDFS directory to a database table
help List available commands
import Import a table from a database to HDFS 导入
import-all-tables Import tables from a database to HDFS
import-mainframe Import datasets from a mainframe server to HDFS
job Work with saved jobs
list-databases List available databases on a server 列出数据库
list-tables List available tables in a database 列出数据表
merge Merge results of incremental imports 合并增量导入
metastore Run a standalone Sqoop metastore 元数据存储
version Display version information 版本号

See 'sqoop help COMMAND' for information on a specific command.
[[email protected] sqoop-1.4.6]$