Amabri2.6.0、hdp2.6.1安装
Amabri2.6.0、hdp2.6.1在centos7下安装
一、环境准备
1、修改各个节点主机名
vi /etc/hostname
2、配置主节点hosts
vi /etc/hosts
2、配置免密
(1)手动配置
主节点执行:
ssh-****** -t rsa
ssh-copy-id $host(第1步中的各个节点名称)
(2)脚本配置
3、同步hosts
主节点执行:scp /etc/hosts $host:/etc/ ($host各从节点名称)
4、关闭防火墙、selinux
各个节点执行:
systemctl stop firewalld.service
systemctl disable firewalld.service
sed -i \”s;enforcing;disabled;g\” /etc/selinux/config
重启:reboot
5、安装httpd
查看httpd是否已安装:rpm -qa |grep httpd
安装:rpm -ivh httpd/*.rpm
6、镜像本地yum源配置
新建文件夹:mkdir /var/www/html/iso
将安装镜像中的文件全部复制到iso文件夹下
启动httpd:systemctl start httpd.service
验证:浏览器打开http://ip/iso/
编辑:vi /etc/yum.repos.d/local.repo
local.repo同步到其他节点
yum clean all
yum list update
yum makecache
yum repolist
7、检查更改最大文件描述符
ulimit -Sn
ulimit -Hn
如果\<10000,运行ulimit -n 10000
二、依赖检查
1、Python 2.7.X(所有节点)
Centos7自带python2.7.5即可。
(Anaconda3.6 还原为系统自带2.7:bashrc
注释掉anaconda的环境变量,退出用户,重新登录。)
2、JDK (所有节点)
HDP 2.6.1 JDK Support
JDK | Version |
---|---|
Open Source | JDK8† |
JDK7† , deprecated | |
Oracle | JDK 8, 64-bit (minimum JDK 1.8.0_77), default |
JDK 7, 64-bit (minimum JDK 1.7_67), deprecated |
(1)检测并卸载之前安装的版本:
rpm -qa |grep java
rpm -e java-1.8.0-openjdk-1.8.0.151-1.b12.el7_4.x86_64 –nodeps
rpm -e java-1.8.0-openjdk-headless-1.8.0.151-1.b12.el7_4.x86_64 –nodeps
(2)安装
rpm -ivh jdk-8u77-linux-x64.rpm
(3)验证
java -version
若找不到java,需配置环境变量:
在/etc/profile添加
source /etc/profile
3、MySQL5.6 (主节点)
(1)下载
http://mirrors.sohu.com/mysql/MySQL-5.6/
(2)查询并卸载系统自带mariadb
rpm -qa | grep mariadb -i
rpm -e MariaDB-common-10.2.13-1.el7.centos.x86_64
MariaDB-compat-10.2.13-1.el7.centos.x86_64
MariaDB-client-10.2.13-1.el7.centos.x86_64
MariaDB-server-10.2.13-1.el7.centos.x86_64 –nodeps
(3)安装perl、net-tools
yum install perl
yum install net-tools
(4)安装
tar -xvf MySQL-5.6.36-1.el7.x86_64.rpm-bundle.tar
rpm -ivh MySQL-server-5.6.36-1.el7.x86_64.rpm
密码在/root/.mysql_secret
service mysql start
rpm -ivh MySQL-client-5.6.36-1.el7.x86_64.rpm
执行mysql_secure_installation
(5)设置用户
验证登陆MySQL控制台:mysql -uroot –p
grant all privileges on *.* to root\@’%’ identified by “123456”;
flush privileges;
创建ambari数据库及用户:
create database ambari character set utf8 ;
CREATE USER ‘ambari’\@’%’ IDENTIFIED BY ‘ambari’;
GRANT ALL PRIVILEGES ON *.* TO ‘ambari’\@’%’ IDENTIFIED BY ‘ambari’;
FLUSH PRIVILEGES;
创建hive数据库及用户:
create database hive character set utf8 ;
CREATE USER ‘hive’\@’%’ IDENTIFIED BY ‘hive’;
GRANT ALL PRIVILEGES ON *.* TO ‘hive’\@’%’ IDENTIFIED BY ‘hive’;
FLUSH PRIVILEGES;
创建oozie数据库及用户:
create database oozie character set utf8 ;
CREATE USER ‘oozie’\@’%’ IDENTIFIED BY ‘oozie’;
GRANT ALL PRIVILEGES ON *.* TO ‘oozie’\@’%’ IDENTIFIED BY ‘oozie’;
FLUSH PRIVILEGES;
(6)设置MySQL为开机启动
chkconfig mysql on;
chkconfig –list mysql;
查看是否如同如下结果
mysql 0:off 1:off 2:on 3:on 4:on 5:on 6:off
(7)安装:
rpm -ivh MySQL-shared-compat-5.6.36-1.el7
rpm -ivh MySQL-shared-5.6.36-1.el7
(file /usr/share/mysql/charsets/README from install of
mariadb-libs-1:5.5.52-1.el7.x86_64 conflicts with file from package
MySQL-server-5.6.36-1.el7.x86_64)
三、ambari安装
hdp、hdp-utils下载地址
ambari下载地址
1、准备本地yum源
mkdir -p /var/www/html/ambari/HDP-UTILS-1.1.0.21
tar -zxf HDP-UTILS-1.1.0.21-centos7.tar.gz -C
/var/www/html/ambari/HDP-UTILS-1.1.0.21
tar -zxf HDP-2.6.1.0-centos7-rpm.tar.gz -C /var/www/html/ambari/
tar -zxf ambari-2.6.0.0-centos7.tar.gz -C /var/www/html/ambari/
重启httpd:systemctl restart httpd.service
验证:
2、配置repo
Ambari
cp /var/www/html/ambari/ambari/centos7/2.6.0.0-267/ambari.repo /etc/yum.repos.d/
HDP
cp /var/www/html/ambari/HDP/centos7/hdp.repo /etc/yum.repos.d
HDP-UTILS
cp /var/www/html/ambari/HDP-UTILS-1.1.0.21/hdp-util.repo /etc/yum.repos.d
scp到所有节点,执行:
yum clean all
yum list update
yum makecache
yum repolist
3、ambari-server安装
(1)安装mysql-connect
yum install mysql-connector-java
(2)安装ambari-server
yum install ambari-server
(3)配置ambari
ambari-server setup
(4)将ambari数据库脚本导入到数据库
ambari用户(上面设置的用户)登录mysql
mysql -u ambari -p
use ambari;
source /var/lib/ambari-server/resources/Ambari-DDL-MySQL-CREATE.sql
(5)设置mysql-connector路径
ambari-server setup –jdbc-db=mysql
–jdbc-driver=/usr/share/java/mysql-connector-java.jar
(6)启动ambari-server
ambari-server start
(7)验证
浏览器打开http://ip:8080/,用户名admin,密码admin
四、hdp安装
1、libtirpc安装
rpm -ivh libtirpc-0.2.4-0.10.el7.x86_64.rpm
libtirpc-devel-0.2.4-0.10.el7.x86_64.rpm
若提示与旧版本冲突,可用rpm -Uvh
2、登录ambari进行安装
点击Launch Install Wizard
(1) 设置集群名称
(2) 选择本地安装源
(3)设置集群
(4)确认hosts
(5)选择要安装的服务
(6)分配各服务master
安装组件时注意hive-metastore与mysql在同一台机器。
(7)分配slaves
(8)更改服务配置
删掉/boot/efi/hadoop/hdfs/namenode、/boot/efi/hadoop/hdfs/data
更改hive数据库为mysql
(9)显示配置信息
(10)开始安装
四、增加节点
五、启用HA
按照提示输入命令进行之后的安装。