前言:
而今姐妹们对“centos6cdh”大致比较关切,咱们都想要剖析一些“centos6cdh”的相关资讯。那么小编在网上搜集了一些有关“centos6cdh””的相关资讯,希望朋友们能喜欢,小伙伴们快快来了解一下吧!本教程为离线安装版本,在线安装太费时间了,本教程是本人根据官方文档亲自操作完成,中间踩过坑,趟过雷,现把过程记录一下。
一、准备工作
1、准备四台电脑
192.168.1.101 node1 2CPU 8G内存 50G硬盘192.168.1.102 node2 2CPU 2G内存 50G硬盘192.168.1.103 node3 2CPU 2G内存 50G硬盘192.168.1.104 mysql.node 2CPU 1G内存 50G硬盘所有电脑禁用防火墙systemctl stop firewalldsystemctl disable firewalld
2、离线文件下载
CM 6.2.0
下载地址:CDH 6.2.0
下载地址:JDK1.8.0_212
准备 linux JDK: jdk-8u212-linux-x64.tar.gz
二、配置hostname及hosts
1、在node1上执行以下命令
#将主机名改为node1> hostnamectl set-hostname node1# 编辑 /etc/hosts文件,增加以下行192.168.1.101 node1192.168.1.102 node2192.168.1.103 node3192.168.1.104 mysql.node
2、在node2上执行以下命令
#将主机名改为node2> hostnamectl set-hostname node2# 编辑 /etc/hosts文件,增加以下行192.168.1.101 node1192.168.1.102 node2192.168.1.103 node3192.168.1.104 mysql.node
3、在node3上执行以下命令
#将主机名改为node3> hostnamectl set-hostname node3# 编辑 /etc/hosts文件,增加以下行192.168.1.101 node1192.168.1.102 node2192.168.1.103 node3192.168.1.104 mysql.node
4、在mysql.node上执行以下命令
> hostnamectl set-hostname mysql.node# 编辑 /etc/hosts文件,增加以下行192.168.1.101 node1192.168.1.102 node2192.168.1.103 node3192.168.1.104 mysql.node
三、JDK安装
分别在 node1、node2、node3电脑上安装JDK,JDK 要求
必需是64位必需1.8及以上相同版本的JDK安装在所有的集群电脑上必需安装在 /usr/java/目录下
四、配置免密登录
1、分别在node1、node2、node3 机器上生成密钥对
# 进入用户主目录> cd ~# 生成密钥对> ssh-keygen -t rsa然后一路回车键
2、在node1上执行
# 进入.ssh> cd ~/.ssh# 把本机的公钥追加到自身的~/.ssh/authorized_keys文件里> ssh-copy-id node1# 将authorized_keys文件复制到第二台主机> scp ~/.ssh/authorized_keys node2:~/.ssh/
3、在node2上执行
> cd ~/.ssh> ssh-copy-id node2> scp ~/.ssh/authorized_keys node3:~/.ssh/
4、在node3上执行
> cd ~/.ssh> ssh-copy-id node3> scp ~/.ssh/authorized_keys mysql.node:~/.ssh/
5、在mysql.node上执行
> cd ~/.ssh> ssh-copy-id mysql.node> scp ~/.ssh/authorized_keys node1:~/.ssh/> scp ~/.ssh/authorized_keys node2:~/.ssh/> scp ~/.ssh/authorized_keys node3:~/.ssh/
6、免密码ssh配置完成
五、安装Cloudera Manager Server
1、将下载下来的cloudera-manager-daemons-6.2.0-968826.el7.x86_64.rpm、cloudera-manager-agent-6.2.0-968826.el7.x86_64.rpm、cloudera-manager-server-6.2.0-968826.el7.x86_64.rpm上传到 node1电脑/tmp目录下
2、node1上安装cloudera-manager-daemons、cloudera-manager-agent、cloudera-manager-server
> cd /tmp> yum -y localinstall cloudera-manager-daemons-6.2.0-968826.el7.x86_64.rpm> yum -y localinstall cloudera-manager-agent-6.2.0-968826.el7.x86_64.rpm> yum -y localinstall cloudera-manager-server-6.2.0-968826.el7.x86_64.rpm
3、node2、node3 上只安装cloudera-manager-daemons、cloudera-manager-agent,把这两个安装文件上传到 node2、node3 /tmp目录下
> cd /tmp> yum -y localinstall cloudera-manager-daemons-6.2.0-968826.el7.x86_64.rpm> yum -y localinstall cloudera-manager-agent-6.2.0-968826.el7.x86_64.rpm
4、将以下文件拷贝到node1 /opt/cloudera/parcel-repo 目录下,安装CDH 本地源
CDH-6.2.0-1.cdh6.2.0.p0.967373-el7.parcelmanifest.jsonCDH-6.2.0-1.cdh6.2.0.p0.967373-el7.parcel.sha1CDH-6.2.0-1.cdh6.2.0.p0.967373-el7.parcel.sha1 将此文件重命名为CDH-6.2.0-1.cdh6.2.0.p0.967373-el7.parcel.sha
5、修改node1、node2、node3电脑中 /etc/cloudera-scm-agent/config.ini文件
将文件中 server_host=localhost 改为 server_host=node1
六、安装MySQL数据库
1、在mysql.node电脑上安装Mysql
# 如果电脑中没有 wget工具> yum -y install wget> wget ; rpm -ivh mysql-community-release-el7-5.noarch.rpm> yum install mysql-server
2、按照官方的建议,将/etc/my.cnf文件内容更改为以下配置
[mysqld]datadir=/var/lib/mysqlsocket=/var/lib/mysql/mysql.socktransaction-isolation = READ-COMMITTED# Disabling symbolic-links is recommended to prevent assorted security risks;# to do so, uncomment this line:symbolic-links = 0key_buffer_size = 32Mmax_allowed_packet = 32Mthread_stack = 256Kthread_cache_size = 64query_cache_limit = 8Mquery_cache_size = 64Mquery_cache_type = 1max_connections = 550#expire_logs_days = 10#max_binlog_size = 100M#log_bin should be on a disk with enough free space.#Replace '/var/lib/mysql/mysql_binary_log' with an appropriate path for your#system and chown the specified folder to the mysql user.log_bin=/var/lib/mysql/mysql_binary_log#In later versions of MySQL, if you enable the binary log and do not set#a server_id, MySQL will not start. The server_id must be unique within#the replicating group.server_id=1binlog_format = mixedread_buffer_size = 2Mread_rnd_buffer_size = 16Msort_buffer_size = 8Mjoin_buffer_size = 8M# InnoDB settingsinnodb_file_per_table = 1innodb_flush_log_at_trx_commit = 2innodb_log_buffer_size = 64Minnodb_buffer_pool_size = 4Ginnodb_thread_concurrency = 8innodb_flush_method = O_DIRECTinnodb_log_file_size = 512M[mysqld_safe]log-error=/var/log/mysqld.logpid-file=/var/run/mysqld/mysqld.pidsql_mode=STRICT_ALL_TABLES
3、将mysql设置为开机启动
> systemctl enable mysqld
4、启动Mysql server
> systemctl start mysqld
5、运行/usr/bin/mysql_secure_installation设置MySQL root密码和其他安全相关设置。
> /usr/bin/mysql_secure_installation[...]Enter current password for root (enter for none):OK, successfully used password, moving on...[...]Set root password? [Y/n] YNew password:Re-enter new password:Remove anonymous users? [Y/n] Y[...]Disallow root login remotely? [Y/n] N[...]Remove test database and access to it [Y/n] Y[...]Reload privilege tables now? [Y/n] YAll done!
6、Mysql安装完成
7、为Cloudera 软件创建相应的数据库及用户
mysql> CREATE USER '<user>' IDENTIFIED BY '<password>';mysql> CREATE DATABASE <database> DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;mysql> GRANT ALL ON <database>.* TO '<user>'@'%' IDENTIFIED BY '<password>';
注:可能Mysql会无法远程连接,如果无法连接请进行以下步骤
> mysql -uroot -p[password]mysql> show databases;mysql> use mysql;mysql> select host from user where user = 'root';mysql> update user set host='%' where user='root' and host='localhost';mysql> exit
七、Cloudera Manager Server 连接数据库
1、在node1上,提供JDBC连接驱动
这里提供的是 mysql-connector-java-5.1.46.jar,将此文件放到 node1电脑/usr/share/java/(必需这个目录)目录下,并重命名为mysql-connector-java.jar
2、Cloudera Manager Server 提供了一个脚本来配置数据库
> /opt/cloudera/cm/schema/scm_prepare_database.sh [options] <databaseType> <databaseName> <databaseUser> <password>databaseType: [mysql | oracle | postgresql]databaseName: 数据库名databaseUser: 数据库用户名password: 数据库密码
3、如果Mysql和Cloudera Manager Server 在同一台机器上,则执行以下命令
# 此示例假定您已创建Cloudera Management Server数据库和数据库用户,同时命名为scm:> /opt/cloudera/cm/schema/scm_prepare_database.sh mysql scm scmEnter SCM password:...[main] DbCommandExecutor INFO Successfully connected to database.All done, your SCM database is configured correctly!
4、如果Mysql和Cloudera Manager Server 不在同一台机器上,则执行以下命令
# 本教程为不同机器安装,所以执行这个命令> /opt/cloudera/cm/schema/scm_prepare_database.sh mysql -h mysql.node --scm-host node1 scm scmEnter database password:...[main] DbCommandExecutor INFO Successfully connected to database.All done, your SCM database is configured correctly!
八、启动Cloudera Manager Server 和 Agent
1、启动 Cloudera Manager Server
> systemctl start cloudera-scm-server
2、执行以下命令可以查看server日志
> tail -f /var/log/cloudera-scm-server/cloudera-scm-server.log
当看到此日志条目时,Cloudera Manager管理控制台已准备就绪:
INFO WebServerImpl:com.cloudera.server.cmf.WebServerImpl: Started Jetty server.
3、分别在 node1、node2、node3 上启动agent
> systemctl start cloudera-scm-agent
4、执行以下命令查看agent日志
> tail -f /var/log/cloudera-scm-agent/cloudera-scm-agent.log
5、在浏览器中打开 ;server_host>:7180, 默认用户名、密码:admin
6、登录成功,配置集群,结束
标签: #centos6cdh