目录一、序CDH简介CM简介为什么选择CDH而不是Apache HadoopCDH安装说明二、准备工作前置条件测试服务器准备软件环境准备CDH安装包准备查看CDH6.3.2安装包查看CM6.3.1安装包查看JDK1.8安装包环境配置配置静态网络IP修改主机名修改hosts配置配置SSH免密登录创建密钥对查看密钥对验证免密登录远程批量操作pssh工具关闭防火墙禁用SELinux配置swap关闭透明大页配置最大进程数和最大文件句柄数禁用atime集群时钟同步安装JDK安装MySQL部署MySQLMySQL配置初始化数据库及用户安装JDBC驱动程序搭建私有Yum源安装httpd服务配置web服务制作本地repo源三、CM6.3.1安装步骤集群角色安装CM6.3.1安装cloudera manager server初始化scm数据库启动cloudera manager server四、CDH集群6.3.2安装步骤集群安装向导集群安装设置一、序官方文档Cloudera Enterprise 6.3 DocumentationCDH简介CDH是一个基于Apache Hadoop的开源软件分发版本由Cloudera公司开发和维护。CDH提供了统一批处理、交互式SQL和交互式搜索以及基于角色的访问控制的Hadoop解决方案组件包括Hadoop分布式文件系统HDFS、分布式计算框架MapReduce及其他相关开源组件如Hive、HBase、Impala等。CDH的目标是简化大数据处理和分析的过程使企业能够更好地利用大数据来获取洞察力和价值。CM简介CM是Cloudera提供的一个集中式管理工具用于管理和监控CDH集群。CM提供了直观的Web界面管理员可以使用该界面配置、部署和监控CDH集群。CM还提供了一些高级功能如自动化部署、故障诊断和报警通知来帮助管理员更好地管理和维护CDH集群。通过CM管理员可以轻松管理集群的配置、监控集群的健康状况并执行集群维护任务和性能调优等操作。为什么选择CDH而不是Apache HadoopCDH和Apache Hadoop比较特点CDHApache Hadoop功能全面性提供了广泛的组件和工具如Hive、HBase、Impala等支持不同的大数据处理和分析需求。提供核心的分布式存储HDFS和计算MapReduce功能需要使用其他组件或工具进行更高级的数据处理和分析。管理工具包含Cloudera ManagerCM提供集中式的管理和监控功能支持自动化部署、故障诊断和报警通知等。Apache Hadoop原生没有提供类似的集中式管理工具通常需要使用其他第三方工具进行管理和监控。商业支持Cloudera提供商业支持和服务包括技术支持、培训和咨询等。Apache Hadoop是一个开源项目没有官方提供的商业支持但有许多第三方公司提供相关支持和服务。发展速度CDH的发展速度相对较快并且每个版本都经过Cloudera的测试和验证。Apache Hadoop作为一个开源项目有更多社区贡献者因此发展速度更快有更频繁的更新和新特性引入。综上所述CDH提供了更全面和更好用的功能及配套管理工具使大数据处理和分析更简化和集中化。一般来说集群规模越大对集群的管理和运维能力要求越高此时CDH能够发挥的价值就会越大。CDH安装说明Cloudera企业版收费不差钱又缺技术支持的可以考虑购买Cloudera社区版开源免费社区文档和资源也比较丰富适合小规模集群部署不过根据Cloudera官方发布的公告自2021年7月起Cloudera社区版已经停止更新和发布CDH 6.3.2将是Cloudera社区版的最后一个免费版本。而且从2021年2月1日开始所有的CDH和Cloudera Manager包括历史版本下载都需要用户名和密码就是用户需要订阅 Cloudera 产品获取到对应的授权才能下载。所以我们将使用离线包进行安装部署本文以CDH6.3.2CM6.3.1为例详细介绍大数据集群的安装部署过程供大家参考文中离线安装包也会无偿分享给大家下载使用老铁们码字不易记得点赞收藏关注咱不迷路二、准备工作前置条件测试服务器准备本文以3台测试服务器为例以root用户身份登录节点配置如下测试服务器配置信息序号主机IPCPU内存磁盘hdfs角色1node1192.168.5.1016C40G512G系统盘(xfs)1T数据盘(ext4)namenode1/datanode2node2192.168.5.1116C40G512G系统盘(xfs)1T数据盘(ext4)namenode2/datanode3node3192.168.5.1216C40G512G系统盘(xfs)1T数据盘(ext4)datanode软件环境准备基础软件环境信息序号软件名称版本信息备注1Centos操作系统发行版CentOS Linux release 7.9.2009 x86_64CDH要求7.2或更高2MySQL数据库5.7.43 x86_64CDH6推荐5.7或更高务必使用utf8编码而非utf8mb4编码3JDK1.8.0_181 x86_64CDH6将不再支持JDK1.7推荐1.8u1814python2.7.55CDH6.3.26CM6.3.1CDH安装包准备CDH、CM和JDK安装包需要提前下载目前官方不提供免费下载我已经上传到百度网盘方便大家使用链接地址:CDH6.3.2https://pan.baidu.com/s/1hKuldVcZT2HHGcBEF5VWuQ需要安装包的童鞋请留言或私信我下载网盘里的cdh文件夹并保存到本地。查看CDH6.3.2安装包[rootnode1 cdh]# tree CDH6.3.2/ CDH6.3.2/ ├── CDH-6.3.2-1.cdh6.3.2.p0.1605554-el7.parcel ├── CDH-6.3.2-1.cdh6.3.2.p0.1605554-el7.parcel.sha1 ├── CDH-6.3.2-1.cdh6.3.2.p0.1605554-el7.parcel.sha256 └── manifest.json 0 directories, 4 files查看CM6.3.1安装包[rootnode1 cdh]# tree CM6.3.1 CM6.3.1 ├── allkeys.asc ├── cloudera-manager-agent-6.3.1-1466458.el7.x86_64.rpm ├── cloudera-manager-daemons-6.3.1-1466458.el7.x86_64.rpm ├── cloudera-manager-server-6.3.1-1466458.el7.x86_64.rpm ├── cloudera-manager-server-db-2-6.3.1-1466458.el7.x86_64.rpm └── enterprise-debuginfo-6.3.1-1466458.el7.x86_64.rpm 0 directories, 6 files查看JDK1.8安装包[rootnode1 cdh]# tree JDK JDK └── oracle-j2sdk1.8-1.8.0update181-1.x86_64.rpm 0 directories, 1 file环境配置配置静态网络IPCentos7静态网络配置文件是/etc/sysconfig/network-scripts/ifcfg-interface,我本机网络接口名称为p2p1修改所有节点网络配置具体参数如下[rootnode1 ~]# vim /etc/sysconfig/network-scripts/ifcfg-p2p1 TYPEEthernet PROXY_METHODnone BROWSER_ONLYno BOOTPROTOnone DEFROUTEyes IPV4_FAILURE_FATALyes IPV6INITyes IPV6_AUTOCONFyes IPV6_DEFROUTEyes IPV6_FAILURE_FATALno IPV6_ADDR_GEN_MODEstable-privacy NAMEp2p1 # 网络接口名称 UUID31b15de1-225f-4540-b135-b80d5da6f335 DEVICEp2p1 ONBOOTyes # 设置开机启动确保网络配置有效否则无法建立网络连接 IPADDR192.168.5.10 # 静态IP地址 PREFIX24 # 子网掩码相当于255.255.255.0 GATEWAY192.168.5.1 # 默认网关IP地址 DNS1192.168.1.1 # 首选DNS服务器IP地址上网必备 IPV6_PRIVACYno # 禁用IPV6重启网络可立即生效systemctl restart network修改主机名修改所有节点的hostname重启节点立即生效hostnamectl set-hostname node1 hostnamectl set-hostname node2 hostnamectl set-hostname node3修改hosts配置配置所有节点的IP和域名映射关系# vim /etc/hosts 192.168.5.10 node1 192.168.5.11 node2 192.168.5.12 node3配置SSH免密登录ssh-keygen用于生成和管理 SSH 密钥对SSH 密钥对由公钥和私钥组成公钥用于在 SSH 认证过程中提供给远程服务器进行安全认证通常会添加到远程服务器的authorized_keys文件中而私钥则默认保存在本地root用户默认将私钥对保存在/root/.ssh目录下。创建密钥对node1和node2节点上生成ssh密钥对执行ssh-keygen命令过程如下# 一路回车生成密钥对 [rootnode1 ~]# ssh-keygen -t rsa Generating public/private rsa key pair. Enter file in which to save the key (/root/.ssh/id_rsa): Enter passphrase (empty for no passphrase): Enter same passphrase again: Your identification has been saved in /root/.ssh/id_rsa. Your public key has been saved in /root/.ssh/id_rsa.pub. The key fingerprint is: SHA256:AJWzRl9kwU3kbQ7C518oIKCmQdpvKNeQxiIvxo4ly/w rootnode1 The keys randomart image is: ---[RSA 2048]---- | . ..o. oo | | . o. oo... | | B o.... o | |o O o... . | |o* . S o o .| |*B . o . | |o . | | . | | E | ----[SHA256]-----查看密钥对[rootnode1 ~]# ll .ssh total 8 -rw-------. 1 root root 1675 Aug 22 09:06 id_rsa -rw-r--r--. 1 root root 392 Aug 22 09:06 id_rsa.pubcopy公钥到各个节点以本机为例执行过程如下[rootnode1 ~]# ssh-copy-id node1 /usr/bin/ssh-copy-id: INFO: Source of key(s) to be installed: /root/.ssh/id_rsa.pub The authenticity of host node1 (192.168.5.10) cant be established. ECDSA key fingerprint is SHA256:WoqiUZLXfcdFxBBkVaC/hJybMxqwqwvuHCR1seMkLE. ECDSA key fingerprint is MD5:f5:9e:e3:ba:f4:15:4d:aa:96:16:bc:df:4c:05:98:ac. Are you sure you want to continue connecting (yes/no)? yes /usr/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to filter out any that are already installed /usr/bin/ssh-copy-id: INFO: 1 key(s) remain to be installed -- if you are prompted now it is to install the new keys rootnode1s password: Permission denied, please try again. rootnode1s password: # 首次需要输入密码 Number of key(s) added: 1 Now try logging into the machine, with: ssh node1 and check to make sure that only the key(s) you wanted were added.验证免密登录[rootnode1 ~]# ssh node2 Last login: Sun Aug 20 21:22:36 2023 from 192.168.5.117远程批量操作pssh工具开源工具pssh工具可用于在多台服务器上执行远程批量操作命令大大简化了集群管理提升了效率和准确性。node1节点安装pssh工具包# 下载 https://pypi.python.org/packages/source/p/pssh/pssh-2.3.1.tar.gz # 解压 tar xvf pssh-2.3.1.tar.gz # 安装 python setup.py install # 验证 pssh --version执行命令前务必提前配置好ssh免密登录验证如下[rootnode1 ~]# pssh -H rootnode2 date [1] 10:43:30 [SUCCESS] rootnode2创建nodes文件配置好所有节点[rootnode1 ~]# vim nodes rootnode1 rootnode2 rootnode3关闭防火墙pssh批量禁用和关闭所有节点防火墙[rootnode1 ~]# pssh -h ./nodes -i systemctl disable firewalld [1] 10:50:18 [SUCCESS] rootnode1 [2] 10:50:18 [SUCCESS] rootnode2 [3] 10:50:18 [SUCCESS] rootnode3 [rootnode1 ~]# pssh -h ./nodes -i systemctl stop firewalld [1] 10:49:59 [SUCCESS] rootnode1 [2] 10:49:59 [SUCCESS] rootnode2 [3] 10:49:59 [SUCCESS] rootnode3禁用SELinuxpssh批量修改所有节点的/etc/sysconfig/selinux永久禁用pssh -h ./nodes -i sed -i s/SELINUXenforcing/SELINUXdisabled/ /etc/sysconfig/selinuxpssh批量验证修改是否成功[rootnode1 ~]# pssh -h ./nodes -i cat /etc/sysconfig/selinux | grep SELINUX [1] 11:05:38 [SUCCESS] rootnode1 # SELINUX can take one of these three values: SELINUXdisabled [2] 11:05:38 [SUCCESS] rootnode2 # SELINUX can take one of these three values: SELINUXdisabled [3] 11:05:38 [SUCCESS] rootnode3 # SELINUX can take one of these three values: SELINUXdisabled配置swap内核参数swappiness用于控制系统在内存紧张时将内存中的数据交换到硬盘交换空间swap中 一般而言swappiness值可以设置在 0 到 100 之间。内核版本3.5及以上、Red Hat内核版本2.6.32-303及以上推荐设置一个比较小的值而不要禁用交换。pssh批量给所有节点添加vm.swappiness1重启机器永久生效pssh -h ./nodes -i echo vm.swappiness 1 /etc/sysctl.conf # 永久生效配置 pssh -h ./nodes -i sysctl -p # 动态加载使当前会话生效验证是否添加成功[rootnode1 ~]# pssh -h ./nodes -i cat /etc/sysctl.conf | grep vm.swappiness [1] 11:26:41 [SUCCESS] rootnode1 vm.swappiness 1 [2] 11:26:41 [SUCCESS] rootnode2 vm.swappiness 1 [3] 11:26:41 [SUCCESS] rootnode3 vm.swappiness 1关闭透明大页[rootnode1 log]# vim /etc/rc.d/rc.local if test -f /sys/kernel/mm/transparent_hugepage/enabled; then echo never /sys/kernel/mm/transparent_hugepage/enabled fi if test -f /sys/kernel/mm/transparent_hugepage/defrag; then echo never /sys/kernel/mm/transparent_hugepage/defrag fiRAC环境下透明大页Transparent HugePages )会导致异常节点重启和性能问题即使单机环境也可能有性能问题总之不建议启用透明大页。注意如果不关闭透明大页CM Server安装过程中检查主机项还会显示黄色告警。临时有效解决方法pssh批量关闭透明大页[rootnode1 ~]# pssh -h ./nodes -i echo never /sys/kernel/mm/transparent_hugepage/defrag [1] 11:38:37 [SUCCESS] rootnode1 [2] 11:38:37 [SUCCESS] rootnode2 [3] 11:38:37 [SUCCESS] rootnode3 [rootnode1 ~]# pssh -h ./nodes -i echo never /sys/kernel/mm/transparent_hugepage/enabled [1] 11:38:39 [SUCCESS] rootnode1 [2] 11:38:39 [SUCCESS] rootnode3 [3] 11:38:39 [SUCCESS] rootnode2永久有效解决方法[rootnode1 log]# vim /etc/rc.d/rc.local if test -f /sys/kernel/mm/transparent_hugepage/enabled; then echo never /sys/kernel/mm/transparent_hugepage/enabled fi if test -f /sys/kernel/mm/transparent_hugepage/defrag; then echo never /sys/kernel/mm/transparent_hugepage/defrag fi [rootnode1 log]# chmod x /etc/rc.d/rc.local验证是否关闭成功[rootnode1 ~]# pssh -h ./nodes -i cat /sys/kernel/mm/transparent_hugepage/defrag;cat /sys/kernel/mm/transparent_hugepage/enabled [1] 10:14:28 [SUCCESS] rootnode2 always madvise [never] always madvise [never] [2] 10:14:28 [SUCCESS] rootnode3 always madvise [never] always madvise [never] [3] 10:14:28 [SUCCESS] rootnode1 always madvise [never] always madvise [never]配置最大进程数和最大文件句柄数用户的进程和文件句柄数配置文件是/etc/security/limits.conf类型包括硬限制(hard)和软限制(soft)*表示所有用户nproc表示打开最大进程数nofile表示最大文件句柄数。Cloudera推荐最大进程数nproc为65536或26214。pssh批量修改所有用户的进程和文件句柄最大限制pssh -h ./nodes -i echo * soft nproc 65535 /etc/security/limits.conf pssh -h ./nodes -i echo * hard nproc 65535 /etc/security/limits.conf pssh -h ./nodes -i echo * soft nofile 65535 /etc/security/limits.conf pssh -h ./nodes -i echo * hard nofile 65535 /etc/security/limits.conf禁用atimeLinux文件系统会保留记录每个文件访问时间的元数据这可能会导致即使是读取操作也会导致磁盘写入。为了提高文件读取性能Cloudera建议在/etc/fstab配置文件中使用noatime挂载选项来禁用atime选项。pssh批量禁用文件最后访问时间本机的挂载数据目录是/data修改如下# 数据目录添加noatime pssh -h ./nodes -i sed -i s/\/data ext4 defaults/\/data ext4 defaults,noatime/ /etc/fstab # 重新挂载不需要重启节点 mount -o remount /data集群时钟同步Centos7默认使用Chrony工具而非NTP进行时间同步Chrony和NTP两者有冲突只能选其一CDH6推荐使用Chrony安装和启动命令如下# 安装 yum -y install chrony # 启动 systemctl start chronyd如果已安装可跳过修改硬件时钟为UTC时区为本地时区# 硬件时钟设置为UTC pssh -h ./nodes -i timedatectl set-local-rtc 0 # 设置本地时区显示本地时间 pssh -h ./nodes -i timedatectl set-timezone Asia/Shanghai # 手动加载RTC设置 pssh -h ./nodes -i hwclock --systohc通过timedatectl查看时间[rootnode1 ~]# timedatectl Local time: Tue 2023-08-22 15:14:57 CST Universal time: Tue 2023-08-22 07:14:57 UTC RTC time: Tue 2023-08-22 07:14:57 Time zone: Asia/Shanghai (CST, 0800) NTP enabled: yes NTP synchronized: yes RTC in local TZ: no DST active: n/a安装JDKCloudera推荐安装JDK1.8u181版本打开CDH安装包oracle-j2sdk1.8-1.8.0update181-1.x86_64.rpm包已经放在了JDK目录中在所有节点上安装该版本JDK安装配置比较简单请移步我的另一篇文章Centos7安装64位JDK1.8参考”使用rpm安装“内容执行安装。Centos7安装64位JDK1.8安装MySQL部署MySQLnode1节点安装部署MySQL5.7服务器具体安装步骤请移步我的另一篇文章Linux安装和卸载MySQL5.7详细教程。MySQL配置MySQL的配置文件是/etc/my.cnf, Cloudera的推荐配置如下注意此处有坑对于MySQL5.7高版本启动CM Server连接MySQL时可能会报com.mysql.jdbc.exceptions.jdbc4.CommunicationsException异常原因是默认开启了SSL认证相当于useSSLtrue解决方法就是在配置中添加skip-ssl重启MySQL服务另外[mysqld_safe]配置项已废弃不用以下的配置均已经过验证。[mysqld] # 务必禁用掉ssl相当于useSSLfalse skip-ssl datadir/var/lib/mysql socket/var/lib/mysql/mysql.sock # transaction-isolation READ-COMMITTED # Disabling symbolic-links is recommended to prevent assorted security risks; # to do so, uncomment this line: symbolic-links 0 key_buffer_size 32M max_allowed_packet 16M thread_stack 256K thread_cache_size 64 query_cache_limit 8M query_cache_size 64M query_cache_type 1 # 设置数据库错误日志时间显示为系统时间 log_timestampssystem # 设置安全策略 validate_password_policyLOW validate_password_length4 validate_password_mixed_case_count1 validate_password_number_count1 validate_password_special_char_count1 max_connections 550 #expire_logs_days 10 #max_binlog_size 100M #log_bin should be on a disk with enough free space. #Replace /var/lib/mysql/mysql_binary_log with an appropriate path for your #system and chown the specified folder to the mysql user. log_bin/var/lib/mysql/mysql_binary_log #In later versions of MySQL, if you enable the binary log and do not set #a server_id, MySQL will not start. The server_id must be unique within #the replicating group. server_id1 binlog_format mixed read_buffer_size 2M read_rnd_buffer_size 16M sort_buffer_size 8M join_buffer_size 8M # InnoDB settings innodb_file_per_table 1 innodb_flush_log_at_trx_commit 2 innodb_log_buffer_size 64M innodb_buffer_pool_size 4G innodb_thread_concurrency 8 innodb_flush_method O_DIRECT innodb_log_file_size 512M # [mysqld_safe] 5.7已废弃该配置 log-error/var/log/mysqld.log pid-file/var/run/mysqld/mysqld.pid sql_modeSTRICT_ALL_TABLES重启MySQL服务器使配置生效systemctl restart mysqld初始化数据库及用户Cloudera官网要求初始化的数据库及用户如下创建以上所有组件数据库执行以下脚本[rootnode1 ~]# mysql -uroot -p # scm CREATE DATABASE scm DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci; GRANT ALL ON scm.* TO scm% IDENTIFIED BY abcd1234; # amon CREATE DATABASE amon DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci; GRANT ALL ON amon.* TO amon% IDENTIFIED BY abcd1234; # rman CREATE DATABASE rman DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci; GRANT ALL ON rman.* TO rman% IDENTIFIED BY abcd1234; # hue CREATE DATABASE hue DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci; GRANT ALL ON hue.* TO hue% IDENTIFIED BY abcd1234; # hive CREATE DATABASE metastore DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci; GRANT ALL ON metastore.* TO hive% IDENTIFIED BY abcd1234; # sentry CREATE DATABASE sentry DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci; GRANT ALL ON sentry.* TO sentry% IDENTIFIED BY abcd1234; # nav CREATE DATABASE nav DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci; GRANT ALL ON nav.* TO nav% IDENTIFIED BY abcd1234; # navms CREATE DATABASE navms DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci; GRANT ALL ON navms.* TO navms% IDENTIFIED BY abcd1234; # oozie CREATE DATABASE oozie DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci; GRANT ALL ON oozie.* TO oozie% IDENTIFIED BY abcd1234; # 最后刷新一下权限 flush privileges; # 检查 show databases;安装JDBC驱动程序用于各节点连接数据库Cloudera要求必须创建在/usr/share/java目录下才能被组件使用。# 创建目录 pssh -h ./nodes -i mkdir -p /usr/share/java/ # 复制到各个节点 pscp -h ./nodes /var/www/html/cdh/mysql/mysql-connector-java-5.1.46-bin.jar /usr/share/java/ # 修改权限 pssh -h ./nodes -i cd /usr/share/java;chmod 777 mysql-connector-java-5.1.46-bin.jar # 重命名jar包以便找到 pssh -h ./nodes -i cd /usr/share/java;ln -s mysql-connector-java-5.1.46-bin.jar mysql-connector-java.jar # 验证是否创建成功 pssh -h ./nodes -i tree /usr/share/java搭建私有Yum源制作CDH和CM的本地repo源方便内网集群节点访问。安装httpd服务node1节点安装Apache httpd服务用于后续通过Http方式访问和下载CDH和CM安装包。# 安装httpd和createrepo yum -y install httpd createrepo # 启动 systemctl start httpd # 开机启动 systemctl enable httpd注意httpd服务在传输parcel文件时默认会设置“Content-Encoding”导致CM Server认为parcel文件已经被httpd压缩并尝试解压缩结果会报错解决办法是修改httpd的配置文件。修改配置文件 /etc/httpd/conf/httpd.conf在IfModule mime_module中添加支持.parcel文件类型用于远程下载parcel文件。[rootnode1 ~]# vim /etc/httpd/conf/httpd.conf IfModule mime_module AddType application/x-compress .Z AddType application/x-gzip .gz .tgz .parcel AddType text/html .shtml AddOutputFilter INCLUDES .shtml /IfModule重启httpd服务systemctl restart httpd配置web服务node1节点移动cdh到httpd服务的根目录/var/www/html下生成repo元数据用于通过Yum访问。[rootnode1 ~]# mv /opt/apps/cdh /var/www/html [rootnode1 ~]# tree /var/www/html /var/www/html └── cdh ├── CDH6.3.2 │ ├── CDH-6.3.2-1.cdh6.3.2.p0.1605554-el7.parcel │ ├── CDH-6.3.2-1.cdh6.3.2.p0.1605554-el7.parcel.sha1 │ ├── CDH-6.3.2-1.cdh6.3.2.p0.1605554-el7.parcel.sha256 │ └── manifest.json ├── CM6.3.1 │ ├── allkeys.asc │ ├── cloudera-manager-agent-6.3.1-1466458.el7.x86_64.rpm │ ├── cloudera-manager-daemons-6.3.1-1466458.el7.x86_64.rpm │ ├── cloudera-manager-server-6.3.1-1466458.el7.x86_64.rpm │ ├── cloudera-manager-server-db-2-6.3.1-1466458.el7.x86_64.rpm │ └── enterprise-debuginfo-6.3.1-1466458.el7.x86_64.rpm ├── JDK │ └── oracle-j2sdk1.8-1.8.0update181-1.x86_64.rpm └── mysql ├── 01_mysql-community-common-5.7.16-1.el7.x86_64.rpm ├── 02_mysql-community-libs-5.7.16-1.el7.x86_64.rpm ├── 03_mysql-community-libs-compat-5.7.16-1.el7.x86_64.rpm ├── 04_mysql-community-client-5.7.16-1.el7.x86_64.rpm ├── 05_mysql-community-server-5.7.16-1.el7.x86_64.rpm └── mysql-connector-java-5.1.46-bin.jar 5 directories, 17 files [rootnode1 ~]# cd /var/www/html/cdh/CM6.3.1/ [rootnode1 CM6.3.1]# createrepo .验证http访问制作本地repo源在/etc/yum.repos.d目录下创建cm-local.repo配置文件添加如下内容[rootnode1 yum.repos.d]# vim cm-local.repo [cm-repo] namecm-repo baseurlhttp://192.168.5.10/cdh/CM6.3.1 enabled1 # 启用yum gpgcheck0 # 不检查检查gpg复制到集群其他节点执行如下操作# 远程复制 pscp -h ./nodes /etc/yum.repos.d/cm-local.repo /etc/yum.repos.d/ # 先清空再添加缓存 pssh -h ./nodes -i yum clean all yum makecache # 查看当前所有yum源 pssh -h ./nodes -i yum repolist三、CM6.3.1安装步骤集群角色集群CM角色序号节点CM服务角色1node1cloudera-scm-server cloudera-scm-agent2node2cloudera-scm-agent3node3cloudera-scm-agent安装CM6.3.1安装cloudera manager server首先node1节点安装cloudera manager server方便后续通过Web UI安装CDH。yum -y install cloudera-manager-server初始化scm数据库[rootnode1 ~]# /opt/cloudera/cm/schema/scm_prepare_database.sh mysql scm scm abcd1234 JAVA_HOME/usr/java/jdk1.8.0_181-cloudera Verifying that we can write to /etc/cloudera-scm-server Creating SCM configuration file in /etc/cloudera-scm-server Executing: /usr/java/jdk1.8.0_181-cloudera/bin/java -cp /usr/share/java/mysql-connector-java.jar:/usr/share/java/oracle-connector-java.jar:/usr/share/java/postgresql-connector-java.jar:/opt/cloudera/cm/schema/../lib/* com.cloudera.enterprise.dbutil.DbCommandExecutor /etc/cloudera-scm-server/db.properties com.cloudera.cmf.db. Tue Aug 22 22:00:51 CST 2023 WARN: Establishing SSL connection without servers identity verification is not recommended. According to MySQL 5.5.45, 5.6.26 and 5.7.6 requirements SSL connection must be established by default if explicit option isnt set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to false. You need either to explicitly disable SSL by setting useSSLfalse, or set useSSLtrue and provide truststore for server certificate verification. [ main] DbCommandExecutor INFO Successfully connected to database. All done, your SCM database is configured correctly!启动cloudera manager server# 启动 systemctl start cloudera-scm-server # 开机启动 systemctl enable cloudera-scm-server # 查看状态 systemctl status cloudera-scm-server查看启动日志/var/log/cloudera-scm-server/cloudera-scm-server.log无异常表示启动成功。[rootnode1 ~]# tail -3 /var/log/cloudera-scm-server/cloudera-scm-server.log 2023-08-22 22:11:22,656 INFO WebServerImpl:org.eclipse.jetty.server.AbstractConnector: Started ServerConnector30096cd2{HTTP/1.1,[http/1.1]}{0.0.0.0:7180} 2023-08-22 22:11:22,656 INFO WebServerImpl:org.eclipse.jetty.server.Server: Started 39407ms 2023-08-22 22:11:22,656 INFO WebServerImpl:com.cloudera.server.cmf.WebServerImpl: Started Jetty server.四、CDH集群6.3.2安装步骤集群安装向导浏览器地址栏输入http://192.168.5.10:7180/cmf/login打开登录Web页面。使用默认用户名/密码admin/admin登录CM进入欢迎页面点击“继续”。勾选接受许可条款和条件点击“继续”。选择Cloudera Express免费版本点击“继续”。进入集群安装页面点击“继续”。编辑集群名称填入cdh6.3.2安装成功后可以重命名集群点击“继续”。使用模式输入主机名称或者IP地址点击“搜索”找到主机后勾选节点点击“继续”。Cloudera Manager Agent的自定义存储库输入CM仓库地址:http://192.168.5.10/cdh/CM6.3.1。CDH and other software选择“使用 Parcel (建议)”点击“更多选项”进入Parcel存储库设置页面。“Parcel目录”和“本地Parcel存储库路径”不用编辑选择默认目录就好编辑“远程Parcel存储库URL”输入CDH仓库地址: http://192.168.5.10/cdh/CDH6.3.2/点击“-”删除其它URL点击“保存更改”。上一步保存后此处的CDH版本会自动刷新点击“继续”。安装JDK如果已安装请跳过点击“继续”。输入root用户的密码用于SSH登录点击“继续”。安装Agent到集群各节点点击“详细信息”可查看安装过程。安装完毕自动跳转到下一步开始下载并安装Parcels。点击“Inspect Network Performance”和“Inspect Hosts”检查网络和主机性能。如果出现黄色告警查看“显示检查器结果”逐项解决后点击“重新运行”直到所有的检查都通过方可点击“继续”也可以直接选择最后一项跳过告警点击“继续”。集群安装设置进入集群设置页面选择需要安装的服务组合。我这里选择自定义服务选择自己需要的组件点击“继续”。进入集群自定义角色分配根据自己的规划给节点分配角色。配置完成点击“按主机查看”点击“继续”进入数据库设置页面选择数据库类型MySQL、输入主机名称、数据库、用户及密码点击“测试连接”测试成功后点击“继续”。进入审核更改页面检查默认的目录设置是否合适可根据实际情况调整确认无误后点击“继续”。开始运行命令进行各个组件安装完成后点击“继续”。进入汇总页面表示集群组件安装成功并启动点“完成”。最后进入CM管理主页至此CDH6.3.2安装成功 CDH集群6.3.2 + CM6.3.1离线安装部署实操保姆级详细教程
目录一、序CDH简介CM简介为什么选择CDH而不是Apache HadoopCDH安装说明二、准备工作前置条件测试服务器准备软件环境准备CDH安装包准备查看CDH6.3.2安装包查看CM6.3.1安装包查看JDK1.8安装包环境配置配置静态网络IP修改主机名修改hosts配置配置SSH免密登录创建密钥对查看密钥对验证免密登录远程批量操作pssh工具关闭防火墙禁用SELinux配置swap关闭透明大页配置最大进程数和最大文件句柄数禁用atime集群时钟同步安装JDK安装MySQL部署MySQLMySQL配置初始化数据库及用户安装JDBC驱动程序搭建私有Yum源安装httpd服务配置web服务制作本地repo源三、CM6.3.1安装步骤集群角色安装CM6.3.1安装cloudera manager server初始化scm数据库启动cloudera manager server四、CDH集群6.3.2安装步骤集群安装向导集群安装设置一、序官方文档Cloudera Enterprise 6.3 DocumentationCDH简介CDH是一个基于Apache Hadoop的开源软件分发版本由Cloudera公司开发和维护。CDH提供了统一批处理、交互式SQL和交互式搜索以及基于角色的访问控制的Hadoop解决方案组件包括Hadoop分布式文件系统HDFS、分布式计算框架MapReduce及其他相关开源组件如Hive、HBase、Impala等。CDH的目标是简化大数据处理和分析的过程使企业能够更好地利用大数据来获取洞察力和价值。CM简介CM是Cloudera提供的一个集中式管理工具用于管理和监控CDH集群。CM提供了直观的Web界面管理员可以使用该界面配置、部署和监控CDH集群。CM还提供了一些高级功能如自动化部署、故障诊断和报警通知来帮助管理员更好地管理和维护CDH集群。通过CM管理员可以轻松管理集群的配置、监控集群的健康状况并执行集群维护任务和性能调优等操作。为什么选择CDH而不是Apache HadoopCDH和Apache Hadoop比较特点CDHApache Hadoop功能全面性提供了广泛的组件和工具如Hive、HBase、Impala等支持不同的大数据处理和分析需求。提供核心的分布式存储HDFS和计算MapReduce功能需要使用其他组件或工具进行更高级的数据处理和分析。管理工具包含Cloudera ManagerCM提供集中式的管理和监控功能支持自动化部署、故障诊断和报警通知等。Apache Hadoop原生没有提供类似的集中式管理工具通常需要使用其他第三方工具进行管理和监控。商业支持Cloudera提供商业支持和服务包括技术支持、培训和咨询等。Apache Hadoop是一个开源项目没有官方提供的商业支持但有许多第三方公司提供相关支持和服务。发展速度CDH的发展速度相对较快并且每个版本都经过Cloudera的测试和验证。Apache Hadoop作为一个开源项目有更多社区贡献者因此发展速度更快有更频繁的更新和新特性引入。综上所述CDH提供了更全面和更好用的功能及配套管理工具使大数据处理和分析更简化和集中化。一般来说集群规模越大对集群的管理和运维能力要求越高此时CDH能够发挥的价值就会越大。CDH安装说明Cloudera企业版收费不差钱又缺技术支持的可以考虑购买Cloudera社区版开源免费社区文档和资源也比较丰富适合小规模集群部署不过根据Cloudera官方发布的公告自2021年7月起Cloudera社区版已经停止更新和发布CDH 6.3.2将是Cloudera社区版的最后一个免费版本。而且从2021年2月1日开始所有的CDH和Cloudera Manager包括历史版本下载都需要用户名和密码就是用户需要订阅 Cloudera 产品获取到对应的授权才能下载。所以我们将使用离线包进行安装部署本文以CDH6.3.2CM6.3.1为例详细介绍大数据集群的安装部署过程供大家参考文中离线安装包也会无偿分享给大家下载使用老铁们码字不易记得点赞收藏关注咱不迷路二、准备工作前置条件测试服务器准备本文以3台测试服务器为例以root用户身份登录节点配置如下测试服务器配置信息序号主机IPCPU内存磁盘hdfs角色1node1192.168.5.1016C40G512G系统盘(xfs)1T数据盘(ext4)namenode1/datanode2node2192.168.5.1116C40G512G系统盘(xfs)1T数据盘(ext4)namenode2/datanode3node3192.168.5.1216C40G512G系统盘(xfs)1T数据盘(ext4)datanode软件环境准备基础软件环境信息序号软件名称版本信息备注1Centos操作系统发行版CentOS Linux release 7.9.2009 x86_64CDH要求7.2或更高2MySQL数据库5.7.43 x86_64CDH6推荐5.7或更高务必使用utf8编码而非utf8mb4编码3JDK1.8.0_181 x86_64CDH6将不再支持JDK1.7推荐1.8u1814python2.7.55CDH6.3.26CM6.3.1CDH安装包准备CDH、CM和JDK安装包需要提前下载目前官方不提供免费下载我已经上传到百度网盘方便大家使用链接地址:CDH6.3.2https://pan.baidu.com/s/1hKuldVcZT2HHGcBEF5VWuQ需要安装包的童鞋请留言或私信我下载网盘里的cdh文件夹并保存到本地。查看CDH6.3.2安装包[rootnode1 cdh]# tree CDH6.3.2/ CDH6.3.2/ ├── CDH-6.3.2-1.cdh6.3.2.p0.1605554-el7.parcel ├── CDH-6.3.2-1.cdh6.3.2.p0.1605554-el7.parcel.sha1 ├── CDH-6.3.2-1.cdh6.3.2.p0.1605554-el7.parcel.sha256 └── manifest.json 0 directories, 4 files查看CM6.3.1安装包[rootnode1 cdh]# tree CM6.3.1 CM6.3.1 ├── allkeys.asc ├── cloudera-manager-agent-6.3.1-1466458.el7.x86_64.rpm ├── cloudera-manager-daemons-6.3.1-1466458.el7.x86_64.rpm ├── cloudera-manager-server-6.3.1-1466458.el7.x86_64.rpm ├── cloudera-manager-server-db-2-6.3.1-1466458.el7.x86_64.rpm └── enterprise-debuginfo-6.3.1-1466458.el7.x86_64.rpm 0 directories, 6 files查看JDK1.8安装包[rootnode1 cdh]# tree JDK JDK └── oracle-j2sdk1.8-1.8.0update181-1.x86_64.rpm 0 directories, 1 file环境配置配置静态网络IPCentos7静态网络配置文件是/etc/sysconfig/network-scripts/ifcfg-interface,我本机网络接口名称为p2p1修改所有节点网络配置具体参数如下[rootnode1 ~]# vim /etc/sysconfig/network-scripts/ifcfg-p2p1 TYPEEthernet PROXY_METHODnone BROWSER_ONLYno BOOTPROTOnone DEFROUTEyes IPV4_FAILURE_FATALyes IPV6INITyes IPV6_AUTOCONFyes IPV6_DEFROUTEyes IPV6_FAILURE_FATALno IPV6_ADDR_GEN_MODEstable-privacy NAMEp2p1 # 网络接口名称 UUID31b15de1-225f-4540-b135-b80d5da6f335 DEVICEp2p1 ONBOOTyes # 设置开机启动确保网络配置有效否则无法建立网络连接 IPADDR192.168.5.10 # 静态IP地址 PREFIX24 # 子网掩码相当于255.255.255.0 GATEWAY192.168.5.1 # 默认网关IP地址 DNS1192.168.1.1 # 首选DNS服务器IP地址上网必备 IPV6_PRIVACYno # 禁用IPV6重启网络可立即生效systemctl restart network修改主机名修改所有节点的hostname重启节点立即生效hostnamectl set-hostname node1 hostnamectl set-hostname node2 hostnamectl set-hostname node3修改hosts配置配置所有节点的IP和域名映射关系# vim /etc/hosts 192.168.5.10 node1 192.168.5.11 node2 192.168.5.12 node3配置SSH免密登录ssh-keygen用于生成和管理 SSH 密钥对SSH 密钥对由公钥和私钥组成公钥用于在 SSH 认证过程中提供给远程服务器进行安全认证通常会添加到远程服务器的authorized_keys文件中而私钥则默认保存在本地root用户默认将私钥对保存在/root/.ssh目录下。创建密钥对node1和node2节点上生成ssh密钥对执行ssh-keygen命令过程如下# 一路回车生成密钥对 [rootnode1 ~]# ssh-keygen -t rsa Generating public/private rsa key pair. Enter file in which to save the key (/root/.ssh/id_rsa): Enter passphrase (empty for no passphrase): Enter same passphrase again: Your identification has been saved in /root/.ssh/id_rsa. Your public key has been saved in /root/.ssh/id_rsa.pub. The key fingerprint is: SHA256:AJWzRl9kwU3kbQ7C518oIKCmQdpvKNeQxiIvxo4ly/w rootnode1 The keys randomart image is: ---[RSA 2048]---- | . ..o. oo | | . o. oo... | | B o.... o | |o O o... . | |o* . S o o .| |*B . o . | |o . | | . | | E | ----[SHA256]-----查看密钥对[rootnode1 ~]# ll .ssh total 8 -rw-------. 1 root root 1675 Aug 22 09:06 id_rsa -rw-r--r--. 1 root root 392 Aug 22 09:06 id_rsa.pubcopy公钥到各个节点以本机为例执行过程如下[rootnode1 ~]# ssh-copy-id node1 /usr/bin/ssh-copy-id: INFO: Source of key(s) to be installed: /root/.ssh/id_rsa.pub The authenticity of host node1 (192.168.5.10) cant be established. ECDSA key fingerprint is SHA256:WoqiUZLXfcdFxBBkVaC/hJybMxqwqwvuHCR1seMkLE. ECDSA key fingerprint is MD5:f5:9e:e3:ba:f4:15:4d:aa:96:16:bc:df:4c:05:98:ac. Are you sure you want to continue connecting (yes/no)? yes /usr/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to filter out any that are already installed /usr/bin/ssh-copy-id: INFO: 1 key(s) remain to be installed -- if you are prompted now it is to install the new keys rootnode1s password: Permission denied, please try again. rootnode1s password: # 首次需要输入密码 Number of key(s) added: 1 Now try logging into the machine, with: ssh node1 and check to make sure that only the key(s) you wanted were added.验证免密登录[rootnode1 ~]# ssh node2 Last login: Sun Aug 20 21:22:36 2023 from 192.168.5.117远程批量操作pssh工具开源工具pssh工具可用于在多台服务器上执行远程批量操作命令大大简化了集群管理提升了效率和准确性。node1节点安装pssh工具包# 下载 https://pypi.python.org/packages/source/p/pssh/pssh-2.3.1.tar.gz # 解压 tar xvf pssh-2.3.1.tar.gz # 安装 python setup.py install # 验证 pssh --version执行命令前务必提前配置好ssh免密登录验证如下[rootnode1 ~]# pssh -H rootnode2 date [1] 10:43:30 [SUCCESS] rootnode2创建nodes文件配置好所有节点[rootnode1 ~]# vim nodes rootnode1 rootnode2 rootnode3关闭防火墙pssh批量禁用和关闭所有节点防火墙[rootnode1 ~]# pssh -h ./nodes -i systemctl disable firewalld [1] 10:50:18 [SUCCESS] rootnode1 [2] 10:50:18 [SUCCESS] rootnode2 [3] 10:50:18 [SUCCESS] rootnode3 [rootnode1 ~]# pssh -h ./nodes -i systemctl stop firewalld [1] 10:49:59 [SUCCESS] rootnode1 [2] 10:49:59 [SUCCESS] rootnode2 [3] 10:49:59 [SUCCESS] rootnode3禁用SELinuxpssh批量修改所有节点的/etc/sysconfig/selinux永久禁用pssh -h ./nodes -i sed -i s/SELINUXenforcing/SELINUXdisabled/ /etc/sysconfig/selinuxpssh批量验证修改是否成功[rootnode1 ~]# pssh -h ./nodes -i cat /etc/sysconfig/selinux | grep SELINUX [1] 11:05:38 [SUCCESS] rootnode1 # SELINUX can take one of these three values: SELINUXdisabled [2] 11:05:38 [SUCCESS] rootnode2 # SELINUX can take one of these three values: SELINUXdisabled [3] 11:05:38 [SUCCESS] rootnode3 # SELINUX can take one of these three values: SELINUXdisabled配置swap内核参数swappiness用于控制系统在内存紧张时将内存中的数据交换到硬盘交换空间swap中 一般而言swappiness值可以设置在 0 到 100 之间。内核版本3.5及以上、Red Hat内核版本2.6.32-303及以上推荐设置一个比较小的值而不要禁用交换。pssh批量给所有节点添加vm.swappiness1重启机器永久生效pssh -h ./nodes -i echo vm.swappiness 1 /etc/sysctl.conf # 永久生效配置 pssh -h ./nodes -i sysctl -p # 动态加载使当前会话生效验证是否添加成功[rootnode1 ~]# pssh -h ./nodes -i cat /etc/sysctl.conf | grep vm.swappiness [1] 11:26:41 [SUCCESS] rootnode1 vm.swappiness 1 [2] 11:26:41 [SUCCESS] rootnode2 vm.swappiness 1 [3] 11:26:41 [SUCCESS] rootnode3 vm.swappiness 1关闭透明大页[rootnode1 log]# vim /etc/rc.d/rc.local if test -f /sys/kernel/mm/transparent_hugepage/enabled; then echo never /sys/kernel/mm/transparent_hugepage/enabled fi if test -f /sys/kernel/mm/transparent_hugepage/defrag; then echo never /sys/kernel/mm/transparent_hugepage/defrag fiRAC环境下透明大页Transparent HugePages )会导致异常节点重启和性能问题即使单机环境也可能有性能问题总之不建议启用透明大页。注意如果不关闭透明大页CM Server安装过程中检查主机项还会显示黄色告警。临时有效解决方法pssh批量关闭透明大页[rootnode1 ~]# pssh -h ./nodes -i echo never /sys/kernel/mm/transparent_hugepage/defrag [1] 11:38:37 [SUCCESS] rootnode1 [2] 11:38:37 [SUCCESS] rootnode2 [3] 11:38:37 [SUCCESS] rootnode3 [rootnode1 ~]# pssh -h ./nodes -i echo never /sys/kernel/mm/transparent_hugepage/enabled [1] 11:38:39 [SUCCESS] rootnode1 [2] 11:38:39 [SUCCESS] rootnode3 [3] 11:38:39 [SUCCESS] rootnode2永久有效解决方法[rootnode1 log]# vim /etc/rc.d/rc.local if test -f /sys/kernel/mm/transparent_hugepage/enabled; then echo never /sys/kernel/mm/transparent_hugepage/enabled fi if test -f /sys/kernel/mm/transparent_hugepage/defrag; then echo never /sys/kernel/mm/transparent_hugepage/defrag fi [rootnode1 log]# chmod x /etc/rc.d/rc.local验证是否关闭成功[rootnode1 ~]# pssh -h ./nodes -i cat /sys/kernel/mm/transparent_hugepage/defrag;cat /sys/kernel/mm/transparent_hugepage/enabled [1] 10:14:28 [SUCCESS] rootnode2 always madvise [never] always madvise [never] [2] 10:14:28 [SUCCESS] rootnode3 always madvise [never] always madvise [never] [3] 10:14:28 [SUCCESS] rootnode1 always madvise [never] always madvise [never]配置最大进程数和最大文件句柄数用户的进程和文件句柄数配置文件是/etc/security/limits.conf类型包括硬限制(hard)和软限制(soft)*表示所有用户nproc表示打开最大进程数nofile表示最大文件句柄数。Cloudera推荐最大进程数nproc为65536或26214。pssh批量修改所有用户的进程和文件句柄最大限制pssh -h ./nodes -i echo * soft nproc 65535 /etc/security/limits.conf pssh -h ./nodes -i echo * hard nproc 65535 /etc/security/limits.conf pssh -h ./nodes -i echo * soft nofile 65535 /etc/security/limits.conf pssh -h ./nodes -i echo * hard nofile 65535 /etc/security/limits.conf禁用atimeLinux文件系统会保留记录每个文件访问时间的元数据这可能会导致即使是读取操作也会导致磁盘写入。为了提高文件读取性能Cloudera建议在/etc/fstab配置文件中使用noatime挂载选项来禁用atime选项。pssh批量禁用文件最后访问时间本机的挂载数据目录是/data修改如下# 数据目录添加noatime pssh -h ./nodes -i sed -i s/\/data ext4 defaults/\/data ext4 defaults,noatime/ /etc/fstab # 重新挂载不需要重启节点 mount -o remount /data集群时钟同步Centos7默认使用Chrony工具而非NTP进行时间同步Chrony和NTP两者有冲突只能选其一CDH6推荐使用Chrony安装和启动命令如下# 安装 yum -y install chrony # 启动 systemctl start chronyd如果已安装可跳过修改硬件时钟为UTC时区为本地时区# 硬件时钟设置为UTC pssh -h ./nodes -i timedatectl set-local-rtc 0 # 设置本地时区显示本地时间 pssh -h ./nodes -i timedatectl set-timezone Asia/Shanghai # 手动加载RTC设置 pssh -h ./nodes -i hwclock --systohc通过timedatectl查看时间[rootnode1 ~]# timedatectl Local time: Tue 2023-08-22 15:14:57 CST Universal time: Tue 2023-08-22 07:14:57 UTC RTC time: Tue 2023-08-22 07:14:57 Time zone: Asia/Shanghai (CST, 0800) NTP enabled: yes NTP synchronized: yes RTC in local TZ: no DST active: n/a安装JDKCloudera推荐安装JDK1.8u181版本打开CDH安装包oracle-j2sdk1.8-1.8.0update181-1.x86_64.rpm包已经放在了JDK目录中在所有节点上安装该版本JDK安装配置比较简单请移步我的另一篇文章Centos7安装64位JDK1.8参考”使用rpm安装“内容执行安装。Centos7安装64位JDK1.8安装MySQL部署MySQLnode1节点安装部署MySQL5.7服务器具体安装步骤请移步我的另一篇文章Linux安装和卸载MySQL5.7详细教程。MySQL配置MySQL的配置文件是/etc/my.cnf, Cloudera的推荐配置如下注意此处有坑对于MySQL5.7高版本启动CM Server连接MySQL时可能会报com.mysql.jdbc.exceptions.jdbc4.CommunicationsException异常原因是默认开启了SSL认证相当于useSSLtrue解决方法就是在配置中添加skip-ssl重启MySQL服务另外[mysqld_safe]配置项已废弃不用以下的配置均已经过验证。[mysqld] # 务必禁用掉ssl相当于useSSLfalse skip-ssl datadir/var/lib/mysql socket/var/lib/mysql/mysql.sock # transaction-isolation READ-COMMITTED # Disabling symbolic-links is recommended to prevent assorted security risks; # to do so, uncomment this line: symbolic-links 0 key_buffer_size 32M max_allowed_packet 16M thread_stack 256K thread_cache_size 64 query_cache_limit 8M query_cache_size 64M query_cache_type 1 # 设置数据库错误日志时间显示为系统时间 log_timestampssystem # 设置安全策略 validate_password_policyLOW validate_password_length4 validate_password_mixed_case_count1 validate_password_number_count1 validate_password_special_char_count1 max_connections 550 #expire_logs_days 10 #max_binlog_size 100M #log_bin should be on a disk with enough free space. #Replace /var/lib/mysql/mysql_binary_log with an appropriate path for your #system and chown the specified folder to the mysql user. log_bin/var/lib/mysql/mysql_binary_log #In later versions of MySQL, if you enable the binary log and do not set #a server_id, MySQL will not start. The server_id must be unique within #the replicating group. server_id1 binlog_format mixed read_buffer_size 2M read_rnd_buffer_size 16M sort_buffer_size 8M join_buffer_size 8M # InnoDB settings innodb_file_per_table 1 innodb_flush_log_at_trx_commit 2 innodb_log_buffer_size 64M innodb_buffer_pool_size 4G innodb_thread_concurrency 8 innodb_flush_method O_DIRECT innodb_log_file_size 512M # [mysqld_safe] 5.7已废弃该配置 log-error/var/log/mysqld.log pid-file/var/run/mysqld/mysqld.pid sql_modeSTRICT_ALL_TABLES重启MySQL服务器使配置生效systemctl restart mysqld初始化数据库及用户Cloudera官网要求初始化的数据库及用户如下创建以上所有组件数据库执行以下脚本[rootnode1 ~]# mysql -uroot -p # scm CREATE DATABASE scm DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci; GRANT ALL ON scm.* TO scm% IDENTIFIED BY abcd1234; # amon CREATE DATABASE amon DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci; GRANT ALL ON amon.* TO amon% IDENTIFIED BY abcd1234; # rman CREATE DATABASE rman DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci; GRANT ALL ON rman.* TO rman% IDENTIFIED BY abcd1234; # hue CREATE DATABASE hue DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci; GRANT ALL ON hue.* TO hue% IDENTIFIED BY abcd1234; # hive CREATE DATABASE metastore DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci; GRANT ALL ON metastore.* TO hive% IDENTIFIED BY abcd1234; # sentry CREATE DATABASE sentry DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci; GRANT ALL ON sentry.* TO sentry% IDENTIFIED BY abcd1234; # nav CREATE DATABASE nav DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci; GRANT ALL ON nav.* TO nav% IDENTIFIED BY abcd1234; # navms CREATE DATABASE navms DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci; GRANT ALL ON navms.* TO navms% IDENTIFIED BY abcd1234; # oozie CREATE DATABASE oozie DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci; GRANT ALL ON oozie.* TO oozie% IDENTIFIED BY abcd1234; # 最后刷新一下权限 flush privileges; # 检查 show databases;安装JDBC驱动程序用于各节点连接数据库Cloudera要求必须创建在/usr/share/java目录下才能被组件使用。# 创建目录 pssh -h ./nodes -i mkdir -p /usr/share/java/ # 复制到各个节点 pscp -h ./nodes /var/www/html/cdh/mysql/mysql-connector-java-5.1.46-bin.jar /usr/share/java/ # 修改权限 pssh -h ./nodes -i cd /usr/share/java;chmod 777 mysql-connector-java-5.1.46-bin.jar # 重命名jar包以便找到 pssh -h ./nodes -i cd /usr/share/java;ln -s mysql-connector-java-5.1.46-bin.jar mysql-connector-java.jar # 验证是否创建成功 pssh -h ./nodes -i tree /usr/share/java搭建私有Yum源制作CDH和CM的本地repo源方便内网集群节点访问。安装httpd服务node1节点安装Apache httpd服务用于后续通过Http方式访问和下载CDH和CM安装包。# 安装httpd和createrepo yum -y install httpd createrepo # 启动 systemctl start httpd # 开机启动 systemctl enable httpd注意httpd服务在传输parcel文件时默认会设置“Content-Encoding”导致CM Server认为parcel文件已经被httpd压缩并尝试解压缩结果会报错解决办法是修改httpd的配置文件。修改配置文件 /etc/httpd/conf/httpd.conf在IfModule mime_module中添加支持.parcel文件类型用于远程下载parcel文件。[rootnode1 ~]# vim /etc/httpd/conf/httpd.conf IfModule mime_module AddType application/x-compress .Z AddType application/x-gzip .gz .tgz .parcel AddType text/html .shtml AddOutputFilter INCLUDES .shtml /IfModule重启httpd服务systemctl restart httpd配置web服务node1节点移动cdh到httpd服务的根目录/var/www/html下生成repo元数据用于通过Yum访问。[rootnode1 ~]# mv /opt/apps/cdh /var/www/html [rootnode1 ~]# tree /var/www/html /var/www/html └── cdh ├── CDH6.3.2 │ ├── CDH-6.3.2-1.cdh6.3.2.p0.1605554-el7.parcel │ ├── CDH-6.3.2-1.cdh6.3.2.p0.1605554-el7.parcel.sha1 │ ├── CDH-6.3.2-1.cdh6.3.2.p0.1605554-el7.parcel.sha256 │ └── manifest.json ├── CM6.3.1 │ ├── allkeys.asc │ ├── cloudera-manager-agent-6.3.1-1466458.el7.x86_64.rpm │ ├── cloudera-manager-daemons-6.3.1-1466458.el7.x86_64.rpm │ ├── cloudera-manager-server-6.3.1-1466458.el7.x86_64.rpm │ ├── cloudera-manager-server-db-2-6.3.1-1466458.el7.x86_64.rpm │ └── enterprise-debuginfo-6.3.1-1466458.el7.x86_64.rpm ├── JDK │ └── oracle-j2sdk1.8-1.8.0update181-1.x86_64.rpm └── mysql ├── 01_mysql-community-common-5.7.16-1.el7.x86_64.rpm ├── 02_mysql-community-libs-5.7.16-1.el7.x86_64.rpm ├── 03_mysql-community-libs-compat-5.7.16-1.el7.x86_64.rpm ├── 04_mysql-community-client-5.7.16-1.el7.x86_64.rpm ├── 05_mysql-community-server-5.7.16-1.el7.x86_64.rpm └── mysql-connector-java-5.1.46-bin.jar 5 directories, 17 files [rootnode1 ~]# cd /var/www/html/cdh/CM6.3.1/ [rootnode1 CM6.3.1]# createrepo .验证http访问制作本地repo源在/etc/yum.repos.d目录下创建cm-local.repo配置文件添加如下内容[rootnode1 yum.repos.d]# vim cm-local.repo [cm-repo] namecm-repo baseurlhttp://192.168.5.10/cdh/CM6.3.1 enabled1 # 启用yum gpgcheck0 # 不检查检查gpg复制到集群其他节点执行如下操作# 远程复制 pscp -h ./nodes /etc/yum.repos.d/cm-local.repo /etc/yum.repos.d/ # 先清空再添加缓存 pssh -h ./nodes -i yum clean all yum makecache # 查看当前所有yum源 pssh -h ./nodes -i yum repolist三、CM6.3.1安装步骤集群角色集群CM角色序号节点CM服务角色1node1cloudera-scm-server cloudera-scm-agent2node2cloudera-scm-agent3node3cloudera-scm-agent安装CM6.3.1安装cloudera manager server首先node1节点安装cloudera manager server方便后续通过Web UI安装CDH。yum -y install cloudera-manager-server初始化scm数据库[rootnode1 ~]# /opt/cloudera/cm/schema/scm_prepare_database.sh mysql scm scm abcd1234 JAVA_HOME/usr/java/jdk1.8.0_181-cloudera Verifying that we can write to /etc/cloudera-scm-server Creating SCM configuration file in /etc/cloudera-scm-server Executing: /usr/java/jdk1.8.0_181-cloudera/bin/java -cp /usr/share/java/mysql-connector-java.jar:/usr/share/java/oracle-connector-java.jar:/usr/share/java/postgresql-connector-java.jar:/opt/cloudera/cm/schema/../lib/* com.cloudera.enterprise.dbutil.DbCommandExecutor /etc/cloudera-scm-server/db.properties com.cloudera.cmf.db. Tue Aug 22 22:00:51 CST 2023 WARN: Establishing SSL connection without servers identity verification is not recommended. According to MySQL 5.5.45, 5.6.26 and 5.7.6 requirements SSL connection must be established by default if explicit option isnt set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to false. You need either to explicitly disable SSL by setting useSSLfalse, or set useSSLtrue and provide truststore for server certificate verification. [ main] DbCommandExecutor INFO Successfully connected to database. All done, your SCM database is configured correctly!启动cloudera manager server# 启动 systemctl start cloudera-scm-server # 开机启动 systemctl enable cloudera-scm-server # 查看状态 systemctl status cloudera-scm-server查看启动日志/var/log/cloudera-scm-server/cloudera-scm-server.log无异常表示启动成功。[rootnode1 ~]# tail -3 /var/log/cloudera-scm-server/cloudera-scm-server.log 2023-08-22 22:11:22,656 INFO WebServerImpl:org.eclipse.jetty.server.AbstractConnector: Started ServerConnector30096cd2{HTTP/1.1,[http/1.1]}{0.0.0.0:7180} 2023-08-22 22:11:22,656 INFO WebServerImpl:org.eclipse.jetty.server.Server: Started 39407ms 2023-08-22 22:11:22,656 INFO WebServerImpl:com.cloudera.server.cmf.WebServerImpl: Started Jetty server.四、CDH集群6.3.2安装步骤集群安装向导浏览器地址栏输入http://192.168.5.10:7180/cmf/login打开登录Web页面。使用默认用户名/密码admin/admin登录CM进入欢迎页面点击“继续”。勾选接受许可条款和条件点击“继续”。选择Cloudera Express免费版本点击“继续”。进入集群安装页面点击“继续”。编辑集群名称填入cdh6.3.2安装成功后可以重命名集群点击“继续”。使用模式输入主机名称或者IP地址点击“搜索”找到主机后勾选节点点击“继续”。Cloudera Manager Agent的自定义存储库输入CM仓库地址:http://192.168.5.10/cdh/CM6.3.1。CDH and other software选择“使用 Parcel (建议)”点击“更多选项”进入Parcel存储库设置页面。“Parcel目录”和“本地Parcel存储库路径”不用编辑选择默认目录就好编辑“远程Parcel存储库URL”输入CDH仓库地址: http://192.168.5.10/cdh/CDH6.3.2/点击“-”删除其它URL点击“保存更改”。上一步保存后此处的CDH版本会自动刷新点击“继续”。安装JDK如果已安装请跳过点击“继续”。输入root用户的密码用于SSH登录点击“继续”。安装Agent到集群各节点点击“详细信息”可查看安装过程。安装完毕自动跳转到下一步开始下载并安装Parcels。点击“Inspect Network Performance”和“Inspect Hosts”检查网络和主机性能。如果出现黄色告警查看“显示检查器结果”逐项解决后点击“重新运行”直到所有的检查都通过方可点击“继续”也可以直接选择最后一项跳过告警点击“继续”。集群安装设置进入集群设置页面选择需要安装的服务组合。我这里选择自定义服务选择自己需要的组件点击“继续”。进入集群自定义角色分配根据自己的规划给节点分配角色。配置完成点击“按主机查看”点击“继续”进入数据库设置页面选择数据库类型MySQL、输入主机名称、数据库、用户及密码点击“测试连接”测试成功后点击“继续”。进入审核更改页面检查默认的目录设置是否合适可根据实际情况调整确认无误后点击“继续”。开始运行命令进行各个组件安装完成后点击“继续”。进入汇总页面表示集群组件安装成功并启动点“完成”。最后进入CM管理主页至此CDH6.3.2安装成功