Hive 组件安装与配置

854 views

目录:

Hive 组件安装与配置

1.基础环境和安装准备

Hive 组件的部署规划和软件包路径如下：

（1）当前环境中已安装 Hadoop 全分布系统。

（2）本地安装 MySQL 数据库（账号 root，密码 root123）

（3）MySQL 端口号（3306）。

（4）MySQL 的 JDBC 驱动包/opt/software/mysql-connector-java-5.1.49.jar，

在此基础上更新 Hive 元数据存储。

（5）Hive 软件包/opt/software/apache-hive-2.3.9-bin.tar.gz

2.安装 MySQL

# 1.下载MySQL
wget https://dev.mysql.com/get/mysql57-community-release-el7-9.noarch.rpm

rpm -ivh mysql57-community-release-el7-9.noarch.rpm

# 或者下载rpm包直接安装
准备好mysql-community-common、mysql-community-libs、mysql-community-client、mysql-community-server 的rpm包 下载地址

http://repo.mysql.com/yum/mysql-5.7-community/el/7/x86_64/mysql-community-common-5.7.35-1.el7.x86_64.rpm
http://repo.mysql.com/yum/mysql-5.7-community/el/7/x86_64/mysql-community-libs-5.7.35-1.el7.x86_64.rpm
http://repo.mysql.com/yum/mysql-5.7-community/el/7/x86_64/mysql-community-client-5.7.35-1.el7.x86_64.rpm
http://repo.mysql.com/yum/mysql-5.7-community/el/7/x86_64/mysql-community-server-5.7.35-1.el7.x86_64.rpm
rpm -ivh mysql-community-common-5.7.35-1.el7.x86_64.rpm  --force --nodeps
rpm -ivh mysql-community-libs-5.7.35-1.el7.x86_64.rpm    --force --nodeps
rpm -ivh mysql-community-client-5.7.35-1.el7.x86_64.rpm  --force --nodeps
rpm -ivh mysql-community-server-5.7.35-1.el7.x86_64.rpm  --force --nodeps

# 2.安装与启动MySQL-Server
yum -y install mysql-server

systemctl start mysqld

# 3.MySQL密码设置
# 1.获取临时密码
cat /var/log/mysqld.log  |grep "password" | cut -d":" -f4
### 例如:  fNIFmZX,u94>

# 2.跳过MySQL的密码认证过程
vim /etc/my.cnf
# 按i进入编辑模式
skip-grant-tables
# 在[mysqld]后面任意一行添加“skip-grant-tables”用来跳过密码验证的过程，
# 使用 [Esc]:wq  退出vim编辑器

# 重启mysql
systemctl restart mysqld

# 3.登入MySQL
mysql -uroot -p
# 输入密码

# 4.修改密码

Use mysql;
# 缓存写入磁盘
flush privileges;
# 修改root密码为'000000'
ALTER USER 'root'@'localhost' IDENTIFIED BY 'root123';
# 退出
\q;

# 5.修改MySQL的密码认证过程和修改字符集
vim /etc/my.cnf
# 注释掉之前添加的 skip-grant-tables
# skip-grant-tables
# 将服务端字符集修改为utf8mb4以更好的支持中文
character-set-server=utf8mb4

# 6.重启mysql
systemctl restart mysqld

# 7.设置为开启自动启动MySQL-Server
systemctl enable mysqld

4.开启远程控制
# 1.登录
mysql -uroot -proot123

# 2.修改登录地址
use mysql;
# % 表示任意地址都可以登录
update user set Host='%' where User='root';
# 刷新缓存
flush privileges;
# 退出
\q;

3.安装 Hive

[root@master ~]# cd /opt/software/
[root@master software]#  tar -zxvf /opt/software/apache-hive-2.3.9-bin.tar.gz -C  /usr/local/

[root@master local]# ln -s /usr/local/apache-hive-2.3.9-bin/ /usr/local/hive

[root@master ~]# chown hadoop.hadoop -R /usr/local/hive/

[root@master ~]# vim /etc/profile
export HIVE_HOME=/usr/local/hive
export PATH=$PATH:$JAVA_HOME/bin:$HADOOP_HOME/bin:$HADOOP_HOME/sbin:$HIVE_HOME/bin

[root@master ~]# source /etc/profile

4.配置Hive

[root@master ~]# su - hadoop
[hadoop@master conf]$ cp hive-default.xml.template hive-site.xml
[hadoop@master conf]$ vim hive-site.xml 
# 1）设置 MySQL 数据库连接。
<name>javax.jdo.option.ConnectionURL</name>
<value>jdbc:mysql://master:3306/hive?createDatabaseIfNotExist=true&amp;useSSL=false</value>
<description>JDBC connect string for a JDBC metastore</description>
# 2）配置 MySQL 数据库 root 的密码。
<name>javax.jdo.option.ConnectionPassword</name>
<value>root123</value>
<description>password to use against s database</description>
</property>
# 3）验证元数据存储版本一致性。若默认 false，则不用修改。
<property>
<name>hive.metastore.schema.verification</name>
<value>false</value>
<description>
 Enforce metastore schema version consistency.
True: Verify that version information stored in is compatible with one from 
Hive jars. Also disable automatic
 False: Warn if the version information stored in metastore doesn't match 
with one from in Hive jars.
</description>
</property>
# 4）配置数据库驱动。
<property>
<name>javax.jdo.option.ConnectionDriverName</name>
<value>com.mysql.jdbc.Driver</value>
<description>Driver class name for a JDBC metastore</description>
</property>
# 5）配置数据库用户名 javax.jdo.option.ConnectionUserName 为 root。
<property>
<name>javax.jdo.option.ConnectionUserName</name>
<value>root</value>
<description>Username to use against metastore database</description>
</property>
# 6 ）将以下位置的 ${system:java.io.tmpdir}/${system:user.name} 替换为“/usr/local/src/hive/tmp”目录及其子目录
 <name>hive.querylog.location</name>
 <value>/usr/local/hive/tmp</value>
 <description>Location of Hive run time structured log 
file</description>
 <name>hive.exec.local.scratchdir</name>
 <value>/usr/local/hive/tmp</value>
 <name>hive.downloaded.resources.dir</name>
 <value>/usr/local/hive/tmp/resources</value>
 <name>hive.server2.logging.operation.log.location</name>
 <value>/usr/local/hive/tmp/operation_logs</value>
 # 7）在 Hive 安装目录中创建临时文件夹 tmp。
 [hadoop@master ~]$ mkdir /usr/local/hive/tmp

5.初始化 hive 元数据

# 1) 配置 MySQL 驱动
[root@master ~]$ su -
[root@master ~]# cd /opt/software/
[root@master software]# tar -xvf mysql-connector-java-5.1.49.tar.gz 
[root@master software]# cd mysql-connector-java-5.1.49
[root@master mysql-connector-java-5.1.49]# cp mysql-connector-java-5.1.49.jar  /usr/local/hive/lib/
[root@master mysql-connector-java-5.1.49]#  su - hadoop

# 2）重新启动 hadooop 即可
[hadoop@master ~]$ stop-all.sh
[hadoop@master ~]$ start-all.sh

# 3 ）启动 hive
[hadoop@master ~]$ hive

image-20220413170022169