DSS部署在 bigdata4 节点上。

1.1 下载安装包

地址：

版本：1.1.1

1.2 部署前注意事项（重要）确保安装的系统为CentOS为6或者7服务器存在多网卡问题。首先通过命令ifconfig命令查看服务器激活状态的网卡，若激活状态的网卡数大于1，那么用户就需要通过命令ifconfig [NIC_NAME] down([NIC_NAME]为网卡名称)来关闭多余的网卡，以确保激活的网卡数只有1个网卡多IP问题。在确保服务器只存在一个网卡是激活状态的情况下，通过命令echo $(hostname -I)查看网卡对应的IP数，若大于1，那么就需要去掉网卡中指定的IP，采用动态获取IP的方式，具体命令如下：ip addr flush dev [NIC_NAME]

ifdown [NIC_NAME]

ifup [NIC_NAME]hostname配置。在安装前用户需要配置hostname到ip的映射1.3 基础软件安装1.3.1 概述需要的命令工具（在正式安装前，脚本会自动检测这些命令是否可用，如果不存在会尝试自动安装，安装失败则需用户手动安装以下基础shell命令工具）：

telnet; tar; sed; dos2unix; mysql; yum; java; unzip; zip; expect需要安装的软件：

MySQL (5.5+); JDK (1.8.0_141以上); Python(2.x和3.x都支持); Nginx下面的服务必须可从本机访问：

Hadoop(2.7.2，Hadoop其他版本需自行编译Linkis) ，安装的机器必须支持执行 hdfs dfs -ls / 命令

Hive(2.3.3，Hive其他版本需自行编译Linkis)，安装的机器必须支持执行hive -e “show databases”命令

Spark(支持2.0以上所有版本) ，安装的机器必须支持执行spark-sql -e “show databases” 命令Tips:

如用户是第一次安装Hadoop，单机部署Hadoop可参考：Hadoop单机部署 ；分布式部署Hadoop可参考：Hadoop分布式部署

如用户是第一次安装Hive，可参考：Hive快速安装部署

如用户是第一次安装Spark，On Yarn模式可参考：Spark on Yarn部署1.3.2 基础软件安装1.3.2.1 Linux命令安装

yum install -y telnet-server.x86_64yum install -y telnet.x86_64yum -y install sedyum install -y dos2unixyum -y install expect

python -m pip install matplotlib

①. 在 bigdata4 节点上，安装MySQL8.0.31

下载地址：

②. 上传并解压解压mysql-8.0.31-linux-glibc2.12-x86_64.tar.xz

#1.上传jar包：cd /optsudo rz  Ctrl + 点击鼠标右键上传jar包#2.解压：sudo tar -xvf mysql-8.0.31-linux-glibc2.12-x86_64.tar.xz#3.删掉安装包：sudo rm -rf mysql-8.0.31-linux-glibc2.12-x86_64.tar.xz#4.更改名称sudo mv mysql-8.0.31-linux-glibc2.12-x86_64 mysql-8.0.31#5.更改用户组sudo chown -R admin:admin mysql-8.0.31/

③. 填写配置文件

在MySQL根目录下创建配置文件 my.cnf

vim /opt/mysql-8.0.31/my.cnf

配置文件如下：

[mysqld]# 设置3306端口port=3306# 设置mysql的安装目录basedir=/opt/mysql-8.0.31# 设置mysql数据库的数据的存放目录datadir=/opt/mysql-8.0.31/data# 允许最大连接数max_connections=2000# 允许连接失败的次数。这是为了防止有人从该主机试图攻击数据库系统max_connect_errors=10# 服务端使用的字符集默认为UTF8character-set-server=utf8# 创建新表时将使用的默认存储引擎default-storage-engine=INNODB#sql_mode中去掉了ONLY_FULL_GROUP_BYsql_mode=STRICT_TRANS_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,NO_ENGINE_SUBSTITUTION[mysql]# 设置mysql客户端默认字符集default-character-set=utf8[client]# 设置mysql客户端连接服务端时默认使用的端口port=3306default-character-set=utf8

在MySQL根目录下创建数据文件夹：

cd /opt/mysql-8.0.31mkdir data

④. MySQL初始化

cd /opt/mysql-8.0.31/bin./mysqld --defaults-file=/opt/mysql-8.0.31/my.cnf  --initialize-insecure

⑤. 启动MySQL

/opt/mysql-8.0.31/bin/mysqld_safe --defaults-file=/opt/mysql-8.0.31/my.cnf &

⑥. 查看是否启动

ps -ef|grep mysql

⑦. 配置环境变量

打开 /etc/profile.d 目录下的 my_env.sh 文件sudo vim /etc/profile.d/my_env.sh添加如下内容，然后保存（:wq）退出#MYSQL_HOME

export MYSQL_HOME=/opt/mysql-8.0.31

export PATH=$PATH:$MYSQL_HOME/bin让环境变量生效source /etc/profile.d/my_env.sh

⑧. 登录数据库

mysql -u root -p

⑨. 修改密码

flush privileges;ALTER USER 'root'@'localhost'IDENTIFIED BY 'root' PASSWORD EXPIRE NEVER;ALTER USER 'root'@'localhost'IDENTIFIED wITH mysql_native_password BY 'Xl123456';flush privileges;

①. 在 bigdata4 节点上，安装nginx-1.22.1

下载地址：

②. 上传并解压解压 nginx-1.22.1.tar.gz

#1.上传jar包：cd /optsudo rz  Ctrl + 点击鼠标右键上传jar包#2.解压：sudo tar -xvf nginx-1.22.1.tar.gz#3.删掉安装包：sudo rm -rf nginx-1.22.1.tar.gz#4.更改用户组sudo chown -R admin:admin nginx-1.22.1/

③. 安装nginx

# 创建 /usr/local/nginx 文件夹cd /usr/localsudo mkdir nginxsudo chown -R admin:admin nginx/cd /opt/nginx-1.22.1/./configure --with-http_ssl_module --with-ipv6 --with-stream./configure --prefix=/usr/local/nginx --with-http_stub_status_module --with-http_ssl_module --with-http_flv_module --with-http_stub_status_module --with-stream --with-http_gzip_static_modulemake && make install

④. 启动nginx

cd /usr/local/nginx/sbin/sudo ./nginx

#1.上传jar包：cd /optsudo rz  Ctrl + 点击鼠标右键上传jar包#2.解压：sudo unzip dss_linkis_one-click_install_20221201.zip#3.删掉安装包：sudo rm -rf dss_linkis_one-click_install_20221201.zip#4.更改名称sudo mv dss_linkis_one-click_install_20221201/ dss_linkis#5.更改用户组sudo chown -R admin:admin dss_linkis/

vim /opt/dss_linkis/conf/config.sh

修改文件内容如下：

### deploy userdeployUser=admin### Linkis_VERSIONLINKIS_VERSION=1.1.1### DSS WebDSS_NGINX_IP=127.0.0.1DSS_WEB_PORT=8085### DSS VERSIONDSS_VERSION=1.1.1############## ############## linkis的其他默认配置信息 start ############## ################# Specifies the user workspace, which is used to store the user's script files and log files.### Generally local directory##file:// requiredWORKSPACE_USER_ROOT_PATH=/home/admin/linkis### User's root hdfs path##hdfs:// requiredHDFS_USER_ROOT_PATH=hdfs:///tmp/linkis ### Path to store job ResultSet:file or hdfs path##hdfs:// requiredRESULT_SET_ROOT_PATH=hdfs:///tmp/linkis ### Path to store started engines and engine logs, must be localENGINECONN_ROOT_PATH=/home/admin/linkis/tmp#ENTRANCE_CONFIG_LOG_PATH=hdfs:///tmp/linkis/ ##hdfs:// required###HADOOP CONF DIR #/appcom/config/hadoop-configHADOOP_CONF_DIR=/opt/hadoop-3.1.3/etc/hadoop###HIVE CONF DIR  #/appcom/config/hive-configHIVE_CONF_DIR=/opt/hive/conf###SPARK CONF DIR #/appcom/config/spark-configSPARK_CONF_DIR=/opt/hive/conf# for installLINKIS_PUBLIC_MODULE=lib/linkis-commons/public-module##YARN REST URL  spark engine requiredYARN_RESTFUL_URL= Engine version conf#SPARK_VERSIONSPARK_VERSION=3.0.0##HIVE_VERSIONHIVE_VERSION=3.1.2PYTHON_VERSION=python2## LDAP is for enterprise authorization, if you just want to have a try, ignore it.#LDAP_URL=ldap://localhost:1389/#LDAP_BASEDN=dc=webank,dc=com#LDAP_USER_NAME_FORMAT=cn=%s@xxx.com,OU=xxx,DC=xxx,DC=com################### The install Configuration of all Linkis's Micro-Services #######################    NOTICE:#       1. If you just wanna try, the following micro-service configuration can be set without any settings.#            These services will be installed by default on this machine.#       2. In order to get the most complete enterprise-level features, we strongly recommend that you install#          the following microservice parameters####  EUREKA install information###  You can access it in your browser at the address below:{EUREKA_INSTALL_IP}:${EUREKA_PORT}###  Microservices Service Registration Discovery CenterLINKIS_EUREKA_INSTALL_IP=127.0.0.1LINKIS_EUREKA_PORT=9600#LINKIS_EUREKA_PREFER_IP=true###  Gateway install information#LINKIS_GATEWAY_INSTALL_IP=127.0.0.1LINKIS_GATEWAY_PORT=9001### ApplicationManager#LINKIS_MANAGER_INSTALL_IP=127.0.0.1LINKIS_MANAGER_PORT=9101### EngineManager#LINKIS_ENGINECONNMANAGER_INSTALL_IP=127.0.0.1LINKIS_ENGINECONNMANAGER_PORT=9102### EnginePluginServer#LINKIS_ENGINECONN_PLUGIN_SERVER_INSTALL_IP=127.0.0.1LINKIS_ENGINECONN_PLUGIN_SERVER_PORT=9103### LinkisEntrance#LINKIS_ENTRANCE_INSTALL_IP=127.0.0.1LINKIS_ENTRANCE_PORT=9104###  publicservice#LINKIS_PUBLICSERVICE_INSTALL_IP=127.0.0.1LINKIS_PUBLICSERVICE_PORT=9105### cs#LINKIS_CS_INSTALL_IP=127.0.0.1LINKIS_CS_PORT=9108########## Linkis微服务配置完毕##### ################### The install Configuration of all DataSphereStudio's Micro-Services #######################    NOTICE:#       1. If you just wanna try, the following micro-service configuration can be set without any settings.#            These services will be installed by default on this machine.#       2. In order to get the most complete enterprise-level features, we strongly recommend that you install#          the following microservice parameters#### DSS_SERVER### This service is used to provide dss-server capability.### project-server#DSS_FRAMEWORK_PROJECT_SERVER_INSTALL_IP=127.0.0.1#DSS_FRAMEWORK_PROJECT_SERVER_PORT=9002### orchestrator-server#DSS_FRAMEWORK_ORCHESTRATOR_SERVER_INSTALL_IP=127.0.0.1#DSS_FRAMEWORK_ORCHESTRATOR_SERVER_PORT=9003### apiservice-server#DSS_APISERVICE_SERVER_INSTALL_IP=127.0.0.1#DSS_APISERVICE_SERVER_PORT=9004### dss-workflow-server#DSS_WORKFLOW_SERVER_INSTALL_IP=127.0.0.1#DSS_WORKFLOW_SERVER_PORT=9005### dss-flow-execution-server#DSS_FLOW_EXECUTION_SERVER_INSTALL_IP=127.0.0.1#DSS_FLOW_EXECUTION_SERVER_PORT=9006###dss-scriptis-server#DSS_SCRIPTIS_SERVER_INSTALL_IP=127.0.0.1#DSS_SCRIPTIS_SERVER_PORT=9008###dss-data-api-server#DSS_DATA_API_SERVER_INSTALL_IP=127.0.0.1#DSS_DATA_API_SERVER_PORT=9208###dss-data-governance-server#DSS_DATA_GOVERNANCE_SERVER_INSTALL_IP=127.0.0.1#DSS_DATA_GOVERNANCE_SERVER_PORT=9209###dss-guide-server#DSS_GUIDE_SERVER_INSTALL_IP=127.0.0.1#DSS_GUIDE_SERVER_PORT=9210########## DSS微服务配置完毕################### ############## other default configuration 其他默认配置信息  ############## ################ java application default jvm memoryexport SERVER_HEAP_SIZE="512M"##sendemail配置，只影响DSS工作流中发邮件功能EMAIL_HOST=smtp.163.comEMAIL_PORT=25EMAIL_USERNAME=mixiangchong@xianglin.cnEMAIL_PASSWORD=Xl123456EMAIL_PROTOCOL=smtp### Save the file path exported by the orchestrator serviceORCHESTRATOR_FILE_PATH=/home/admin/dss### Save DSS flow execution service log pathEXECUTION_LOG_PATH=/home/admin/dss

### for DSS-Server and Eventchecker APPCONNMYSQL_HOST=172.16.3.26MYSQL_PORT=3306MYSQL_DB=dssMYSQL_USER=rootMYSQL_PASSWORD=Xl123456#主要是配合scriptis一起使用，如果不配置，会默认尝试通过$HIVE_CONF_DIR 中的配置文件获取# HiveMeta元数据库的URLHIVE_META_URL=jdbc:mysql://172.16.3.26:3306/metastore?useSSL=false&useUnicode=true&characterEncoding=UTF-8# HiveMeta元数据库的用户HIVE_META_USER=root # HiveMeta元数据库的密码HIVE_META_PASSWORD=Xl123456

cd /opt/dss_linkis/bin

sh install.sh

# 切换到Linkis配置文件目录cd /opt/dss_linkis/linkis/conf# 打开配置文件linkis-env.shvim linkis-env.sh# 将如下配置改为trueexport ENABLE_METADATA_MANAGER=true

## 切换到Linkis配置文件目录cd /opt/dss_linkis/linkis/conf/## 打开配置文件linkis-mg-gateway.propertiesvim linkis-mg-gateway.properties## 修改密码wds.linkis.admin.password=hadoop

sh /opt/dss_linkis/bin/start-all.sh

# 切换目录到dss，正常情况下dss目录就在xx/dss_linkis目录下，cd /opt/dss_linkis/dss/bin# 执行启动默认Appconn脚本sh install-default-appconn.sh

sh /opt/dss_linkis/bin/stop-all.sh