源码编译安装 Ambari 2.7.5，记录步骤如下
1.1 安装依赖
官方Build过程：https://cwiki.apache.org/confluence/display/AMBARI/Ambari+Development
1# 先安装下面依赖 2JDK 8 3maven 3.3.9 及以上，并配置setting.xml为wmqhealth内网仓库地址 4python 2.7 5 6# 安装python setuptools 7wget https://pypi.python.org/packages/2.7/s/setuptools/setuptools-0.6c11-py2.7.egg#md5=fe1f997bc722265116870bc7919059ea 8sh setuptools-0.6c11-py2.7.egg 9 10# yum安装 11yum install -y rpm-build // 12yum install -y gcc-c++ //ambari-logsearch-web模块会报错
1.2 下载和编译
参考：https://cwiki.apache.org/confluence/display/AMBARI/Installation+Guide+for+Ambari+2.7.5
1# 下载 2mkdir -p /opt/ambari && cd /opt/ambari 3wget https://www-eu.apache.org/dist/ambari/ambari-2.7.5/apache-ambari-2.7.5-src.tar.gz 4tar xfvz apache-ambari-2.7.5-src.tar.gz 5cd apache-ambari-2.7.5-src 6mvn versions:set -DnewVersion=2.7.5.0.0 7 8pushd ambari-metrics 9mvn versions:set -DnewVersion=2.7.5.0.0 10popd 11 12# 编译 13mvn -B clean install rpm:rpm -DnewVersion=2.7.5.0.0 -DbuildNumber=5895e4ed6b30a2da8a90fee2403b6cab91d19972 -DskipTests -Dpython.ver="python >= 2.6" 14
参数说明
1# -B参数：该参数表示让Maven使用批处理模式构建项目，能够避免一些需要人工参与交互而造成的挂起状态。 2 3# 如果第一次编译报错之后,重新编译有问题，则在命令中加上 -Drat.skip=true 4# -X：如果看详细debug信息可加上该参数 5#[root@master apache-ambari-2.7.5-src]# mvn -B clean install rpm:rpm -DnewVersion=2.7.5.0.0 -DbuildNumber=5895e4ed6b30a2da8a90fee2403b6cab91d19972 -DskipTests -Dpython.ver="python >= 2.6" -Drat.skip=true -X 6 7#因为会报错，重复执行mvn，把clean去掉可以快不少 8#[root@master apache-ambari-2.7.5-src]# mvn -B install rpm:rpm -DnewVersion=2.7.5.0.0 -DbuildNumber=5895e4ed6b30a2da8a90fee2403b6cab91d19972 -DskipTests -Dpython.ver="python >= 2.6" -Drat.skip=true -X 9 10#从某个点开始恢复，就不用把之前build好的再重新来一次了 11#如果报错末提示如：mvn <args> -rf :ambari-metrics-storm-sink-legacy 12#[root@master apache-ambari-2.7.5-src]# mvn -B install rpm:rpm -DnewVersion=2.7.5.0.0 -DbuildNumber=5895e4ed6b30a2da8a90fee2403b6cab91d19972 -DskipTests -Dpython.ver="python >= 2.6" -Drat.skip=true -X -rf :ambari-web
1.3 编译报错解决
1）ambari-web 报错1：
mvn <args> -rf :ambari-web
1[ERROR] Failed to execute goal com.github.eirslett:frontend-maven-plugin:1.4:install-node-and-yarn (install node and yarn) on project ambari-web: Could not extract the Yarn archive: Could not extract archive: '/root/.m2/repository/com/github/eirslett/yarn/0.23.2/yarn-0.23.2./yarn-v0.23.2.tar.gz': EOFException -> [Help 1] 2org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal com.github.eirslett:frontend-maven-plugin:1.4:install-node-and-yarn (install node and yarn) on project ambari-web: Could not extract the Yarn archive: Could not extract archive: '/root/.m2/repository/com/github/eirslett/yarn/0.23.2/yarn-0.23.2./yarn-v0.23.2.tar.gz' 3 4# 有可能是因为网络问题包没有下载完整，我遇到了几个包都不是完整的，所以直接去外面下载然后覆盖到仓库中 5# 删除yarn-v0.23.2.tar.gz手动下载并放于/root/.m2/repository/com/github/eirslett/yarn/0.23.2/yarn-0.23.2./目录下： 6https://github.com/yarnpkg/yarn/releases/download/v0.23.2/yarn-v0.23.2.tar.gz 7 8# 重新编译 9mvn -B install rpm:rpm -DnewVersion=2.7.5.0.0 -DbuildNumber=5895e4ed6b30a2da8a90fee2403b6cab91d19972 -DskipTests -Dpython.ver="python >= 2.6" -Drat.skip=true -X -rf :ambari-web 10
2）ambari-web 报错2：
mvn <args> -rf :ambari-web
1[ERROR] Failed to execute goal com.github.eirslett:frontend-maven-plugin:1.4:yarn (yarn install) on project ambari-web: Failed to run task: 'yarn install --ignore-engines --pure-lockfile' failed. org.apache.commons.exec.ExecuteException: Process exited with an error: 1 (Exit value: 1) -> [Help 1] 2 3# 这个问题最容易出现，主要是nodejs会通过 yarn去加载依赖包，解决方式是保证网络环境好。 4# 重试即可 5mvn -B install rpm:rpm -DnewVersion=2.7.5.0.0 -DbuildNumber=5895e4ed6b30a2da8a90fee2403b6cab91d19972 -DskipTests -Dpython.ver="python >= 2.6" -Drat.skip=true -rf :ambari-web 6
3）ambari-web 卡住：
mvn <args> -rf :ambari-web
1[INFO] Downloading https://github.com/Medium/phantomjs/releases/download/v1.9.19/phantomjs-1.9.8-linux-x86_64.tar.bz2 2[INFO] Saving to /tmp/phantomjs/phantomjs-1.9.8-linux-x86_64.tar.bz2 3[INFO] Receiving... 4 5# 卡在这一步半小时，进该目录发现还下载了 0Kb，手动下载该文件并存于/tmp/phantomjs/目录下。 6# 继续编译 7mvn -B install rpm:rpm -DnewVersion=2.7.5.0.0 -DbuildNumber=5895e4ed6b30a2da8a90fee2403b6cab91d19972 -DskipTests -Dpython.ver="python >= 2.6" -Drat.skip=true -rf :ambari-web 8
4）ambari-metrics-timelineservice 报错：
mvn <args> -rf :ambari-metrics-timelineservice
1[ERROR] Failed to execute goal on project ambari-metrics-timelineservice: Could not resolve dependencies for project org.apache.ambari:ambari-metrics-timelineservice:jar:2.7.5.0.0: Failed to collect dependencies at org.apache.phoenix:phoenix-core:jar:5.0.0.3.1.4.0-315 -> org.apache.hbase:hbase-mapreduce:jar:2.0.2.3.1.4.0-315 -> org.apache.hbase:hbase-server:jar:2.0.2.3.1.4.0-315 -> org.glassfish.web:javax.servlet.jsp:jar:2.3.2 -> org.glassfish:javax.el:jar:3.0.1-b06-SNAPSHOT: Failed to read artifact descriptor for org.glassfish:javax.el:jar:3.0.1-b06-SNAPSHOT: Could not transfer artifact org.glassfish:javax.el:pom:3.0.1-b06-SNAPSHOT from/to thirdparty (http://nexus.wmqhealth.com/repository/thirdparty/): Transfer failed for http://nexus.wmqhealth.com/repository/thirdparty/org/glassfish/javax.el/3.0.1-b06-SNAPSHOT/javax.el-3.0.1-b06-SNAPSHOT.pom 400 Repository version policy: RELEASE does not allow version: 3.0.1-b06-SNAPSHOT -> [Help 1] 2 3# 解决 4# 增加华为的maven镜像仓库，修改setting.xml 5<settings> 6 <mirrors> 7 <mirror> 8 <id>huaweicloud</id> 9 <mirrorOf>*</mirrorOf> 10 <url>https://mirrors.huaweicloud.com/repository/maven/</url> 11 </mirror> 12 </mirrors> 13</settings> 14 15# 删除 16find ~/.m2/repository/ -name "*.lastUpdated" -exec rm -rf {} \; 17 18# 重新编译 19mvn -B install rpm:rpm -DnewVersion=2.7.5.0.0 -DbuildNumber=5895e4ed6b30a2da8a90fee2403b6cab91d19972 -DskipTests -Dpython.ver="python >= 2.6" -Drat.skip=true -rf :ambari-metrics-timelineservice
5）手动下载hbase，phoenix：
1Download HBase: 2 [mkdir] Created dir: /opt/ambari/apache-ambari-2.7.5-src/ambari-metrics/ambari-metrics-timelineservice/target/embedded 3 [get] Getting: https://s3.amazonaws.com/dev.hortonworks.com/HDP/centos7/3.x/BUILDS/3.1.4.0-315/tars/hbase/hbase-2.0.2.3.1.4.0-315-bin.tar.gz 4 [get] To: /opt/ambari/apache-ambari-2.7.5-src/ambari-metrics/ambari-metrics-timelineservice/target/embedded/hbase.tar.gz 5 6 7# 下载hbase.tar.gz特别慢，手动下载并存于/opt/ambari/apache-ambari-2.7.5-src/ambari-metrics/ambari-metrics-timelineservice/target/embedded/目录下。 8mv hbase-2.0.2.3.1.4.0-315-bin.tar.gz hbase.tar.gz 9 10# 继续编译 11mvn -B install rpm:rpm -DnewVersion=2.7.5.0.0 -DbuildNumber=5895e4ed6b30a2da8a90fee2403b6cab91d19972 -DskipTests -Dpython.ver="python >= 2.6" -Drat.skip=true -rf :ambari-metrics-timelineservice 12 13 14Download Phoenix: 15 [get] Getting: https://s3.amazonaws.com/dev.hortonworks.com/HDP/centos7/3.x/BUILDS/3.1.4.0-315/tars/phoenix/phoenix-5.0.0.3.1.4.0-315.tar.gz 16 [get] To: /opt/ambari/apache-ambari-2.7.5-src/ambari-metrics/ambari-metrics-timelineservice/target/embedded/phoenix.tar.gz 17 18# 下载慢，手动下载，并重命名 19mv phoenix-5.0.0.3.1.4.0-315.tar.gz phoenix.tar.gz 20# 继续编译 21mvn -B install rpm:rpm -DnewVersion=2.7.5.0.0 -DbuildNumber=5895e4ed6b30a2da8a90fee2403b6cab91d19972 -DskipTests -Dpython.ver="python >= 2.6" -Drat.skip=true -rf :ambari-metrics-timelineservice
6）手动下载hadoop：
1[INFO] -------------< org.apache.ambari:ambari-metrics-assembly >-------------- 2[INFO] Building Ambari Metrics Assembly 2.7.5.0.0 [5/29] 3 4Downloading Hadoop: 5 [mkdir] Created dir: /opt/ambari/apache-ambari-2.7.5-src/ambari-metrics/ambari-metrics-assembly/target/embedded 6 [get] Getting: https://s3.amazonaws.com/dev.hortonworks.com/HDP/centos7/3.x/BUILDS/3.1.4.0-315/tars/hadoop/hadoop-3.1.1.3.1.4.0-315.tar.gz 7 [get] To: /opt/ambari/apache-ambari-2.7.5-src/ambari-metrics/ambari-metrics-assembly/target/embedded/hadoop.tar.gz 8 9# 下载慢，手动下载，并重命名 10mv hadoop-3.1.1.3.1.4.0-315.tar.gz hadoop.tar.gz 11# 继续编译 12mvn -B install rpm:rpm -DnewVersion=2.7.5.0.0 -DbuildNumber=5895e4ed6b30a2da8a90fee2403b6cab91d19972 -DskipTests -Dpython.ver="python >= 2.6" -Drat.skip=true -rf :ambari-metrics-assembly
7）ambari-logsearch-web报错：
1[INFO] --- frontend-maven-plugin:1.6:yarn (yarn install) @ ambari-logsearch-web --- 2[INFO] Running 'yarn install --ignore-engines --pure-lockfile' in /opt/ambari/apache-ambari-2.7.5-src/ambari-logsearch/ambari-logsearch-web/target/webapp-build 3[INFO] yarn install v1.1.0 4[INFO] [1/4] Resolving packages... 5[INFO] [2/4] Fetching packages... 6[INFO] info fsevents@1.1.1: The platform "linux" is incompatible with this module. 7[INFO] info "fsevents@1.1.1" is an optional dependency and failed compatibility check. Excluding it from installation. 8[INFO] [3/4] Linking dependencies... 9[ERROR] warning "@angular/animations@4.4.6" has incorrect peer dependency "@angular/core@4.4.6". 10[ERROR] warning "@angular/core@4.4.3" has unmet peer dependency "zone.js@^0.8.4". 11[ERROR] warning "angular2-notifications@0.4.46" has incorrect peer dependency "@angular/core@^2.1.2". 12[ERROR] warning "angular2-notifications@0.4.46" has incorrect peer dependency "@angular/common@^2.1.2". 13[ERROR] warning "angular2-notifications@0.4.46" has incorrect peer dependency "@angular/platform-browser@^2.1.1". 14[ERROR] warning "angular2-notifications@0.4.46" has incorrect peer dependency "rxjs@5.0.0-beta.12". 15[ERROR] warning "license-webpack-plugin@1.0.0" has unmet peer dependency "webpack-sources@>= 1.0.0". 16[INFO] [4/4] Building fresh packages... 17 18# 卡在这里半小时了也没动静。 19# 进入 /opt/ambari/apache-ambari-2.7.5-src/ambari-logsearch/ambari-logsearch-web/target/webapp-build 目录手动执行 yarn install --ignore-engines --pure-lockfile 命令 20 21# 会输出报错信息如下： 22[4/4] Building fresh packages... 23[-/3] ⠂ waiting... 24[2/3] ⠂ node-sass 25warning Error running install script for optional dependency: "/opt/ambari/apache-ambari-2.7.5-src/ambari-logsearch/ambari-logsearch-web/target/webapp-build/node_modules/node-sass: Command failed. 26Exit code: 1 27Command: node scripts/build.js 28Arguments: 29Directory: /opt/ambari/apache-ambari-2.7.5-src/ambari-logsearch/ambari-logsearch-web/target/webapp-build/node_modules/node-sass 30Output: 31Building: /usr/bin/node /opt/ambari/apache-ambari-2.7.5-src/ambari-logsearch/ambari-logsearch-web/target/webapp-build/node_modules/node-gyp/bin/node-gyp.js rebuild --verbose --libsass_ext= --libsass_cflags= --libsass_ldflags= --libsass_library= 32gyp info it worked if it ends with ok 33gyp verb cli [ '/usr/bin/node', 34gyp verb cli '/opt/ambari/apache-ambari-2.7.5-src/ambari-logsearch/ambari-logsearch-web/target/webapp-build/node_modules/node-gyp/bin/node-gyp.js', 35gyp verb cli 'rebuild', 36gyp verb cli '--verbose', 37gyp verb cli '--libsass_ext=', 38gyp verb cli '--libsass_cflags=', 39gyp verb cli '--libsass_ldflags=', 40gyp verb cli '--libsass_library=' ] 41gyp info using node-gyp@3.6.2 42gyp info using node@10.16.3 | linux | x64 43gyp verb command rebuild [] 44gyp verb command clean [] 45gyp verb clean removing \"build\" directory 46gyp verb command configure [] 47gyp verb check python checking for Python executable \"python2\" in the PATH 48gyp verb `which` succeeded python2 /bin/python2 49gyp verb check python version `/bin/python2 -c \"import platform; print(platform.python_version());\"` returned: \"2.7.5\\n\" 50gyp verb get node dir no --target version specified, falling back to host node version: 10.16.3 51gyp verb command install [ '10.16.3' ] 52gyp verb install input version string \"10.16.3\" 53gyp verb install installing version: 10.16.3 54gyp verb install --ensure was passed, so won't reinstall if already installed 55gyp verb install version not already installed, continuing with install 10.16.3 56gyp verb ensuring nodedir is created /root/.node-gyp/10.16.3 57gyp verb created nodedir /root/.node-gyp/10.16.3 58gyp http GET https://nodejs.org/download/release/v10.16.3/node-v10.16.3-headers.tar.gz 59gyp WARN install got an error, rolling back install 60gyp verb command remove [ '10.16.3' ] 61gyp verb remove using node-gyp dir: /root/.node-gyp 62gyp verb remove removing target version: 10.16.3 63gyp verb remove removing development files for version: 10.16.3 64gyp ERR! configure error 65gyp ERR! stack Error: This is most likely not a problem with node-gyp or the package itself and 66gyp ERR! stack is related to network connectivity. In most cases you are behind a proxy or have bad 67gyp ERR! stack network settings. 68gyp ERR! stack at Request.<anonymous> (/opt/ambari/apache-ambari-2.7.5-src/ambari-logsearch/ambari-logsearch-web/target/webapp-build/node_modules/node-gyp/lib/install.js:193:21) 69gyp ERR! stack at Request.emit (events.js:198:13) 70gyp ERR! stack at Request.onRequestError (/opt/ambari/apache-ambari-2.7.5-src/ambari-logsearch/ambari-logsearch-web/target/webapp-build/node_modules/request/request.js:884:8) 71gyp ERR! stack at ClientRequest.emit (events.js:198:13) 72gyp ERR! stack at TLSSocket.socketErrorListener (_http_client.js:392:9) 73gyp ERR! stack at TLSSocket.emit (events.js:198:13) 74gyp ERR! stack at emitErrorNT (internal/streams/destroy.js:91:8) 75gyp ERR! stack at emitErrorAndCloseNT (internal/streams/destroy.js:59:3) 76gyp ERR! stack at process._tickCallback (internal/process/next_tick.js:63:19) 77gyp ERR! System Linux 3.10.0-957.21.3.el7.x86_64 78gyp ERR! command \"/usr/bin/node\" \"/opt/ambari/apache-ambari-2.7.5-src/ambari-logsearch/ambari-logsearch-web/target/webapp-build/node_modules/node-gyp/bin/node-gyp.js\" \"rebuild\" \"--verbose\" \"--libsass_ext=\" \"--libsass_cflags=\" \"--libsass_ldflags=\" \"--libsass_library=\" 79gyp ERR! cwd /opt/ambari/apache-ambari-2.7.5-src/ambari-logsearch/ambari-logsearch-web/target/webapp-build/node_modules/node-sass 80gyp ERR! node -v v10.16.3 81gyp ERR! node-gyp -v v3.6.2 82gyp ERR! not ok 83[-/3] ⢀ waiting... 84[-/3] ⢀ waiting... 85[3/3] ⢀ phantomjs-prebuilt 86 87# 提示的是 node-gyp configure 报错 88# 解决：移除 $HOME/.node-gyp，然后重新安装最新的 node-gyp 包 89mv /root/.node-gyp /root/.node-gyp-bk 90npm i node-gyp -g 91 92# 删除 93find ~/.m2/repository/ -name "*.lastUpdated" -exec rm -rf {} \; 94 95# 重新编译 96mvn -B install rpm:rpm -DnewVersion=2.7.5.0.0 -DbuildNumber=5895e4ed6b30a2da8a90fee2403b6cab91d19972 -DskipTests -Dpython.ver="python >= 2.6" -Drat.skip=true -rf :ambari-logsearch-web 97 98# 接着报错 99[ERROR] PhantomJS not found on PATH 100[ERROR] Downloading https://github.com/Medium/phantomjs/releases/download/v2.1.1/phantomjs-2.1.1-linux-x86_64.tar.bz2 101[ERROR] Saving to /tmp/logsearch_npm_config_tmp/phantomjs/phantomjs-2.1.1-linux-x86_64.tar.bz2 102[ERROR] Receiving... 103[ERROR] 104[ERROR] Error making request. 105[ERROR] Error: connect ETIMEDOUT 52.216.16.232:443 106[ERROR] at Object._errnoException (util.js:1019:11) 107[ERROR] at _exceptionWithHostPort (util.js:1041:20) 108[ERROR] at TCPConnectWrap.afterConnect [as oncomplete] (net.js:1175:14) 109[ERROR] 110[ERROR] Please report this full log at https://github.com/Medium/phantomjs 111 112# cp -a /root/.node-gyp-bk/* /root/.node-gyp/ 113# 手动下载 phantomjs-2.1.1，放在/tmp/logsearch_npm_config_tmp/phantomjs/目录下 114# 继续重新编译 115mvn -B install rpm:rpm -DnewVersion=2.7.5.0.0 -DbuildNumber=5895e4ed6b30a2da8a90fee2403b6cab91d19972 -DskipTests -Dpython.ver="python >= 2.6" -Drat.skip=true -rf :ambari-logsearch-web
1.4 编译成功
1[INFO] Ambari Logsearch Web ............................... SUCCESS [05:16 min] 2[INFO] Ambari Logsearch Server ............................ SUCCESS [ 32.776 s] 3[INFO] Ambari Logsearch Assembly .......................... SUCCESS [ 0.450 s] 4[INFO] Ambari Logsearch Integration Test .................. SUCCESS [ 23.482 s] 5[INFO] ambari-infra ....................................... SUCCESS [ 0.352 s] 6[INFO] Ambari Infra Solr Client ........................... SUCCESS [ 7.718 s] 7[INFO] Ambari Infra Solr Plugin ........................... SUCCESS [ 10.133 s] 8[INFO] Ambari Infra Manager ............................... SUCCESS [ 23.206 s] 9[INFO] Ambari Infra Assembly .............................. SUCCESS [ 0.327 s] 10[INFO] Ambari Infra Manager Integration Tests ............. SUCCESS [ 12.458 s] 11[INFO] ------------------------------------------------------------------------ 12[INFO] BUILD SUCCESS 13[INFO] ------------------------------------------------------------------------ 14[INFO] Total time: 07:07 min 15[INFO] Finished at: 2020-03-09T11:42:28+08:00 16[INFO] ------------------------------------------------------------------------
生成的rpm包如下：
1# 服务端 2/opt/ambari/apache-ambari-2.7.5-src/ambari-server/target/rpm/ambari-server/RPMS/x86_64/ 3ambari-server-2.7.5.0-0.x86_64.rpm 4 5# 客户端 6/opt/ambari/apache-ambari-2.7.5-src/ambari-agent/target/rpm/ambari-agent/RPMS/x86_64/ 7mbari-agent-2.7.5.0-0.x86_64.rpm
1.5 安装 Ambari Server
Install the rpm package from ambari-server/target/rpm/ambari-server/RPMS/noarch/
1# 安装 2cd /opt/ambari/apache-ambari-2.7.5-src/ambari-server/target/rpm/ambari-server/RPMS/x86_64/ 3yum install ambari-server*.rpm 4 5 6# 配置 7ambari-server setup 8 9# 10Database admin user (postgres): 11Enter choice (1): 1 12Database admin user (postgres): 13Database name (ambari): 14Postgres schema (ambari): 15Username (ambari): 16Enter Database Password (bigdata): 17 18 19# 启动 20ambari-server start 21 22 23# 访问 24ip:8080 25admin/admin
1.6 安装
登入后，点击 launch install wizard 启动安装向导
查看版本的时候为空。
从Ambari 2.7.5版本开始，访问Ambari存储库需要进行身份验证。 要访问二进制文件，您必须首先具有必需的身份验证凭据（用户名和密码）。Cloudera向已注册的支持联系人发送的电子邮件中提供了新客户和合作伙伴的身份验证凭据。 现有用户可以在支持门户（https://my.cloudera.com）上提交非技术案例以获取凭证。以前，Ambari存储库位于AWS S3上。 自HDP 3.1.5 / Ambari 2.7.5起，存储库已移至https://archive.cloudera.com
1#1. 访问下面网址进行注册 2https://sso.cloudera.com/register.html