准备主机
Master bigdata-1 10.110.18.221
修改/etc/hosts
10.110.18.221 bigdata-1.bigdata.com bigdata-1
安装并配置jdk(略过)
解压并安装spark,配置spark
export HADOOP_CONF_DIR=/usr/local/hadoop-2.7.2/etc/hadoop
验证Spark
bin/spark-submit \
--class org.apache.spark.examples.SparkPi \
--master yarn \
--deploy-mode cluster \
lib/spark-examples*.jar \
10
此时可以在hadoop目录/usr/local/hadoop-2.7.2/logs/userlogs
查看日志
访问spark应用的web ui地址http://10.110.18.221:8080
在yarn中查看http://10.110.18.222.8080