准备主机

Master bigdata-1 10.110.18.221

修改/etc/hosts

10.110.18.221 bigdata-1.bigdata.com bigdata-1

安装并配置jdk(略过)

解压并安装spark,配置spark

export HADOOP_CONF_DIR=/usr/local/hadoop-2.7.2/etc/hadoop

验证Spark

 bin/spark-submit \
--class org.apache.spark.examples.SparkPi \
    --master yarn \
    --deploy-mode cluster \
    lib/spark-examples*.jar \
    10

此时可以在hadoop目录/usr/local/hadoop-2.7.2/logs/userlogs 查看日志

访问spark应用的web ui地址http://10.110.18.221:8080

在yarn中查看http://10.110.18.222.8080

results matching ""

    No results matching ""