spark常见问题

spark与scala的对应关系

scala与spark的对应关系

spark提交作业方式

折叠代码块BASH 复制代码

# 使用yarn-client
/usr/local/spark/bin/spark-submit \
--class org.apache.spark.example.JavaSparkPi \
--master yarn-client \
--num-executors 1 \
--driver-memory 10m \
--executor-memory 10m \
--executor-cores 1 \
/usr/local/spark/lib/spark-examples-1.5.1-hadoop2.4.0.jar \

# 使用yarn-cluster
/usr/local/spark/bin/spark-submit \
--class org.apache.spark.example.JavaSparkPi \
--master yarn-cluster \
--num-executors 1 \
--driver-memory 10m \
--executor-memory 10m \
--executor-cores 1 \
/usr/local/spark/lib/spark-examples-1.5.1-hadoop2.4.0.jar \