星火对象运行时错误

Question

在本地系统中得到错误程序运行时可作为 我的内存大小为3GB，需要解决

Exception in thread "main" java.lang.IllegalArgumentException: System memory 259522560 must be at least 471859200. Please increase heap size using the --driver-memory option or spark.driver.memory in Spark configuration. 
    at org.apache.spark.memory.UnifiedMemoryManager$.getMaxMemory(UnifiedMemoryManager.scala:216) 
    at org.apache.spark.memory.UnifiedMemoryManager$.apply(UnifiedMemoryManager.scala:198) 
    at org.apache.spark.SparkEnv$.create(SparkEnv.scala:330) 
    at org.apache.spark.SparkEnv$.createDriverEnv(SparkEnv.scala:174) 
    at org.apache.spark.SparkContext.createSparkEnv(SparkContext.scala:257) 
    at org.apache.spark.SparkContext.<init>(SparkContext.scala:432) 
    at org.apache.spark.SparkContext$.getOrCreate(SparkContext.scala:2313) 
    at org.apache.spark.sql.SparkSession$Builder$$anonfun$6.apply(SparkSession.scala:868) 
    at org.apache.spark.sql.SparkSession$Builder$$anonfun$6.apply(SparkSession.scala:860) 
    at scala.Option.getOrElse(Option.scala:121) 
    at org.apache.spark.sql.SparkSession$Builder.getOrCreate(SparkSession.scala:860) 
    at SparkCore.cartesianTransformation$.main(cartesianTransformation.scala:11) 
    at SparkCore.cartesianTransformation.main(cartesianTransformation.scala) 

Answer 1

看来你的火花驱动程序在小内存中运行的尝试增加的大小驱动器内存。 您可以使用--driver-memory 4g向驱动程序提供内存大小。

希望这会有所帮助！