斯卡拉对象与ArrayBuffer - 反序列化问题，使用GSON

Question

我有一个顶层的Scala类类似下面：

FinalOutput.scala：

class FinalOutput extends Serializable { 

    @BeanProperty 
    var userId: String = _ 

    @BeanProperty 
    var tenantId: String = _ 

    @BeanProperty 
    @SerializedName("type") 
    var dataType: String = _ 

    @BeanProperty 
    var data: FinalData = _ 

    @BeanProperty 
    var userCreatedDate: String = _ 
} 

FinalData.scala：

class FinalData extends Serializable { 

    @BeanProperty 
    var list1: ArrayBuffer[DataType1] = _ 

    @BeanProperty 
    var list2: ArrayBuffer[DataType2] = _ 

    @BeanProperty 
    var list3: ArrayBuffer[DataType3] = _ 

    @BeanProperty 
    var list4: ArrayBuffer[DataType4] = _ 

    .... 
    .... 

    @BeanProperty 
    var list15: ArrayBuffer[DataType15] = _ 

    @BeanProperty 
    var userName: String = _ 
} 

以及全部DataType*延伸的类BaseBean

我用this将Scala对象序列化成json字符串。

ArrayBufferSerializer.scala

class ArrayBufferSerializer[T: ClassTag] extends JsonSerializer[ArrayBuffer[T]] { 
    override def serialize(src: ArrayBuffer[T], typeOfSrc: Type, context: JsonSerializationContext): JsonElement = { 
    context.serialize(src.toArray[Any]) 
    } 
} 

然后序列化为字符串中使用这样的：

val gson = new GsonBuilder().registerTypeAdapter(classOf[ArrayBuffer[FinalData]], new ArrayBufferSerializer[FinalData]()).serializeNulls.create 
val data = gson.toJson(row) 

现在我想做的反序列化JSON字符串FinalOutput对象，所以我创建ArrayBufferDeSerializer东西像这样

class ArrayBufferDeSerializer[T: ClassTag] extends JsonDeserializer[ArrayBuffer[T]] { 
    override def deserialize(json: JsonElement, typeOfT: Type, context: JsonDeserializationContext): ArrayBuffer[T] = { 
    context.deserialize(json, typeOfT) 
    } 
} 

然后调用下面deserialzie：

val gson = new GsonBuilder().registerTypeAdapter(classOf[ArrayBuffer[FinalData]], new ArrayBufferSerializer[FinalData]()).serializeNulls.create 
gson.fromJson(row, classOf[FinalLevelOneSmsOutput]) 

得到以下错误：

Exception in thread "main" org.apache.spark.SparkException: Job aborted due to stage failure: Task 0 in stage 2.0 failed 1 times, most recent failure: Lost task 0.0 in stage 2.0 (TID 3, localhost, executor driver): java.lang.StackOverflowError 
    at com.google.gson.internal.bind.TypeAdapters$29.read(TypeAdapters.java:720) 
    at com.google.gson.internal.bind.TypeAdapters$29.read(TypeAdapters.java:743) 
    at com.google.gson.internal.bind.TypeAdapters$29.read(TypeAdapters.java:735) 
    at com.google.gson.internal.bind.TypeAdapters$29.read(TypeAdapters.java:718) 
    at com.google.gson.internal.Streams.parse(Streams.java:48) 
    at com.google.gson.TreeTypeAdapter.read(TreeTypeAdapter.java:54) 
    at com.google.gson.Gson.fromJson(Gson.java:861) 
    at com.google.gson.Gson.fromJson(Gson.java:926) 
    at com.google.gson.Gson$1.deserialize(Gson.java:131) 
    at com.cv.util.ArrayBufferDeSerializer.deserialize(ArrayBufferDeSerializer.scala:15) 
    at com.cv.util.ArrayBufferDeSerializer.deserialize(ArrayBufferDeSerializer.scala:13) 
    at com.google.gson.TreeTypeAdapter.read(TreeTypeAdapter.java:58) 

Answer 1

你解串器不只是委派反序列化回context使用相同的参数（相同的JSON和同类型），这会导致上下文再次调用解串行器 - 这会创建无限循环并生成StackOverflowError。

解串器必须改进 - 因为我们已经系列化ArrayBuffer s转换“简单”的阵列，我们必须反序列化他们相应。这里有一种方法可以做到这一点：

import com.google.gson.reflect.TypeToken 

class ArrayBufferDeSerializer[T: ClassTag] extends JsonDeserializer[ArrayBuffer[T]] { 
    override def deserialize(json: JsonElement, typeOfT: Type, context: JsonDeserializationContext): ArrayBuffer[T] = { 
    // since we've serialized ArrayBuffers by converting them to simple Arrays, we deserialize the 
    // input as a simple Array first: 
    val array: util.ArrayList[T] = context.deserialize(json, new TypeToken[Array[T]](){}.getType) 

    // Then, we convert it back into an ArrayBuffer: 
    import collection.JavaConverters._ 
    ArrayBuffer[T](array.asScala: _*) 
    } 
}