Kord•11mo ago

(Not Kord) Deserialize JSON with unordered fields (Kotlin Serialization)

Hello

I have this deserialization function from my custom Serializer:

override fun deserialize(decoder: Decoder): PacketWrapper {
        return decoder.decodeStructure(descriptor) {
            var type: Byte? = null
            var data: Packet? = null

            if(decodeSequentially()) {
                type = decodeByteElement(descriptor, 0)
                val serializer = typeToPacket[type] ?: throw SerializationException("The field 'type' is required")
                data = decodeSerializableElement(descriptor, 1, serializer)
                PacketWrapper(data)
            } else {
                loop@ while (true) {
                    when (val index = decodeElementIndex(descriptor)) {
                        0 -> type = decodeByteElement(descriptor, index)
                        1 -> {
                            val serializer = typeToPacket[type] ?: throw SerializationException("The field 'type' is required")
                            data = decodeSerializableElement(descriptor, index, serializer as KSerializer<Packet>)
                        }
                        CompositeDecoder.DECODE_DONE -> break@loop
                        else -> throw SerializationException("Unexpected index: $index")
                    }
                }

                type ?: throw SerializationException("The field 'type' is required")
                data ?: throw SerializationException("The field 'data' is required")
                PacketWrapper(data)
            }
        }
    }

override fun deserialize(decoder: Decoder): PacketWrapper {
        return decoder.decodeStructure(descriptor) {
            var type: Byte? = null
            var data: Packet? = null

            if(decodeSequentially()) {
                type = decodeByteElement(descriptor, 0)
                val serializer = typeToPacket[type] ?: throw SerializationException("The field 'type' is required")
                data = decodeSerializableElement(descriptor, 1, serializer)
                PacketWrapper(data)
            } else {
                loop@ while (true) {
                    when (val index = decodeElementIndex(descriptor)) {
                        0 -> type = decodeByteElement(descriptor, index)
                        1 -> {
                            val serializer = typeToPacket[type] ?: throw SerializationException("The field 'type' is required")
                            data = decodeSerializableElement(descriptor, index, serializer as KSerializer<Packet>)
                        }
                        CompositeDecoder.DECODE_DONE -> break@loop
                        else -> throw SerializationException("Unexpected index: $index")
                    }
                }

                type ?: throw SerializationException("The field 'type' is required")
                data ?: throw SerializationException("The field 'data' is required")
                PacketWrapper(data)
            }
        }
    }

When the JSON is:

{"type":0,"data":{"message":"Hello, World!"}}

{"type":0,"data":{"message":"Hello, World!"}}

That works perfectly.

But when the json is:

{"data":{"message":"Hello, World!"},"type":0}

{"data":{"message":"Hello, World!"},"type":0}

The exception

The field 'type' is required

The field 'type' is required

(from the loop) is thrown.

is there a way to support unordered fields?

Solution

@Serializable
sealed interface Packet {
  val data: Data
  
  interface Data
}

@Serializable
@SerialName("hello")
data class HelloPacket(override val data: Data) : Packet {
  @Serializable
  data class Data(val pingInterval: Int) : Packet.Data
}

@Serializable
sealed interface Packet {
  val data: Data
  
  interface Data
}

@Serializable
@SerialName("hello")
data class HelloPacket(override val data: Data) : Packet {
  @Serializable
  data class Data(val pingInterval: Int) : Packet.Data
}

Jump to solution

SchlaubiBus•1/25/25, 5:15 PM

Yeah don't use a custom serializr, an auto generated serializer could handle that

SchlaubiBus•1/25/25, 5:16 PM

You can also use a JsonContentPolymorphicSerializer (i think that was the name), or save everything into a map first

SSchlaubiBus You can also use a JsonContentPolymorphicSerializer (i think that was the name),...

TicOP•1/25/25, 7:53 PM

Yes and no
In the idea, I would to have something like that:

If I use JsonDiscriminator:

The type is a name, but I would like to have a byte
The type will be in the nested
```
data
```
```
data
```
object and not out with other metadata

SchlaubiBus•1/25/25, 7:55 PM

yeah sadly kx.ser cannot do that, the best way is to not use a byte or use the JsonContentPolymorphicSerializer

TicOP•1/25/25, 7:56 PM

Yes but there is a problem again, the structure will be:

SchlaubiBus•1/25/25, 7:56 PM

doesn't have to

SchlaubiBus•1/25/25, 7:56 PM

you can make something like this

Solution

SchlaubiBus•1/25/25, 7:58 PM

@Serializable
sealed interface Packet {
  val data: Data
  
  interface Data
}

@Serializable
@SerialName("hello")
data class HelloPacket(override val data: Data) : Packet {
  @Serializable
  data class Data(val pingInterval: Int) : Packet.Data
}

@Serializable
sealed interface Packet {
  val data: Data
  
  interface Data
}

@Serializable
@SerialName("hello")
data class HelloPacket(override val data: Data) : Packet {
  @Serializable
  data class Data(val pingInterval: Int) : Packet.Data
}

TicOP•1/25/25, 8:02 PM

Yes that's what I did at the beginning
I found the syntax a little bit annoying (but probably less than my terrible serializer ah ah)
And also the thing that pushed me on my weird solution is for the byte for the type. I need to reduce the network latency and of course a byte is more quick to transfer than string

SchlaubiBus•1/25/25, 8:03 PM

do you use json?