K
Kord•3d ago
Tic

(Not Kord) Deserialize JSON with unordered fields (Kotlin Serialization)

Hello I have this deserialization function from my custom Serializer: 
override fun deserialize(decoder: Decoder): PacketWrapper {
        return decoder.decodeStructure(descriptor) {
            var type: Byte? = null
            var data: Packet? = null

            if(decodeSequentially()) {
                type = decodeByteElement(descriptor, 0)
                val serializer = typeToPacket[type] ?: throw SerializationException("The field 'type' is required")
                data = decodeSerializableElement(descriptor, 1, serializer)
                PacketWrapper(data)
            } else {
                loop@ while (true) {
                    when (val index = decodeElementIndex(descriptor)) {
                        0 -> type = decodeByteElement(descriptor, index)
                        1 -> {
                            val serializer = typeToPacket[type] ?: throw SerializationException("The field 'type' is required")
                            data = decodeSerializableElement(descriptor, index, serializer as KSerializer<Packet>)
                        }
                        CompositeDecoder.DECODE_DONE -> break@loop
                        else -> throw SerializationException("Unexpected index: $index")
                    }
                }

                type ?: throw SerializationException("The field 'type' is required")
                data ?: throw SerializationException("The field 'data' is required")
                PacketWrapper(data)
            }
        }
    }
override fun deserialize(decoder: Decoder): PacketWrapper {
        return decoder.decodeStructure(descriptor) {
            var type: Byte? = null
            var data: Packet? = null

            if(decodeSequentially()) {
                type = decodeByteElement(descriptor, 0)
                val serializer = typeToPacket[type] ?: throw SerializationException("The field 'type' is required")
                data = decodeSerializableElement(descriptor, 1, serializer)
                PacketWrapper(data)
            } else {
                loop@ while (true) {
                    when (val index = decodeElementIndex(descriptor)) {
                        0 -> type = decodeByteElement(descriptor, index)
                        1 -> {
                            val serializer = typeToPacket[type] ?: throw SerializationException("The field 'type' is required")
                            data = decodeSerializableElement(descriptor, index, serializer as KSerializer<Packet>)
                        }
                        CompositeDecoder.DECODE_DONE -> break@loop
                        else -> throw SerializationException("Unexpected index: $index")
                    }
                }

                type ?: throw SerializationException("The field 'type' is required")
                data ?: throw SerializationException("The field 'data' is required")
                PacketWrapper(data)
            }
        }
    }
When the JSON is: 
{"type":0,"data":{"message":"Hello, World!"}}
{"type":0,"data":{"message":"Hello, World!"}}
That works perfectly. But when the json is: 
{"data":{"message":"Hello, World!"},"type":0}
{"data":{"message":"Hello, World!"},"type":0}
The exception The field 'type' is required (from the loop) is thrown. is there a way to support unordered fields?
Solution:
```kotlin @Serializable sealed interface Packet { val data: Data ...
Jump to solution
18 Replies
SchlaubiBus
SchlaubiBus•3d ago
Yeah don't use a custom serializr, an auto generated serializer could handle that You can also use a JsonContentPolymorphicSerializer (i think that was the name), or save everything into a map first
Tic
TicOP•3d ago
Yes and no In the idea, I would to have something like that: 
// PacketWrapper
{
  "type": 0,
  "data": {
    ... my packet structure
  }
  "other metadata": ..
}
// PacketWrapper
{
  "type": 0,
  "data": {
    ... my packet structure
  }
  "other metadata": ..
}
If I use JsonDiscriminator: - The type is a name, but I would like to have a byte - The type will be in the nested data object and not out with other metadata
SchlaubiBus
SchlaubiBus•3d ago
yeah sadly kx.ser cannot do that, the best way is to not use a byte or use the JsonContentPolymorphicSerializer
Tic
TicOP•3d ago
Yes but there is a problem again, the structure will be: 
{
  "data": {
    "type": 0,
    ... my packet structure
  }
  "other metadata": ..
}
{
  "data": {
    "type": 0,
    ... my packet structure
  }
  "other metadata": ..
}
SchlaubiBus
SchlaubiBus•3d ago
doesn't have to you can make something like this
Solution
SchlaubiBus
SchlaubiBus•3d ago
@Serializable
sealed interface Packet {
  val data: Data
  
  interface Data
}

@Serializable
@SerialName("hello")
data class HelloPacket(override val data: Data) : Packet {
  @Serializable
  data class Data(val pingInterval: Int) : Packet.Data
}
@Serializable
sealed interface Packet {
  val data: Data
  
  interface Data
}

@Serializable
@SerialName("hello")
data class HelloPacket(override val data: Data) : Packet {
  @Serializable
  data class Data(val pingInterval: Int) : Packet.Data
}
Tic
TicOP•3d ago
Yes that's what I did at the beginning I found the syntax a little bit annoying (but probably less than my terrible serializer ah ah) And also the thing that pushed me on my weird solution is for the byte for the type. I need to reduce the network latency and of course a byte is more quick to transfer than string
SchlaubiBus
SchlaubiBus•3d ago
do you use json?
Tic
TicOP•3d ago
yes
SchlaubiBus
SchlaubiBus•3d ago
well using a byte will make little difference then especially if you use numbers whether you encode 0 or "0" won't make a difference
Tic
TicOP•3d ago
aah and the serialName should be the number instead of "hello" (for example) that could work I wil ltry with JsonContentPolymorphicSerializer too to see this thing So sure there is no way for the number? :/
SchlaubiBus
SchlaubiBus•3d ago
not without a custom serializer no
Tic
TicOP•3d ago
Well I imagine two extra bytes is acceptable for the gain of maintenability
SchlaubiBus
SchlaubiBus•3d ago
if you're worried about traffic you shouldn't use json use some binary format like protobuf
Tic
TicOP•3d ago
true I will maybe check that. But need to check how handle that in front-end Hum I think proto could be a better solution
Tic
TicOP•3d ago
Protobuf read sequentially we're agree no? Because in a decoder, when I try to use decodeSequentially it returns false .. According to the documentation, that's not
SchlaubiBus
SchlaubiBus•3d ago
No idea tbh
Tic
TicOP•2d ago
Protobuf is definively the solution 👍 Thanks

Did you find this page helpful?