Kord•5mo ago

[Kotlin/Native] Slow connection issues

i'm experiencing noticeable connection issues with Kord (branch feature/native on Linux). about 90% of interactions fail, i can't even defer them to respond later. so far i couldn't upload files too, and it takes 10 seconds for a message to be sent (and retrieved afterwards). what's strange is that it always takes 10 seconds. here's the code for the last part:

val (message, restLatency) = measureTimedValue {
    ctx.message.reply("Calculating...") // this is sent instantly though
}

message.edit {
    content = buildString {
        append("Gateway: ")
        appendLine(ctx.kord.gateway.averagePing ?: "unavailable")
        append("REST: ")
        append(restLatency)
    }
}

val (message, restLatency) = measureTimedValue {
    ctx.message.reply("Calculating...") // this is sent instantly though
}

message.edit {
    content = buildString {
        append("Gateway: ")
        appendLine(ctx.kord.gateway.averagePing ?: "unavailable")
        append("REST: ")
        append(restLatency)
    }
}

no such issues occur when i run on JVM. could this be related to Ktor, or maybe the curl engine itself?

32 Replies

igrushka otchimaOP•5mo ago

Kord won't let me use the CIO engine either: TLS sessions are not supported on Native platform.

viztea•5mo ago

well its a ktor thing not kord send trace logs

igrushka otchimaOP•5mo ago

https://pastebin.com/9erkUipj

igrushka otchimaOP•5mo ago

though the first 2-3 requests were fine

igrushka otchimaOP•5mo ago

i dont even know whats going on anymore

viztea•5mo ago

you can try manually instrumenting the http client to rule out rate limiter problems but it’s most likely a curl problem

igrushka otchimaOP•5mo ago

how would i do that?

viztea•5mo ago

custom plugin or i think Http call logging is an official one

viztea•5mo ago

https://xgob.in/hvbkk7ss

gobin

viztea•5mo ago

is mine would need to remove the engine name thingy

igrushka otchimaOP•5mo ago

did i just need to install it as a plugin? if so, here are the logs:

[] [DISCOVERED]:[BUCKET]:Bucket discovered for 3df15bae86f6647dd4dfcbd5c6949480
TRACE: [] [Http/Ktor: Curl] <- POST https://discord.com/api/v10/channels/1191449585458094180/messages 200 OK - 10.772144703s
DEBUG: [[R]:[KTOR]:[ExclusionRequestRateLimiter]] [RESPONSE]:200:POST:https://discord.com/api/v10/channels/1191449585458094180/messages body:{"content":"Calculating...", ...}
DEBUG: [[R]:[KTOR]:[ExclusionRequestRateLimiter]] [REQUEST]:PATCH:/channels/{channel.id}/messages/{message.id} params:[{channel.id}=1191449585458094180,{message.id}=1274360378347884575] body:{"content":"Gateway: 39.631ms\nREST: 10.773614199s"}
TRACE: [] [Http/Ktor: Curl] <- PATCH https://discord.com/api/v10/channels/1191449585458094180/messages/1274360378347884575
TRACE: [] Received raw frame: Frame BINARY (fin=true, buffer len = 91)
TRACE: [] Gateway <<< {"t":"MESSAGE_UPDATE","s":4,"op":0,"d":{"content":"y.ping", ...}
TRACE: [] [DISCOVERED]:[BUCKET]:Bucket discovered for 3df15bae86f6647dd4dfcbd5c6949480
TRACE: [] [Http/Ktor: Curl] <- PATCH https://discord.com/api/v10/channels/1191449585458094180/messages/1274360378347884575 200 OK - 10.153825131s
DEBUG: [[R]:[KTOR]:[ExclusionRequestRateLimiter]] [RESPONSE]:200:PATCH:https://discord.com/api/v10/channels/1191449585458094180/messages/1274360378347884575 body:{"content":"Gateway: 39.631ms\nREST: 10.773614199s", ...}

[] [DISCOVERED]:[BUCKET]:Bucket discovered for 3df15bae86f6647dd4dfcbd5c6949480
TRACE: [] [Http/Ktor: Curl] <- POST https://discord.com/api/v10/channels/1191449585458094180/messages 200 OK - 10.772144703s
DEBUG: [[R]:[KTOR]:[ExclusionRequestRateLimiter]] [RESPONSE]:200:POST:https://discord.com/api/v10/channels/1191449585458094180/messages body:{"content":"Calculating...", ...}
DEBUG: [[R]:[KTOR]:[ExclusionRequestRateLimiter]] [REQUEST]:PATCH:/channels/{channel.id}/messages/{message.id} params:[{channel.id}=1191449585458094180,{message.id}=1274360378347884575] body:{"content":"Gateway: 39.631ms\nREST: 10.773614199s"}
TRACE: [] [Http/Ktor: Curl] <- PATCH https://discord.com/api/v10/channels/1191449585458094180/messages/1274360378347884575
TRACE: [] Received raw frame: Frame BINARY (fin=true, buffer len = 91)
TRACE: [] Gateway <<< {"t":"MESSAGE_UPDATE","s":4,"op":0,"d":{"content":"y.ping", ...}
TRACE: [] [DISCOVERED]:[BUCKET]:Bucket discovered for 3df15bae86f6647dd4dfcbd5c6949480
TRACE: [] [Http/Ktor: Curl] <- PATCH https://discord.com/api/v10/channels/1191449585458094180/messages/1274360378347884575 200 OK - 10.153825131s
DEBUG: [[R]:[KTOR]:[ExclusionRequestRateLimiter]] [RESPONSE]:200:PATCH:https://discord.com/api/v10/channels/1191449585458094180/messages/1274360378347884575 body:{"content":"Gateway: 39.631ms\nREST: 10.773614199s", ...}

viztea•5mo ago

yeah curl moment yeah this is happening to me as well

viztea•5mo ago

lmao

viztea•5mo ago

it's weird though, some requests go through instantly like that 'Connected to message'

igrushka otchimaOP•5mo ago

hmm i'll go ahead and test ktor itself

viztea•5mo ago

jesus fuck kotlin/native regex parsing is SLOW

igrushka otchimaOP•5mo ago

how slow are we talking lmao i didnt notice anything strange, only the first requests seem to be a little slow due to warmup that aside, my bot also does http requests here and there, and uses supabase-kt to communicate with the database with no connection issues whatsoever

viztea•5mo ago

yeah it might be discord only lol 38.561442049s

igrushka otchimaOP•5mo ago

goddamn

viztea•3mo ago

DOT_MATCHES_ALL has shit perf bit of a necropost but @vyfor can you try using a noop request rate limiter?

object NoopRequestRateLimiter : RequestRateLimiter {
    override suspend fun await(request: Request<*, *>): RequestToken = NoopRequestToken()
}

class NoopRequestToken : RequestToken {
    override var completed: Boolean = false

    override suspend fun complete(response: RequestResponse) {
        completed = true
    }
}

object NoopRequestRateLimiter : RequestRateLimiter {
    override suspend fun await(request: Request<*, *>): RequestToken = NoopRequestToken()
}

class NoopRequestToken : RequestToken {
    override var completed: Boolean = false

    override suspend fun complete(response: RequestResponse) {
        completed = true
    }
}

        requestHandler {
            KtorRequestHandler(token, NoopRequestRateLimiter)
        }

        requestHandler {
            KtorRequestHandler(token, NoopRequestRateLimiter)
        }

this fixes the problem for me so if it does for you too then it would narrow down the problem cc @LustigerLurch i should try using the parallel rate-limiter doesn't seem to happen with parallel either

LustigerLurch•3mo ago

gotta look into rest ratelimiters at some point

igrushka otchimaOP•3mo ago

sorry for the very late reply, i totally missed this message. i'll give it a go rn

igrushka otchimaOP•3mo ago

of course kord wont build for me

viztea•3mo ago

windows moment

igrushka otchimaOP•3mo ago

doubt it https://discord.com/channels/556525343595298817/1262041229944815709/1263457765272195184

igrushka otchimaOP•3mo ago

yup that was the case

igrushka otchimaOP•3mo ago

wish they had setup an action to republish kordx.emoji whenever kord is republished yup that seems to be working so far

LustigerLurch•3mo ago

pulling all into one repo would also solve this, been having this in the back of my head for some time

igrushka otchimaOP•3mo ago

hmm in case you dont end up merging them into one, you might wanna look into repository_dispatch not sure why it's happening in the first place, having to republish every time is a pain

SchlaubiBus•2mo ago

I am still against putting everything into one repo

LustigerLurch•2mo ago

i forgot why, so why?

SchlaubiBus•2mo ago

Esentially I don't want to make another androidx The biggest problem with monorepos like that is: 1) slower build time 2) slower import time 3) Slower CI runs, since it's very complicated to only trigger the CI for the one part that changed 4) If we do a snapshot, we always need to update kordx.emoji as well, otherwise it won't compile, even if we just want to do a prototype of the actual feature first 5) Either we write a convoluted detection logic mentioned in 3) or kordx (and probabbly cache if we go that route) have identical versions to kord, which would end up in "empty" releases for thoes projects, which is exactly the problem with androidx's jetpack

Gaming

Programming

[Kotlin/Native] Slow connection issues