louiek22
louiek22
CDCloudflare Developers
Created by louiek22 on 10/14/2024 in #workers-help
fetch() sometimes returning 520 even though request to origin was successful
For context on volume: this was 300 req/s pushing an average of 30 rows
9 replies
CDCloudflare Developers
Created by louiek22 on 10/14/2024 in #workers-help
fetch() sometimes returning 520 even though request to origin was successful
I recreated the same requests the SDK did, byte-for-byte and then just compressed the body
9 replies
CDCloudflare Developers
Created by louiek22 on 10/14/2024 in #workers-help
fetch() sometimes returning 520 even though request to origin was successful
It wasn't compressing the request body. It was literally that one change that fixed it.
9 replies
CDCloudflare Developers
Created by louiek22 on 10/14/2024 in #workers-help
fetch() sometimes returning 520 even though request to origin was successful
cc @Chaika, I think you'll find this interesting. And thanks for pointing me in the right direction with mentioning that you use the HTTP api directly!
9 replies
CDCloudflare Developers
Created by louiek22 on 10/14/2024 in #workers-help
fetch() sometimes returning 520 even though request to origin was successful
Here's the compression function I user:
const compress = async (
str: string,
encoding = "gzip" as "gzip"
): Promise<ArrayBuffer> => {
const byteArray = new TextEncoder().encode(str)
const cs = new CompressionStream(encoding)
const writer = cs.writable.getWriter()
writer.write(byteArray)
writer.close()
return new Response(cs.readable).arrayBuffer()
}
const compress = async (
str: string,
encoding = "gzip" as "gzip"
): Promise<ArrayBuffer> => {
const byteArray = new TextEncoder().encode(str)
const cs = new CompressionStream(encoding)
const writer = cs.writable.getWriter()
writer.write(byteArray)
writer.close()
return new Response(cs.readable).arrayBuffer()
}
9 replies
CDCloudflare Developers
Created by louiek22 on 10/14/2024 in #workers-help
fetch() sometimes returning 520 even though request to origin was successful
Solved it somehow. I stopped using the Clickhouse SDK and instead made the requests from scratch myself. It took a bit of work. In addition to making the requests myself I had to compress the request body. No 520s after I started compressing the body.
9 replies