(Relatively) high latency compared to other services

I noticed that Cloudflare Pages latency is (relatively) quite a bit high compared to Vercel and Netlify. I sent 100 requests to each of these providers and measured the latency to download a txt file containing just Hello!, only counting requests that used an already established connection to the server: Vercel:
95th Percentile: 23.9ms
50th Percentile: 9.9ms
95th Percentile: 23.9ms
50th Percentile: 9.9ms
Netlify:
95th Percentile: 7.9ms
50th Percentile: 6.8ms
95th Percentile: 7.9ms
50th Percentile: 6.8ms
Cloudflare Pages:
95th Percentile: 82.3ms
50th Percentile: 52.6ms
95th Percentile: 82.3ms
50th Percentile: 52.6ms
This is from a VPS from Hetzner in Germany. I get similar results from my home connection in the Netherlands. How come Cloudflare Pages has the highest latency, given the amount of PoPs Cloudflare has? Don't get me wrong, I absolutely love Cloudflare Pages, just wondering if there's something unusual going on here.
32 Replies
thanhle
thanhle6mo ago
As far I understand. because you query the database. so it is better to have server close to database (like Vercel and Netlify, they use Serverless Function AWS Lambda under the hood) than the server close to user (like Cloudflare Worker for example). You can watch a video of Fireship explain about this (from 2.32):https://www.youtube.com/watch?v=yOP5-3_WFus
Fireship
YouTube
Is "edge" computing really faster?
Edge computing is becoming a popular way to deliver modern web applications. Let’s find out if the Edge is really faster by comparing Firebase to Vercel Edge Functions with Next.js. #webdevelopment #javascript #vs
🔗 Resources Next Firebase Course https://fireship.io/courses/react-next-firebase/ Next.js 12.2 Release https://nextjs.org/blog/...
Rowin
Rowin6mo ago
I am not querying a database, just serving a static file Not using a serverless/edge function Still wondering about this.
Chaika
Chaika6mo ago
This is from a VPS from Hetzner in Germany. I get similar results from my home connection in the Netherlands. How come Cloudflare Pages has the highest latency, given the amount of PoPs Cloudflare has?
Pages uses KV for static assets. KV only has two central stores, one in EU, one in US, and the rest of the locations just cache the value. More requests, more cache hits, less latency. I'd also check which location you are hitting. You can go to /cdn-cgi/trace path and look at colo= to see airport code of location you are hitting. https://developers.cloudflare.com/ for example is a Pages Static Site with high traffic (and Enterprise routing).
Rowin
Rowin6mo ago
By "the rest of the locations just cache the value", do you mean the regular Cloudflare CDN cache, or something KV specific? Asking because I didn't notice the latency going down after a bunch of requests. Makes sense if it's the former, because I was sending requests to the pages.dev domain, which doesn't cache(?). I already checked the colo and it was fine, my bad for not mentioning it. Seems like you meant the latter, judging by this page: https://developers.cloudflare.com/kv/reference/how-kv-works/
Chaika
Chaika6mo ago
It's pretty close to the normal cdn cache, being colo specific, but not exactly it. It's on both pages.dev and custom domains, ignore cf-cache-status. I would be curious what your latency is to something completely on edge like https://cloudflare.com/cdn-cgi/trace ~50ms is pretty high for cached results Also, what you define as "fine"?
I already checked the colo and it was fine, my bad for not mentioning it.
If you're using Hetzner falkenstein/germany, should be fra
Rowin
Rowin6mo ago
Within the same country, I believe it was indeed FRA AMS from my home connection, ~5 ms ping HTTP:
$ ./httping-windows-amd64.exe -enable-keep-alive -no-new-conn-count -count 10 https://cloudflare.com/cdn-cgi/trace
dns=10.8ms conn=7.4ms tls=22.7ms ttfb=50.2ms dl=0.7ms total=50.8ms reused=false proto=HTTP/2.0 status=200 OK error=N/A
dns=N/A conn=N/A tls=N/A ttfb=11.1ms dl=0.0ms total=12.2ms reused=true proto=HTTP/2.0 status=200 OK error=N/A
dns=N/A conn=N/A tls=N/A ttfb=7.8ms dl=0.0ms total=7.8ms reused=true proto=HTTP/2.0 status=200 OK error=N/A
dns=N/A conn=N/A tls=N/A ttfb=8.4ms dl=0.0ms total=9.0ms reused=true proto=HTTP/2.0 status=200 OK error=N/A
dns=N/A conn=N/A tls=N/A ttfb=8.3ms dl=0.0ms total=8.3ms reused=true proto=HTTP/2.0 status=200 OK error=N/A
dns=N/A conn=N/A tls=N/A ttfb=8.5ms dl=0.5ms total=9.1ms reused=true proto=HTTP/2.0 status=200 OK error=N/A
dns=N/A conn=N/A tls=N/A ttfb=7.9ms dl=0.0ms total=7.9ms reused=true proto=HTTP/2.0 status=200 OK error=N/A
dns=N/A conn=N/A tls=N/A ttfb=8.1ms dl=0.7ms total=8.8ms reused=true proto=HTTP/2.0 status=200 OK error=N/A
dns=N/A conn=N/A tls=N/A ttfb=7.5ms dl=0.6ms total=8.1ms reused=true proto=HTTP/2.0 status=200 OK error=N/A
dns=N/A conn=N/A tls=N/A ttfb=9.9ms dl=0.8ms total=10.7ms reused=true proto=HTTP/2.0 status=200 OK error=N/A
$ ./httping-windows-amd64.exe -enable-keep-alive -no-new-conn-count -count 10 https://cloudflare.com/cdn-cgi/trace
dns=10.8ms conn=7.4ms tls=22.7ms ttfb=50.2ms dl=0.7ms total=50.8ms reused=false proto=HTTP/2.0 status=200 OK error=N/A
dns=N/A conn=N/A tls=N/A ttfb=11.1ms dl=0.0ms total=12.2ms reused=true proto=HTTP/2.0 status=200 OK error=N/A
dns=N/A conn=N/A tls=N/A ttfb=7.8ms dl=0.0ms total=7.8ms reused=true proto=HTTP/2.0 status=200 OK error=N/A
dns=N/A conn=N/A tls=N/A ttfb=8.4ms dl=0.0ms total=9.0ms reused=true proto=HTTP/2.0 status=200 OK error=N/A
dns=N/A conn=N/A tls=N/A ttfb=8.3ms dl=0.0ms total=8.3ms reused=true proto=HTTP/2.0 status=200 OK error=N/A
dns=N/A conn=N/A tls=N/A ttfb=8.5ms dl=0.5ms total=9.1ms reused=true proto=HTTP/2.0 status=200 OK error=N/A
dns=N/A conn=N/A tls=N/A ttfb=7.9ms dl=0.0ms total=7.9ms reused=true proto=HTTP/2.0 status=200 OK error=N/A
dns=N/A conn=N/A tls=N/A ttfb=8.1ms dl=0.7ms total=8.8ms reused=true proto=HTTP/2.0 status=200 OK error=N/A
dns=N/A conn=N/A tls=N/A ttfb=7.5ms dl=0.6ms total=8.1ms reused=true proto=HTTP/2.0 status=200 OK error=N/A
dns=N/A conn=N/A tls=N/A ttfb=9.9ms dl=0.8ms total=10.7ms reused=true proto=HTTP/2.0 status=200 OK error=N/A
ICMP:
Reply from 104.16.133.229: bytes=32 time=5ms TTL=59
Reply from 104.16.133.229: bytes=32 time=5ms TTL=59
Reply from 104.16.133.229: bytes=32 time=5ms TTL=59
Reply from 104.16.133.229: bytes=32 time=5ms TTL=59
Reply from 104.16.133.229: bytes=32 time=5ms TTL=59
Reply from 104.16.133.229: bytes=32 time=5ms TTL=59
Reply from 104.16.133.229: bytes=32 time=5ms TTL=59
Reply from 104.16.133.229: bytes=32 time=5ms TTL=59
Reply from 104.16.133.229: bytes=32 time=5ms TTL=59
Reply from 104.16.133.229: bytes=32 time=5ms TTL=59
Pretty close But then, to a Pages file:
$ ./httping-windows-amd64.exe -enable-keep-alive -no-new-conn-count -count 10 https://hello-7v1.pages.dev/hello.txt
dns=11.2ms conn=7.3ms tls=32.3ms ttfb=117.3ms dl=0.0ms total=117.3ms reused=false proto=HTTP/2.0 status=200 OK error=N/A
dns=N/A conn=N/A tls=N/A ttfb=49.0ms dl=0.0ms total=49.0ms reused=true proto=HTTP/2.0 status=200 OK error=N/A
dns=N/A conn=N/A tls=N/A ttfb=43.7ms dl=0.0ms total=44.4ms reused=true proto=HTTP/2.0 status=200 OK error=N/A
dns=N/A conn=N/A tls=N/A ttfb=37.9ms dl=0.0ms total=38.9ms reused=true proto=HTTP/2.0 status=200 OK error=N/A
dns=N/A conn=N/A tls=N/A ttfb=38.2ms dl=0.0ms total=38.2ms reused=true proto=HTTP/2.0 status=200 OK error=N/A
dns=N/A conn=N/A tls=N/A ttfb=46.0ms dl=0.0ms total=46.8ms reused=true proto=HTTP/2.0 status=200 OK error=N/A
dns=N/A conn=N/A tls=N/A ttfb=42.2ms dl=0.0ms total=43.0ms reused=true proto=HTTP/2.0 status=200 OK error=N/A
dns=N/A conn=N/A tls=N/A ttfb=45.0ms dl=0.0ms total=45.0ms reused=true proto=HTTP/2.0 status=200 OK error=N/A
dns=N/A conn=N/A tls=N/A ttfb=44.3ms dl=0.0ms total=44.9ms reused=true proto=HTTP/2.0 status=200 OK error=N/A
dns=N/A conn=N/A tls=N/A ttfb=37.3ms dl=0.0ms total=37.3ms reused=true proto=HTTP/2.0 status=200 OK error=N/A
$ ./httping-windows-amd64.exe -enable-keep-alive -no-new-conn-count -count 10 https://hello-7v1.pages.dev/hello.txt
dns=11.2ms conn=7.3ms tls=32.3ms ttfb=117.3ms dl=0.0ms total=117.3ms reused=false proto=HTTP/2.0 status=200 OK error=N/A
dns=N/A conn=N/A tls=N/A ttfb=49.0ms dl=0.0ms total=49.0ms reused=true proto=HTTP/2.0 status=200 OK error=N/A
dns=N/A conn=N/A tls=N/A ttfb=43.7ms dl=0.0ms total=44.4ms reused=true proto=HTTP/2.0 status=200 OK error=N/A
dns=N/A conn=N/A tls=N/A ttfb=37.9ms dl=0.0ms total=38.9ms reused=true proto=HTTP/2.0 status=200 OK error=N/A
dns=N/A conn=N/A tls=N/A ttfb=38.2ms dl=0.0ms total=38.2ms reused=true proto=HTTP/2.0 status=200 OK error=N/A
dns=N/A conn=N/A tls=N/A ttfb=46.0ms dl=0.0ms total=46.8ms reused=true proto=HTTP/2.0 status=200 OK error=N/A
dns=N/A conn=N/A tls=N/A ttfb=42.2ms dl=0.0ms total=43.0ms reused=true proto=HTTP/2.0 status=200 OK error=N/A
dns=N/A conn=N/A tls=N/A ttfb=45.0ms dl=0.0ms total=45.0ms reused=true proto=HTTP/2.0 status=200 OK error=N/A
dns=N/A conn=N/A tls=N/A ttfb=44.3ms dl=0.0ms total=44.9ms reused=true proto=HTTP/2.0 status=200 OK error=N/A
dns=N/A conn=N/A tls=N/A ttfb=37.3ms dl=0.0ms total=37.3ms reused=true proto=HTTP/2.0 status=200 OK error=N/A
Chaika
Chaika6mo ago
do you still hit a close pop if you ping hello-7v1.pages.dev or go to https://hello-7v1.pages.dev/cdn-cgi/trace?
Rowin
Rowin6mo ago
Just checked, still AMS (From the Netherlands, so that's good)
Chaika
Chaika6mo ago
do you get anything better if trying against https://developers.cloudflare.com?
Rowin
Rowin6mo ago
$ ./httping-windows-amd64.exe -enable-keep-alive -no-new-conn-count -count 10 https://developers.cloudflare.com/robots.txt
dns=8.9ms conn=7.2ms tls=25.5ms ttfb=128.6ms dl=0.0ms total=129.3ms reused=false proto=HTTP/2.0 status=200 OK error=N/A
dns=N/A conn=N/A tls=N/A ttfb=64.3ms dl=0.7ms total=65.0ms reused=true proto=HTTP/2.0 status=200 OK error=N/A
dns=N/A conn=N/A tls=N/A ttfb=63.7ms dl=0.8ms total=64.5ms reused=true proto=HTTP/2.0 status=200 OK error=N/A
dns=N/A conn=N/A tls=N/A ttfb=64.6ms dl=0.6ms total=65.2ms reused=true proto=HTTP/2.0 status=200 OK error=N/A
dns=N/A conn=N/A tls=N/A ttfb=80.7ms dl=0.0ms total=81.0ms reused=true proto=HTTP/2.0 status=200 OK error=N/A
dns=N/A conn=N/A tls=N/A ttfb=66.6ms dl=0.6ms total=67.2ms reused=true proto=HTTP/2.0 status=200 OK error=N/A
dns=N/A conn=N/A tls=N/A ttfb=61.1ms dl=0.9ms total=62.0ms reused=true proto=HTTP/2.0 status=200 OK error=N/A
dns=N/A conn=N/A tls=N/A ttfb=76.6ms dl=0.0ms total=77.1ms reused=true proto=HTTP/2.0 status=200 OK error=N/A
dns=N/A conn=N/A tls=N/A ttfb=64.5ms dl=0.5ms total=64.9ms reused=true proto=HTTP/2.0 status=200 OK error=N/A
dns=N/A conn=N/A tls=N/A ttfb=77.9ms dl=0.6ms total=78.4ms reused=true proto=HTTP/2.0 status=200 OK error=N/A
$ ./httping-windows-amd64.exe -enable-keep-alive -no-new-conn-count -count 10 https://developers.cloudflare.com/robots.txt
dns=8.9ms conn=7.2ms tls=25.5ms ttfb=128.6ms dl=0.0ms total=129.3ms reused=false proto=HTTP/2.0 status=200 OK error=N/A
dns=N/A conn=N/A tls=N/A ttfb=64.3ms dl=0.7ms total=65.0ms reused=true proto=HTTP/2.0 status=200 OK error=N/A
dns=N/A conn=N/A tls=N/A ttfb=63.7ms dl=0.8ms total=64.5ms reused=true proto=HTTP/2.0 status=200 OK error=N/A
dns=N/A conn=N/A tls=N/A ttfb=64.6ms dl=0.6ms total=65.2ms reused=true proto=HTTP/2.0 status=200 OK error=N/A
dns=N/A conn=N/A tls=N/A ttfb=80.7ms dl=0.0ms total=81.0ms reused=true proto=HTTP/2.0 status=200 OK error=N/A
dns=N/A conn=N/A tls=N/A ttfb=66.6ms dl=0.6ms total=67.2ms reused=true proto=HTTP/2.0 status=200 OK error=N/A
dns=N/A conn=N/A tls=N/A ttfb=61.1ms dl=0.9ms total=62.0ms reused=true proto=HTTP/2.0 status=200 OK error=N/A
dns=N/A conn=N/A tls=N/A ttfb=76.6ms dl=0.0ms total=77.1ms reused=true proto=HTTP/2.0 status=200 OK error=N/A
dns=N/A conn=N/A tls=N/A ttfb=64.5ms dl=0.5ms total=64.9ms reused=true proto=HTTP/2.0 status=200 OK error=N/A
dns=N/A conn=N/A tls=N/A ttfb=77.9ms dl=0.6ms total=78.4ms reused=true proto=HTTP/2.0 status=200 OK error=N/A
(robots.txt because it's a very small file like my hello.txt) (Seems like it returns CF-Cache-Status: MISS every time)
Chaika
Chaika6mo ago
I was picking the index because I doubt robots.txt is very heavily visited/cached ignore that, that's just cdn cache, pages/kv has its own internal
Rowin
Rowin6mo ago
Same idea, just higher dl because the file is bigger:
$ ./httping-windows-amd64.exe -enable-keep-alive -no-new-conn-count -count 10 https://developers.cloudflare.com/
dns=12.5ms conn=7.4ms tls=25.5ms ttfb=142.9ms dl=3.0ms total=145.9ms reused=false proto=HTTP/2.0 status=200 OK error=N/A
dns=N/A conn=N/A tls=N/A ttfb=74.1ms dl=3.1ms total=77.7ms reused=true proto=HTTP/2.0 status=200 OK error=N/A
dns=N/A conn=N/A tls=N/A ttfb=57.6ms dl=5.5ms total=63.1ms reused=true proto=HTTP/2.0 status=200 OK error=N/A
dns=N/A conn=N/A tls=N/A ttfb=61.5ms dl=4.7ms total=66.3ms reused=true proto=HTTP/2.0 status=200 OK error=N/A
dns=N/A conn=N/A tls=N/A ttfb=68.3ms dl=5.7ms total=74.0ms reused=true proto=HTTP/2.0 status=200 OK error=N/A
dns=N/A conn=N/A tls=N/A ttfb=69.8ms dl=4.2ms total=74.0ms reused=true proto=HTTP/2.0 status=200 OK error=N/A
dns=N/A conn=N/A tls=N/A ttfb=58.1ms dl=4.6ms total=62.7ms reused=true proto=HTTP/2.0 status=200 OK error=N/A
dns=N/A conn=N/A tls=N/A ttfb=63.3ms dl=4.3ms total=67.5ms reused=true proto=HTTP/2.0 status=200 OK error=N/A
dns=N/A conn=N/A tls=N/A ttfb=73.2ms dl=4.5ms total=77.7ms reused=true proto=HTTP/2.0 status=200 OK error=N/A
dns=N/A conn=N/A tls=N/A ttfb=68.0ms dl=5.5ms total=73.4ms reused=true proto=HTTP/2.0 status=200 OK error=N/A
$ ./httping-windows-amd64.exe -enable-keep-alive -no-new-conn-count -count 10 https://developers.cloudflare.com/
dns=12.5ms conn=7.4ms tls=25.5ms ttfb=142.9ms dl=3.0ms total=145.9ms reused=false proto=HTTP/2.0 status=200 OK error=N/A
dns=N/A conn=N/A tls=N/A ttfb=74.1ms dl=3.1ms total=77.7ms reused=true proto=HTTP/2.0 status=200 OK error=N/A
dns=N/A conn=N/A tls=N/A ttfb=57.6ms dl=5.5ms total=63.1ms reused=true proto=HTTP/2.0 status=200 OK error=N/A
dns=N/A conn=N/A tls=N/A ttfb=61.5ms dl=4.7ms total=66.3ms reused=true proto=HTTP/2.0 status=200 OK error=N/A
dns=N/A conn=N/A tls=N/A ttfb=68.3ms dl=5.7ms total=74.0ms reused=true proto=HTTP/2.0 status=200 OK error=N/A
dns=N/A conn=N/A tls=N/A ttfb=69.8ms dl=4.2ms total=74.0ms reused=true proto=HTTP/2.0 status=200 OK error=N/A
dns=N/A conn=N/A tls=N/A ttfb=58.1ms dl=4.6ms total=62.7ms reused=true proto=HTTP/2.0 status=200 OK error=N/A
dns=N/A conn=N/A tls=N/A ttfb=63.3ms dl=4.3ms total=67.5ms reused=true proto=HTTP/2.0 status=200 OK error=N/A
dns=N/A conn=N/A tls=N/A ttfb=73.2ms dl=4.5ms total=77.7ms reused=true proto=HTTP/2.0 status=200 OK error=N/A
dns=N/A conn=N/A tls=N/A ttfb=68.0ms dl=5.5ms total=73.4ms reused=true proto=HTTP/2.0 status=200 OK error=N/A
Here's a file that does hit the CF cache (CF-Cache-Status: HIT) for comparison:
$ ./httping-windows-amd64.exe -enable-keep-alive -no-new-conn-count -count 10 https://developers.cloudflare.com/assets/speech-ad68e411.svg
dns=10.1ms conn=6.8ms tls=30.5ms ttfb=87.5ms dl=0.4ms total=87.9ms reused=false proto=HTTP/2.0 status=200 OK error=N/A
dns=N/A conn=N/A tls=N/A ttfb=25.6ms dl=0.8ms total=26.3ms reused=true proto=HTTP/2.0 status=200 OK error=N/A
dns=N/A conn=N/A tls=N/A ttfb=26.3ms dl=0.7ms total=27.0ms reused=true proto=HTTP/2.0 status=200 OK error=N/A
dns=N/A conn=N/A tls=N/A ttfb=27.1ms dl=0.0ms total=27.1ms reused=true proto=HTTP/2.0 status=200 OK error=N/A
dns=N/A conn=N/A tls=N/A ttfb=25.0ms dl=0.3ms total=25.3ms reused=true proto=HTTP/2.0 status=200 OK error=N/A
dns=N/A conn=N/A tls=N/A ttfb=26.0ms dl=0.3ms total=26.3ms reused=true proto=HTTP/2.0 status=200 OK error=N/A
dns=N/A conn=N/A tls=N/A ttfb=26.9ms dl=0.4ms total=27.3ms reused=true proto=HTTP/2.0 status=200 OK error=N/A
dns=N/A conn=N/A tls=N/A ttfb=26.7ms dl=0.3ms total=27.0ms reused=true proto=HTTP/2.0 status=200 OK error=N/A
dns=N/A conn=N/A tls=N/A ttfb=26.8ms dl=0.4ms total=27.2ms reused=true proto=HTTP/2.0 status=200 OK error=N/A
dns=N/A conn=N/A tls=N/A ttfb=27.8ms dl=0.7ms total=28.4ms reused=true proto=HTTP/2.0 status=200 OK error=N/A
$ ./httping-windows-amd64.exe -enable-keep-alive -no-new-conn-count -count 10 https://developers.cloudflare.com/assets/speech-ad68e411.svg
dns=10.1ms conn=6.8ms tls=30.5ms ttfb=87.5ms dl=0.4ms total=87.9ms reused=false proto=HTTP/2.0 status=200 OK error=N/A
dns=N/A conn=N/A tls=N/A ttfb=25.6ms dl=0.8ms total=26.3ms reused=true proto=HTTP/2.0 status=200 OK error=N/A
dns=N/A conn=N/A tls=N/A ttfb=26.3ms dl=0.7ms total=27.0ms reused=true proto=HTTP/2.0 status=200 OK error=N/A
dns=N/A conn=N/A tls=N/A ttfb=27.1ms dl=0.0ms total=27.1ms reused=true proto=HTTP/2.0 status=200 OK error=N/A
dns=N/A conn=N/A tls=N/A ttfb=25.0ms dl=0.3ms total=25.3ms reused=true proto=HTTP/2.0 status=200 OK error=N/A
dns=N/A conn=N/A tls=N/A ttfb=26.0ms dl=0.3ms total=26.3ms reused=true proto=HTTP/2.0 status=200 OK error=N/A
dns=N/A conn=N/A tls=N/A ttfb=26.9ms dl=0.4ms total=27.3ms reused=true proto=HTTP/2.0 status=200 OK error=N/A
dns=N/A conn=N/A tls=N/A ttfb=26.7ms dl=0.3ms total=27.0ms reused=true proto=HTTP/2.0 status=200 OK error=N/A
dns=N/A conn=N/A tls=N/A ttfb=26.8ms dl=0.4ms total=27.2ms reused=true proto=HTTP/2.0 status=200 OK error=N/A
dns=N/A conn=N/A tls=N/A ttfb=27.8ms dl=0.7ms total=28.4ms reused=true proto=HTTP/2.0 status=200 OK error=N/A
Chaika
Chaika6mo ago
hmm yea it's semi-known cf cache is faster then pages internal cache but that's a pretty decent jump pages does have to go cf for saas -> invoke worker -> kv cache (which is slower on its own)
Rowin
Rowin6mo ago
My thoughts exactly. worker processes are reused across requests when possible though, right?
Chaika
Chaika6mo ago
ehh not really and I doubt workers themselves are the issue
Want results from more Discord servers?
Add your server