CD
Cloudflare Developers•9mo ago
Kaien

Increased latency of domain when I turn on "proxied" in Proxy Status in my DNS settings

I am running my production app api on this domain "prod.example.app". I have noticed delays of roughly 100 to 200 ms when I enable Proxy Status to "Proxied". I have verified this multiple ways. I have even added cache rules/page rules to avoid all caching and it does show cache status as "DYNAMIC" which means it being bypassed but then why is the responses slow in general then ? It just works fine of static content. Example: when I hit https://prod.example.app/folders/staticfile.jpg it just works fine and returns me the file in like < 200ms but then when I hit some GET API's on the same domain like: https://prod.example.app/api/v1/news the response time increases to 300-400 which generally without the proxy status - "proxied" is like always < 250ms Is this normal for CF to do this ? I need to have "proxied" on since I want to hide the actual Cname record of my amazon load balancer
21 Replies
Chaika
Chaika•9mo ago
Easiest explaination is usually routing. If you go to the special path /cdn-cgi/trace you can see the Airport Code of the CF location you are being routed to, under colo=, ex: https://prod.example.app/cdn-cgi/trace. Does it make sense? Which country are you in?
Kaien
KaienOP•9mo ago
I am in India following is the output: fl=631f63 h=prod.example.app ip=2405:201:d032:f00e:4dfe:4e79:cc51:dd2 ts=1712804832.246 visit_scheme=https uag=Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/123.0.0.0 Safari/537.36 colo=SIN sliver=010-tier1 http=http/3 loc=IN tls=TLSv1.3 sni=plaintext warp=off gateway=off rbi=off kex=X25519
Chaika
Chaika•9mo ago
There's been quite a few issues reported over the past few months of Indian ISPs with suboptimal routing doesn't look too bad though, looks like you're going to Singapore Where is your AWS backend located?
Kaien
KaienOP•9mo ago
Yeah. I am just a little concerned how this is going to look like to end-users when they get additional of 200 ms of latency Mumbai ap-south-1 region
Chaika
Chaika•9mo ago
It just works fine of static content. Example: when I hit https://prod.example.app/folders/staticfile.jpg it just works fine and returns me the file in like < 200ms but then when I hit some GET API's on the same domain like: https://prod.example.app/api/v1/news the response time increases to 300-400 which generally without the proxy status - "proxied" is like always < 250ms
Kind of curious too if it's just on API routes and not static routes if they're on the same backend, unless staticfile.jpg is being cached at all (which it would be by default, although you said you made rules to stop it) if they're on the exact same hostname, going to the exact same servers/lb, one is static (with low latency) and one is dynamic (with higher latency), and neither have caching enabled, I would think it'd be 99% your backend being slow. The route would be identical from user -> cf -> origin
Kaien
KaienOP•9mo ago
well, the static file is on a different domain which is "proxied" here is the real domain https://medial.app/image/placeholder.jpg ( the static file )
Kaien
KaienOP•9mo ago
and this is the backend endpoint which is being slow https://protoprod.medial.app/api/v1/news
Chaika
Chaika•9mo ago
that's being cached
Kaien
KaienOP•9mo ago
Yes note that, both of these domains have similar configuration. that is "proxied" one @Chaika my question is: why is it being so slower in api response time when I have configured a cache/page rules to bypass ? you say its because of it is routing from a singapore region, right ?
Chaika
Chaika•9mo ago
the disabling cache rules have nothing to do with why your dynamic endpoint would be slow, it's not going to be cached regardless by default. In your original comment, you're trying to compare a cached asset (on CF's Edge) to an uncached dynamic api (which has to go all the way to the origin) what you should be more comparing is just https://prod.medial.app/api/v1/news with https://protoprod.medial.app/api/v1/news, you get a 200ms diference between those?
Kaien
KaienOP•9mo ago
Yes yes, correct.
Kaien
KaienOP•9mo ago
No description
Kaien
KaienOP•9mo ago
No description
Kaien
KaienOP•9mo ago
You see ? rougly 100 ms of difference
Chaika
Chaika•9mo ago
what latency do you get to https://protoprod.medial.app/cdn-cgi/trace? I would also send just a few to try to avoid issues with keepalives and such, REST Clients don't make the best benchmarking tools
Kaien
KaienOP•9mo ago
fl=411f370 h=protoprod.medial.app ip=2405:201:d032:f00e:4dfe:4e79:cc51:dd2 ts=1712805762.345 visit_scheme=https uag=Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/123.0.0.0 Safari/537.36 colo=SIN sliver=010-sin06 http=http/3 loc=IN tls=TLSv1.3 sni=plaintext warp=off gateway=off rbi=off kex=X25519
Chaika
Chaika•9mo ago
nah I meant latency in ms, not the output not the most scientific way of testing, but the latency increase in your screenshots above is 88ms. If you get ~40ms one way to Singapore, it would explain that slowdown
Kaien
KaienOP•9mo ago
roughly 55-60 ms
Chaika
Chaika•9mo ago
think that makes sense then. 88ms isn't too bad imo, espec when your api is already > 200, and your requests are going India (You) -> Singapore (CF) -> India and back I have a VPS in Mumbai which is routed to the local colo (BOM) and the difference is ~10ms or so 
:~#  curl -w "@curl-format.txt" -o /dev/null -s "https://protoprod.medial.app/api/v1/news"
   time_namelookup:  0.007241s
        time_connect:  0.007845s
     time_appconnect:  0.033444s
    time_pretransfer:  0.033612s
       time_redirect:  0.000000s
  time_starttransfer:  0.049732s
                     ----------
          time_total:  0.049806s
:~#  curl -w "@curl-format.txt" -o /dev/null -s "https://prod.medial.app/api/v1/news"
   time_namelookup:  0.006102s
        time_connect:  0.007349s
     time_appconnect:  0.035670s
    time_pretransfer:  0.035783s
       time_redirect:  0.000000s
  time_starttransfer:  0.038462s
                     ----------
          time_total:  0.038528s
:~#  curl -w "@curl-format.txt" -o /dev/null -s "https://protoprod.medial.app/api/v1/news"
   time_namelookup:  0.007241s
        time_connect:  0.007845s
     time_appconnect:  0.033444s
    time_pretransfer:  0.033612s
       time_redirect:  0.000000s
  time_starttransfer:  0.049732s
                     ----------
          time_total:  0.049806s
:~#  curl -w "@curl-format.txt" -o /dev/null -s "https://prod.medial.app/api/v1/news"
   time_namelookup:  0.006102s
        time_connect:  0.007349s
     time_appconnect:  0.035670s
    time_pretransfer:  0.035783s
       time_redirect:  0.000000s
  time_starttransfer:  0.038462s
                     ----------
          time_total:  0.038528s
prob better peering/just better location for aws Routing is a combination of plans (higher plans get more locations, some locations have limited capacity/expensive bandwidth), iirc some of those Indian ISPs were wanting to charge for peering/connectivity as well, which also hurts/prevents CF from making free tier have good local connectivity
Chaika
Chaika•9mo ago
You could try seeing if a higher plan would route you closer: https://debug.chaika.me/?findColo=true https://cloudflare.manfredi.io/en/tools/datacenters/ nothing's guaranteed though, routing can shift.
Cloudflare Tools
Cloudflare Tools
A set of unofficial tools to check connectivity to the Cloudflare network.
Kaien
KaienOP•9mo ago
Well, I appreciate your responses so much. Let me research a little more. I will be back with what I do.
Want results from more Discord servers?
Add your server