Users with slowness and latency coming from Brazil / Rio de Janeiro
Many users specifically from the State of Rio de Janeiro, coming from various ASNs, report latency to reach all of our sites that are routed through Cloudflare.
This problem has been occurring for 5 days. Solutions?
thanks in advance
16 Replies
It shows the round trip avg there as 13.5ms, no? Seems pretty ok to me. Packet loss on a hop doesn't matter if it doesn't continue, which that doesn't
Hi @Chaika thanks!
In general the problems is latency on the 5th hop. All the connections ends on our server but all them that come from Rio de Janeiro happens with slows.
Many users claim to open the website. Avg > 15secs to just open the website. In others cities the avg to open the first page is 1sec
The latency of a specific hop doesn't matter unless it increases there and stays until the end. To routers, responding to Ping/ICMP is lowest priority and has to happen on the CPU instead of normal traffic which goes over hardware accelerated parts. There's a huge difference between sending stuff to a router vs through a router
If you're interested in learning more about how to debug things with traceroute, https://archive.nanog.org/sites/default/files/10_Roisman_Traceroute.pdf is a good read.
Anyway there's nothing that stands out with that trace, no packet loss at destination and the average/jitter isn't very high.
I would try to get more info on the slowness. On what exact asset/resource is being slow? Is it cached? etc
@Chaika thank you!
I dunno if this helps to understand.
We've 15 domains on Cloudflare, some of them in Pro account. We'd tested all of them using Any Desk in our clients and all had the same problem. It's so slow to open the homepages and browse in our systems. And all of then are hosted in Rio de Janeiro. The other 80% of our clients don't have any problems.
We temporarily shutdown the proxy from one new subdomain and the problem doesn't happen.
Can you show the headers tab? There's a few I would pay attention to:
CF-Ray, the end bit of that header ex: -ORD is the Airport Code of the Cloudflare location, and CF-Cache-Status if you hit cache or notHello @Chaika , sorry to no reply before.
I'll gonna check it
ache-control:
no-store, no-cache, must-revalidate
cf-cache-status:
DYNAMIC
cf-ray:
8ace86f93e26cae3-GIG
content-encoding:
br
content-type:
text/html; charset=UTF-8
date:
Fri, 02 Aug 2024 13:48:44 GMT
Request URL:
https://gestao.sistemacorban.com.br/index.php
Request Method:
GET
Status Code:
200 OK
Remote Address:
[2606:4700:20::681a:b21]:443
Referrer Policy:
strict-origin-when-cross-origin
alt-svc:
h3=":443"; ma=86400
cache-control:
no-store, no-cache, must-revalidate
cf-cache-status:
DYNAMIC
cf-ray:
8ace9be419477a78-GIG
content-encoding:
br
It's dynamic so it's going to your origin. If you go to one of those static (hopefully cached) assets like jquery.min.js or any of the .css's, that is cf-cache-status: HIT, what's the latency on those?
@Chaika The latency of these files is at acceptable levels. The problem is already underway, before the package reaches Cloudflare. I believe that in DNS caching by the backbone mesh here in Brazil. We created a subdomain with a CNAME name and it performs very well in traceroute and pingplotter. We asked the customer community in Rio de Janeiro to use a new route and the issue was resolved. In the meantime we will wait for the main link to normalize.
@Chaika Thank you so much for your time, I'm appreciate
Is this still occurring? If so, can you please get a traceroute/mtr to the destination from them, while they're experiencing the slowness?
also the
colo they see from https://gestao.sistemacorban.com.br/cdn-cgi/trace at the same timeThanks for reply @Br1ce . We have not had any reports of new slowdowns since we created a new CNAME entry in our DNS and shared it with our impacted customers. I will ask my ground team to carry out new tests on our main links and collect the requested data and then return this in this post.
Thanks @KAmorim - only if it's still ongoing
You said 5 days - do you know exactly when the issue started?
Hello @Br1ce
I'm sorry for not responding quickly.
I contacted the user support team and they just got back to me now.
The problem of slowness and high latency remains in most requests coming from Rio de Janeiro/Brazil to Clouflare.
Our support started reporting this slowness last Monday (July 29th)
Below is information obtained from users who are currently experiencing slowdowns when accessing the gestao.sistemacorban.com.br domain
When they call an alias (crm.sistemacorban.com.br) this problem does not occur (this alias is just a CNAME entry pointing to gestao.sistemacorban.com.br)
fl=937f70
h=gestao.sistemacorban.com.br
ip=177.23.183.233
ts=1722886018.696
visit_scheme=https
uag=Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/127.0.0.0 Safari/537.36
colo=GIG
sliver=010-gig09
http=http/3
loc=BR
tls=TLSv1.3
sni=plaintext
warp=off
gateway=off
rbi=off
kex=X25519Kyber768Draft00
fl=216f263
h=gestao.sistemacorban.com.br
ip=2a09:bac5:aea:878::d8:107
ts=1722886089.67
visit_scheme=https
uag=Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/127.0.0.0 Safari/537.36
colo=GIG
sliver=none
http=http/3
loc=BR
tls=TLSv1.3
sni=plaintext
warp=on
gateway=off
rbi=off
kex=X25519Kyber768Draft00
fl=937f80
h=gestao.sistemacorban.com.br
ip=2804:16d8:b738💯6ccd:4a98:f67a:57b5
ts=1722886168.668
visit_scheme=https
uag=Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/127.0.0.0 Safari/537.36
colo=GIG
sliver=none
http=http/3
loc=BR
tls=TLSv1.3
sni=plaintext
warp=off
gateway=off
rbi=off
kex=X25519Kyber768Draft00
fl=216f411
h=gestao.sistemacorban.com.br
ip=2804:14d:7285:8a8b:74b9:1ac8:a9cb:730c
ts=1722886334.224
visit_scheme=https
uag=Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/127.0.0.0 Safari/537.36 Edg/127.0.0.0
colo=GIG
sliver=none
http=http/3
loc=BR
tls=TLSv1.3
sni=plaintext
warp=off
gateway=off
rbi=off
kex=X25519Kyber768Draft00
@Br1ce hello!
So, some users from Rio de Janeiro who use the temporary URL (CNAME entry) also claim slowness and high latency.
We created a new one for now (no latency claimed yet)
Trace a slow user
fl=937f76
h=crm.sistemacorban.com.br
ip=191.242.124.222
ts=1722890198.972
visit_scheme=https
uag=Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/127.0.0.0 Safari/537.36
colo=GIG
sliver=none
http=http/2
loc=BR
tls=TLSv1.3
sni=plaintext
warp=off
gateway=off
rbi=off
kex=X25519
Trace for the same user to the new one Entry
fl=937f104
h=crmrj.sistemacorban.com.br
ip=191.242.124.222
ts=1722890968.121
visit_scheme=https
uag=Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/127.0.0.0 Safari/537.36
colo=GIG
sliver=none
http=http/2
loc=BR
tls=TLSv1.3
sni=plaintext
warp=off
gateway=off
rbi=off
kex=X25519
Hello @Br1ce , good morning!
We have reports from other partners with similar problems on Cloudflare sites that pass through the Brazil/Rio de Janeiro Tunnel to Brazil/São Paulo through Equinix Datacenters.
When the Route does not pass through this tunnel, the connection is successful.
All traces point to colo=GIG.
I don't know if it's related to this, but we urgently need support on this as we are experiencing impaired traffic coming from this region on all of our endpoints.