Astro build caching

It’s actually .astro, but yes 🙂
46 Replies
Jacob
Jacob15mo ago
@itsmatteomanf What does an Astro project need for it to generate the .astro folder? I created a project with npm create astro@latest but building doesn't seem to generate the .astro folder anywhere Found this: https://docs.astro.build/en/guides/content-collections/#the-astro-directory Looks like I need a "content collection"
Hello, I’m Allie!
IIRC, that is only for caching Content Collection Types, so there might be something else for triggering the build cache…
Jacob
Jacob15mo ago
I was under the impression that Matteo had other stuff that can be cached via .astro - is it only types?
Hello, I’m Allie!
Found it: node_modules/.astro
Hello, I’m Allie!
Astro Documentation
Configuration Reference
Build faster websites with less client-side JavaScript.
Jacob
Jacob15mo ago
Ah! hmmm I wonder why I don't have a node_modules/.astro maybe the blog template doesn't have anything cachable?
itsmatteomanf
itsmatteomanfOP15mo ago
It’s used only for image caching, so you need to add an image anywhere (using their component)
itsmatteomanf
itsmatteomanfOP15mo ago
Astro Documentation
Images
Learn how to use images in Astro.
Jacob
Jacob15mo ago
How long does your project take to build? Trying to get a sense for how much benefit caching astro could have
itsmatteomanf
itsmatteomanfOP15mo ago
For a non-optimized image, especially if remote, it’s 1s per image even with it cached on Cloudflare. It can go up or down depending on the time it takes to load it.
Jacob
Jacob15mo ago
Do you happen to have a project you could share privately to test with?
itsmatteomanf
itsmatteomanfOP15mo ago
I currently serve basically optimized images and it would go from 3 minutes to 30s It used to be even worse given it did the same lower bound with unoptimized images, but started at like 6 minutes (locally), which Pages slowed to 12-22 minutes (didn’t understand the difference in runs, tbh) I could, yeah. Gonna DM you later 😉
Jacob
Jacob15mo ago
Thanks!
itsmatteomanf
itsmatteomanfOP15mo ago
@geostyx do note that Astro allows for changing that folder, so if you want to create a folder which is cached and people can use, it’s supported. Automatic is always nice, but just in case.
Jacob
Jacob15mo ago
Could potentially always cache .cache <:neocat_think:1139306117353193512> but that kinda feels like adding config in a backwards way Do you happen to know if caching dist in Astro is useful? Or does it always overwrite anyway?
itsmatteomanf
itsmatteomanfOP15mo ago
It’s deleted as soon as the build starts 🙂 They are implementing an incremental build option, not sure exactly what folder they’d use, but I’m presuming the same as the images?
Jacob
Jacob15mo ago
Cool, will stick to .astro and node_modules/.astro
Hello, I’m Allie!
No idea if this is planned, but it would be cool for the framework selector to have default cache locations for each framework
Jacob
Jacob15mo ago
What do you mean?
Hello, I’m Allie!
So like, if I select Astro as my framework in the project setup, it will automatically add node_modules/.astro to my cached directories
Jacob
Jacob15mo ago
There's currently no config for what directories get cached. It's all detected at build time. For example, if astro is in your package.json and node_modules/.astro exists, it will cache node_modules/.astro
itsmatteomanf
itsmatteomanfOP15mo ago
So I presume the cache wasn't caching all of node_modules, then?
Jacob
Jacob15mo ago
Yes it does
itsmatteomanf
itsmatteomanfOP15mo ago
So... why didn't my project see that node_modules/.astro/ and it totally ignored the cache? 🤔 It works perfectly locally, so it's definitely not Astro.
Jacob
Jacob15mo ago
Could you try doing ls -lh node_modules/.astro at the start of your build command?
itsmatteomanf
itsmatteomanfOP15mo ago
Doing it now 🙂 It helps I already have a ./build.sh script set-up ahah
Jacob
Jacob15mo ago
and DM me the deploy ID if you don't mind
itsmatteomanf
itsmatteomanfOP15mo ago
1be71ab3-0538-4a74-99ac-9847f0cfa1ce It's showing 80k, which is definitely small... Ohhh, wait. It's a subfolder. The folder is there.
2023-09-05T19:16:51.370462492Z total 80K
2023-09-05T19:16:51.370532511Z drwxr-xr-x 2 buildbot root 80K Sep 4 13:40 assets
2023-09-05T19:16:51.370462492Z total 80K
2023-09-05T19:16:51.370532511Z drwxr-xr-x 2 buildbot root 80K Sep 4 13:40 assets
Jacob
Jacob15mo ago
hmmm, can you do one more with ls -lhR?
itsmatteomanf
itsmatteomanfOP15mo ago
That one is still building, but gonna abort it lul the files exist. 🤔 7b3a7136-1dff-47ba-bc57-9432f8b41059
Jacob
Jacob15mo ago
I see it's a bunch of .json files - does it store the images somewhere else?
itsmatteomanf
itsmatteomanfOP15mo ago
No, it's saved in there in the JSON, it's weird lul remote images)
Jacob
Jacob15mo ago
wait
itsmatteomanf
itsmatteomanfOP15mo ago
There should be a expires key in the JSON object which is used to invalidate it if the Cache-Control headers expire.
Jacob
Jacob15mo ago
Maybe those are outdated - when we implemented node_modules caching, we weren't expecting it to cache build output. So it only saves that directory when your package-lock.json changes
itsmatteomanf
itsmatteomanfOP15mo ago
Ohhh, then yes. That's what is happening.
Jacob
Jacob15mo ago
Maybe you could try changing something in your package lock, run 2 more builds, and see if the second is faster?
itsmatteomanf
itsmatteomanfOP15mo ago
Gonna try. Maybe there is an update somewhere lul were updates, running it now. Had to remove the ls as the folder doesn't exist anymore now, given the cache is ignored. Oh, wait. It's not using it locally either, anymore. Something is wrong in the Cache-Control headers, I guess. Let me check. @geostyx so yeah, there is a bug in Astro not using the cache now lul sent a PR. It's gonna be merged soon, I bet ahah
Jacob
Jacob15mo ago
oop haha. It's ok, I had to put that down for a bit as it's blocked on some other internal work around build caching
itsmatteomanf
itsmatteomanfOP15mo ago
My PR was merged already, let's just say a PR like this is easy to validate OMEGALUL https://github.com/withastro/astro/pull/8424 And it's also quite a bug. I presume a release is actually coming soon. More precisely, it was using the cache. Just for expired items, and not valid ones ahah
Jacob
Jacob15mo ago
great catch!
itsmatteomanf
itsmatteomanfOP15mo ago
I almost went insane, there was nothing looking wrong... then it dawned on me, it clicked and I was like: waaaaait that is the wrong way around ahah Adding this here, I presume this node_modules/.astro/ folder is gonna end up in the build output caching saving logic? Doesn't make sense to save it only on package-lock.json changes.
Jacob
Jacob15mo ago
Correct
itsmatteomanf
itsmatteomanfOP15mo ago
I didn't post it, but 🎉
No description
itsmatteomanf
itsmatteomanfOP15mo ago
10m+ to 1m 55s 😄
Jacob
Jacob15mo ago
👀
Want results from more Discord servers?
Add your server