Downloading data before building pages (SSG/SSR)
I have the use case that I'm experimenting building a Cloudflare Pages project using Astro.
It will using a dynamic GraphQL endpoint which queries data from a D1 database.
However it will also generate static pages (SSG).
For these static pages I have a local SQLite file that will be queries to dynamically create the (static) pages.
Locally on my development machine this works perfectly.
However, I would like to switch to a mode that I automatically re-deploy my site via a push to Github.
Now that local SQLite is ~60MB (zipped ~6MB) and I don't want to upload that Github...
Is it possible to either download that file during the build process, or upload it somewhere before the build kicks off?
Note that the SQLite file (zip) is publicly available so that's not an issue.
I'm looking at the possibilities I have; and what possible limitations are.
10 Replies
Add a
curl https://your.domain/zipped.sqlite -o data/static.sqlite to the beginning of your build command?Cool, that would be great indeed.
That's why I posted this question to see if something like that is possible, and that CF allow it as well.
Thx!
Does CF also support the unzip command? (to unzip the downloaded zip)
I'm not 100% sure, probably best to just try and see?
Yes I will try, thx for your help! I appreciate it.
have you tried uploading it to R2 and fetching it from your code ?
Not yet, but I assume there must be some kind of hook in Astro (at build start) to retrieve that file during the build process?
I wonder if during the build process I have access to R2?
if your data is public, you can link your r2 bucket to a domain and fetch your file
fetch('https://r2.mydomain.com/mydb.sqlite')
yes it's an asset (but zipped) published on github releases
just download it when you need it in your build
you can use a unzip lib in javascript to unzip it if you need
sql.js
sql.js is an SQL library for javascript containing a version of SQLite compiled for the web.