S
SolidJS•4w ago
oke

Do a one-time server operation upon page load, and then re-use that data

I am looking for a way to do a one-time database query for a very rarely changed dataset -> And then store that data into server memory for subsequent re-use More details: 1. I have a dynamic route /map/[country].tsx 2. Upon loading this page the first time, I want to query database for list of country and store that data into server memory. So that subsequent navigation to the map/[country].tsx can just index the maps object instead of querying the db again Currently, I have 
// map/[country].tsx
async function gatherData() {
  'use server'
  console.log("GATHERING")
  const maps = await db.findAll("maps")
  const mapObject = arrayToObject(maps)
  return mapObject
}

async loadMap(country) {
  'use server'
  console.log("LOADING FOR", country)
  const mapData = gatherData() // How to not call this again
  return mapData[country]
}

export default function CountryPage() {
  const params = useParams()
  const countryMap = createAsync(() => loadMap(params.country))
  return <p>{countryMap.name}</p>
}
// map/[country].tsx
async function gatherData() {
  'use server'
  console.log("GATHERING")
  const maps = await db.findAll("maps")
  const mapObject = arrayToObject(maps)
  return mapObject
}

async loadMap(country) {
  'use server'
  console.log("LOADING FOR", country)
  const mapData = gatherData() // How to not call this again
  return mapData[country]
}

export default function CountryPage() {
  const params = useParams()
  const countryMap = createAsync(() => loadMap(params.country))
  return <p>{countryMap.name}</p>
}
7 Replies
oke
okeOP•4w ago
Actually I just tried using top-level await with 
const MAP_DATA = await gatherData();

async loadMap(country) {
  'use server'
  console.log("LOADING FOR", country)
  return MAP_DATA[country]
}
const MAP_DATA = await gatherData();

async loadMap(country) {
  'use server'
  console.log("LOADING FOR", country)
  return MAP_DATA[country]
}
And it seems to work. Not sure if there are any footgun but subsequent navigation doesn't "GATHERING" anymore I swear there was a Error: Cannot call server functions outside of request error a moment ago, but I couldn't re-produce it anymore, so I guess it works fine ? Also, I have noticed that, on first load, "GATHERING" is always displayed twice on page load. Not sure why. Is it because it's supposed to be called twice: once in server to render the page, and once in client to re-validate the data ? Wonder if there's a way to not re-call it client-side
peerreynders
peerreynders•4w ago
"GATHERING" is always displayed twice on page load.
Right now the SolidStart server runs two distinct workers: - The server responsible for rendering Page and API responses. - A server for RPC responses for clients "use server" sections. "use server" sections run in both these workers, in the first for SSR and in the next to satisfy the RPC requests. Consequently the "GATHERING" module is loaded separately for each worker, each keeping their separate copy of MAP_DATA in the worker's memory space. If the server needs anything out of that module, loading of the module will force the setting of MAP_DATA (i.e. querying the DB). --- Looking at: 
async loadMap(country) {
  'use server'
  console.log("LOADING FOR", country)
  return MAP_DATA[country]
}
async loadMap(country) {
  'use server'
  console.log("LOADING FOR", country)
  return MAP_DATA[country]
}
Based on the contained "use server" section, loadMap is intended for client side access. loadMap is defined on the top level of the module, so from the bundler perspective this module needs to be part of the client side bundle. So again when the client loads this module in the browser MAP_DATA is assigned client side but this time gatherData connects to the RPC worker and runs the DB query all over again (as only loadData uses MAP_DATA). So your "server side in memory copy" isn't actually being used whenever a new client spins up. Then there is the issue whether you should be caching that info anyway because it's your database that should be acting as your data cache (if that's too slow use a caching solution that is collocated with but outside of the SolidStart server). 
// src/server.ts
// i.e. pure server side module
let MAP_DATA: Awaited<ReturnType<getData>> | undefined;

async function getMapData() {
  if (MAP_DATA) return MAP_DATA;

  console.log('GATHERING');
  const maps = await db.findAll('maps');
  return (MAP_DATA = arrayToObject(maps));
}

export { getMapData };
// src/server.ts
// i.e. pure server side module
let MAP_DATA: Awaited<ReturnType<getData>> | undefined;

async function getMapData() {
  if (MAP_DATA) return MAP_DATA;

  console.log('GATHERING');
  const maps = await db.findAll('maps');
  return (MAP_DATA = arrayToObject(maps));
}

export { getMapData };
// src/routes/map/[country].tsx
// i.e. also client module
import { createAsync, query } from '@solidjs/router';
// the RPC compiler will eliminate this import
import { getMapData } from '~/server.ts';

import type {
  RouteDefinition,
  RoutePreloadFuncArgs,
  RouteSectionProps,
} from '@solidjs/router';

const countryData = query((country: string) => {
  'use server';
  console.log('LOADING FOR', country);
  const data = await getMapData()[country];
  return data;
}, 'country-data');

export const route = {
  preload: (args: RoutePreloadFuncArgs) => {
    // warm the query
    countryData(args.params.country);
  },
} satisfies RouteDefinition;

export default function CountryPage(props: RouteSectionProps) {
  const countryMap = createAsync(() => countryData(props.params.country));
  return <p>{countryMap.name}</p>;
}
// src/routes/map/[country].tsx
// i.e. also client module
import { createAsync, query } from '@solidjs/router';
// the RPC compiler will eliminate this import
import { getMapData } from '~/server.ts';

import type {
  RouteDefinition,
  RoutePreloadFuncArgs,
  RouteSectionProps,
} from '@solidjs/router';

const countryData = query((country: string) => {
  'use server';
  console.log('LOADING FOR', country);
  const data = await getMapData()[country];
  return data;
}, 'country-data');

export const route = {
  preload: (args: RoutePreloadFuncArgs) => {
    // warm the query
    countryData(args.params.country);
  },
} satisfies RouteDefinition;

export default function CountryPage(props: RouteSectionProps) {
  const countryMap = createAsync(() => countryData(props.params.country));
  return <p>{countryMap.name}</p>;
}
This still doesn't address that both workers have to have their own map copy (SSR vs RPC) but at least they will reuse the DB results across clients.
oke
okeOP•4w ago
Thanks a lot @peerreynders for such a detailed response full of insights. I have some followup questions if you don't mind: 1. You kept mentioning "use server" sections as a module --> Does the bundler split each server function into a separate module/file ? 2. I understood "loadMap is intended for client side access", since a server function is intended to be used inside some kind of request/createAsync calls on the client. However, I don't understand "loadMap is defined on the top level of the module, so from the bundler perspective this module needs to be part of the client side bundle." --> I have 'use server' but loadMap makes it to the client bundle - aren't it supposed to be server-only? I vaguely remember the compilation result being $createServerReferrence(...) in some cases - is that not the case here ? 3. "When the client loads this module in the browser MAP_DATA is assigned client side" --> So MAP_DATA now lives 3 places - in memory of the two workers, and in browser memory ? 4. You separated the gatherData() function into a separate src/server.ts file and makes it into a pure server side module WITHOUT 'use server' --> Is this a special naming convention when the file name contains the word "server", or has the bundler always worked that way - i.e. stuff in routes folder makes it to client-bundle, and anything outside of routes folder is server-only ?
peerreynders
peerreynders•4w ago
Does the bundler split each server function into a separate module/file ?
No idea. However keep in mind that on the server worker it just runs as plain JS during SSR. Meanwhile in the RPC worker the code is always sandwiched between RPC argument deserialization and result serialization.
I have 'use server' but loadMap makes it to the client bundle - aren't it supposed to be server-only?
loadMap is converted to a RPC stub that is used client-side. It's only inside that stub that the arguments are serialized and sent to the RPC worker. Then the "use server" section is executed inside the worker on the server and the result is serialized and sent back to the client. During SSR the "use server" has no effect, i.e. loadMap is just a normal JS function that runs in the server. So loadMap will end up in both server and client bundles; in the server bundle as a plain JS function in the client bundle as an RPC stub. But from the client's perspective the module that contains the RPC stubs is just a regular client module.
So MAP_DATA now lives 3 places - in memory of the two workers, and in browser memory ?
Correct.
Is this a special naming convention
No. It's just a way of organizing things that makes it easy on the bundler to know where things belong. A more extreme version: 
// src/server.ts
// i.e. pure server side module
let MAP_DATA: Awaited<ReturnType<getData>> | undefined;

async function getMapData() {
  if (MAP_DATA) return MAP_DATA;

  console.log('GATHERING');
  const maps = await db.findAll('maps');
  return (MAP_DATA = arrayToObject(maps));
}

async function countryData(country: string) {
  console.log('LOADING FOR', country);
  const data = await getMapData()[country];
  return data;
}

export { countryData, getMapData };
// src/server.ts
// i.e. pure server side module
let MAP_DATA: Awaited<ReturnType<getData>> | undefined;

async function getMapData() {
  if (MAP_DATA) return MAP_DATA;

  console.log('GATHERING');
  const maps = await db.findAll('maps');
  return (MAP_DATA = arrayToObject(maps));
}

async function countryData(country: string) {
  console.log('LOADING FOR', country);
  const data = await getMapData()[country];
  return data;
}

export { countryData, getMapData };
// src/routes/map/[country].tsx
// i.e. also client module
import { createAsync, query } from '@solidjs/router';
// the RPC compiler will eliminate this import
import { countryData as serverCountryData } from '~/server.ts';

import type {
  RouteDefinition,
  RoutePreloadFuncArgs,
  RouteSectionProps,
} from '@solidjs/router';

const countryData = query((country: string) => {
  'use server';
  return serverCountryData(country);
}, 'country-data');

export const route = {
  preload: (args: RoutePreloadFuncArgs) => {
    // warm the query
    countryData(args.params.country);
  },
} satisfies RouteDefinition;

export default function CountryPage(props: RouteSectionProps) {
  const countryMap = createAsync(() => countryData(props.params.country));
  return <p>{countryMap.name}</p>;
}
// src/routes/map/[country].tsx
// i.e. also client module
import { createAsync, query } from '@solidjs/router';
// the RPC compiler will eliminate this import
import { countryData as serverCountryData } from '~/server.ts';

import type {
  RouteDefinition,
  RoutePreloadFuncArgs,
  RouteSectionProps,
} from '@solidjs/router';

const countryData = query((country: string) => {
  'use server';
  return serverCountryData(country);
}, 'country-data');

export const route = {
  preload: (args: RoutePreloadFuncArgs) => {
    // warm the query
    countryData(args.params.country);
  },
} satisfies RouteDefinition;

export default function CountryPage(props: RouteSectionProps) {
  const countryMap = createAsync(() => countryData(props.params.country));
  return <p>{countryMap.name}</p>;
}
After the RPC compiler replaces 
  'use server';
  return serverCountryData(country);
  'use server';
  return serverCountryData(country);
with the RPC call, tree shaking can remove this import 
import { countryData as serverCountryData } from '~/server.ts';
import { countryData as serverCountryData } from '~/server.ts';
and the client module no longer references the src/server.ts module.
oke
okeOP•4w ago
Oh very cool. So anything a use server function references will be assumed to be inside the server-bundle without needing to explicitly marked "use server", and thus will be tree-shaken away if it lives in a separate file (like src/server.ts lives in a separate file so when the RPC compiler generate the RPC stub, it knows that src/server.ts already will have existed on the server, so the client bundle won't need to import this function). What if the getMapData function lives in the same file as the route page? Would that mean that the getMapData function will also be turned into an RPC stub if it has 'use server', or be included as an actual function in the client bundle if it doesn't have 'use server' ?
So MAP_DATA now lives 3 places - in memory of the two workers, and in browser memory ?
But doesn't countryData gets turned into an RPC stub without the actual function content making it to the client bundle? If it gets turned into an RPC stub, how does MAP_DATA gets created in the browser memory, given that no actual client-side function uses MAP_DATA ? My apologies if I am being a bit dumb. Thanks a lot for your patience
peerreynders
peerreynders•4w ago
Would that mean that the getMapData function will also be turned into an RPC stub if it has 'use server'
Yes.
be included as an actual function in the client bundle if it doesn't have 'use server' ?
Also yes, as that is an entirely different case (i.e. the "or" is confusing here).
how does MAP_DATA gets created in the browser memory,
I was talking about this situation. 
// map/[country].tsx
async function gatherData() {
  'use server';
  console.log('GATHERING');
  const maps = await db.findAll('maps');
  const mapObject = arrayToObject(maps);
  return mapObject;
}

const MAP_DATA = await gatherData();

async function loadMap(country) {
  'use server';
  console.log('LOADING FOR', country);
  return MAP_DATA[country];
}
// map/[country].tsx
async function gatherData() {
  'use server';
  console.log('GATHERING');
  const maps = await db.findAll('maps');
  const mapObject = arrayToObject(maps);
  return mapObject;
}

const MAP_DATA = await gatherData();

async function loadMap(country) {
  'use server';
  console.log('LOADING FOR', country);
  return MAP_DATA[country];
}
As soon as the client loads the module 
const MAP_DATA = await gatherData();
const MAP_DATA = await gatherData();
is run in order to initialize the client side module global (perhaps you'll get lucky and tree shacking will remove it but tree shaking can be tricky) , meanwhile loadMap will use the MAP_DATA on the server worker during SSR and MAP_DATA on the RPC worker during a dynamic route load.
oke
okeOP•4w ago
Thanks. I understand it now. gatherData() is converted into an RPC stub. However, the line const MAP_DATA = await gatherData(); makes it into the client bundle --> const MAP_DATA = await callTheGatherDataRpc(); I think my biggest realization from today is that, in SolidStart, anything inside a route file is automatically included into the client bundle if not explicitly marked as use server. This is a small surprise to me because, coming from React's Remix.run, I have expected that only the component code is included inside the client bundle (or maybe I understood Remix.run wrong).

Did you find this page helpful?