SolidJS•4mo ago

How to create and test a data fetching library?

Hello everyone, I am working on a SolidJS integration for GQty. PR#1758 is the current WIP, I am probably doing a lot of things wrong. There are two areas about Suspense where I would really love some help to learn from the perspective of a library author. 1. Triggering Suspense Having studied the source code of createResource, I am able to trigger an initial Suspense via createSignal by returning a Promise in my function. But Suspense doesn't happen again when I return a Promise during refetches. 2. Testing Suspense It seems that render() does not Suspense at all, instead it renders the Promise object as-is which in turn throws. What would be the best place to start?

43 Replies

Brendonovich•4mo ago

For something like this, createQuery should basically be a wrapper around createResource. I'm not quite sure what you mean by doing Suspense with a signal but createResource is the only proper way to get something to suspend

vicaryOP•4mo ago

There was an attempt to wrap createResource, I don't quite remember what stopped me from doing that in my first attempt. I'll try this again and see how it goes.

Brendonovich•4mo ago

Taking a look at @tanstack/solid-query might be helpful

vicaryOP•4mo ago

That would be helpful, thanks! @Brendonovich Thanks again for the reference, we've made significant processes is made since then! We got a use case where a single fetch would update multiple components, but resolving these promises with batch does not do the job. (i.e. triggerEffect / createEffect needs one pass for each query.) Is there a way to resolve multiple pending promises in the fetcher of createResource at once?

Brendonovich•4mo ago

You're resolving them with the resolve function you pulled out of new Promise yeah?

vicaryOP•4mo ago

Yes, we are using p-defer.

Brendonovich•4mo ago

It depends if promise resolvers work synchronously or not, my guess is they don't If you've got multiple promises and a single createResource then you can just Promise.allSettled inside the fetcher to wait for all of them oh yeah they for sure wouldn't since the promise's then would run on the next microtask

vicaryOP•4mo ago

It is possible for us to maintain a fetch -> query mapping of all the active promises, such that correlated fetchers would return the same promise instance. I am quite confident that it would work. I am just curious if there is some easier ways to batch them up, we are able to restrict all of them to the very next microtask.

Brendonovich•4mo ago

What's the problem with how you've currently got it setup?

vicaryOP•4mo ago

In our test:

return testEffect(() => {
  const query1 = createQuery();
  const query2 = createQuery();
  const query3 = createQuery();

  createEffect(() => {
    query1().data;
    query2().data;
    query3().data;

    // the rest of the test
  });
});

return testEffect(() => {
  const query1 = createQuery();
  const query2 = createQuery();
  const query3 = createQuery();

  createEffect(() => {
    query1().data;
    query2().data;
    query3().data;

    // the rest of the test
  });
});

We see 4 updates as follows, while we want 2. 1. all starts loading 2. query1 resolves 3. query2 resolves 4. query3 resolves It's kind of a final stretch for DX and testing though, users shouldn't notice the perf difference.

Brendonovich•4mo ago

with that setup i don't think you could, shouldn't createQuery be outside the effect?

vicaryOP•4mo ago

Sorry my bad, updating the code snippet.

Brendonovich•4mo ago

unless they all listened to the same createResource or promise maybe?

vicaryOP•4mo ago

In fact render 1 is achieved by using batch() with all the refetch() calls across multiple instance of createQuery/createResource.

Brendonovich•4mo ago

if they were the same resource i think it'd work, but as separate resources even with the same promise i'm not sure

vicaryOP•4mo ago

Oh, got it. We'll use render and waitFor instead for now. But is that something you'd like to make happen in the future?

Brendonovich•4mo ago

with separate promises i don't think you could, with the same promise across multiple resources solid would probably have to keep track of all promises globally to resolve their resources at the same time. idk if that's something that would be considered keep in mind that if all those queries are consumed during rendering as well then the createEffect won't run until the suspense boundary above them has resolved, so by that time all the queries could have data and then effect could only run once

vicaryOP•4mo ago

Then we won't bother maintaining our own global fetch-promise mapping, Suspense is the way to go! Thanks!

Brendonovich•4mo ago

as long as your queries operater similarly to tanstack query i'd say you're fine. does gqty have a way of preloading queries?

vicaryOP•4mo ago

With the prepare option, the fetch can happen onMount. We can make it happen during createQuery if thats what you recommend.

Brendonovich•4mo ago

with solid router the best way of preloading is to use preload, which runs before any components have loaded with the goal of avoiding data waterfalls not sure if gqty would be able to do that since it requires running components to discover what data they require

vicaryOP•4mo ago

Runtime selection is the default for development phase, we encourage users to add prepare in production. We'll try to integrate this option into preload. Thanks for the heads up! A quick search in tanstack shows <A preload={true} />. Just to make sure I understand, are we talking about that, or is it about the load attribute in <Route />?

Brendonovich•4mo ago

load on <Route /> yeah, now renamed to preload ah yeah if you can start fetching without running the component with prepare you should be good

vicaryOP•4mo ago

start fetching without running the component with prepare

We are doing this inside onMount, the fetch starts before any JSX. Do we need to further hoist it up to run synchronously during createQuery?

Brendonovich•4mo ago

Ideally the fetch would start without running the component at all, and seeing as you can use fragements in getStaticProps i think you could do a similar thing with preload functions

Brendonovich•4mo ago

The idea being that preload functions for all route segments can run in parallel, before even the route components have lazy loaded in

vicaryOP•4mo ago

Sounds like our RSC approach, users would have to use resolve in GQty core for preloading, i.e. users should call resolve() in your loadUser example for <Route preload={loadUser} />. Feels like not really possible with an option createQuery in @gqty/solid alone, am I correct?

Brendonovich•4mo ago

Pretty much, since createQuery is for components As long as resolve can populate the same cache that createQuery pulls from it should work

vicaryOP•4mo ago

Yes It shares the same cache. So it is more likely a guide section in our website for solid integration.

Brendonovich•4mo ago

With tanstack query, we call ensureQueryData in preload and then createQuery reuses that fetch to show the data sooner Yeah if the functionality is already there you probably just need a code example

vicaryOP•4mo ago

Sounds good! Hi @Brendonovich! We are hitting an interesting situation when we're doing some e2e tests. When we update a filter on the data-table during an active fetch (i.e. Suspense fallback is visible), the UI seems to be locked in the Suspense fallback. We are suspecting a call to refetch before resolving a pending promise in createResource triggers this issue. I am curious if there is a way to "cancel" or "break free" from a stale pending promise, such that we don't have to queue up these UI state changes after the pending promise.

Brendonovich•4mo ago

hmm if you refetch i think the suspense will switch to the new promise and forget about the old one https://playground.solidjs.com/anonymous/75cea0c9-df32-4e48-be1b-ed08e5c62519

Solid Playground

Quickly discover what the solid compiler will generate from your JSX template

Brendonovich•4mo ago

you can refetch a bunch of times in that playground but only the most recent resolve is needed to un-suspend

vicaryOP•3mo ago

Thanks for confirming the behavior! But I am afraid we may need some help in pinning down the root cause of leaking suspenses. In the screen recording you can see there is a Suspense fallback at the first fetch, and the glimmer at the second fetch means the Suspense failed to show up. 1. This is the location of <Suspense /> 2. And this is the createResource() where we returns the fetch promise What would be the best place to start looking?

Brendonovich•3mo ago

It's because query prefers resource.latest

Brendonovich•3mo ago

resource.latest only suspends when the resource is doing its first fetch, it returns the synchronous value of the resource otherwise

Brendonovich•3mo ago

Using the resource accessor makes it work properly

Brendonovich•3mo ago

Also for your ui components I'd recommend against destructuring props. Solid uses getters and proxies in props so destructing, which eagerly evaluates getters, can break reactivity

vicaryOP•3mo ago

Thank you, it's working properly now! The .latest was an attempt to solve a rare case when the UI was locked in a Suspense state, we might have solved it when we refactor the core. We know that the exact location of function calls is where it triggers reactivity, for example in <Characters /> we do this:

<For each={query()....}>

<For each={query()....}>

Do you see a destructuring that we've missed?

Brendonovich•3mo ago

Ah i specifically mean destructing in props like this

vicaryOP•3mo ago

No worries because Suspense removes the needs for a Skeleton compoent with fallbacks, we may simply remove it from the example.

Brendonovich•3mo ago

You'll need to remove destructuring in the other components too but apart from that the example looks good

vicaryOP•3mo ago

We'll update in the next patch, thanks!

Gaming

Programming

How to create and test a data fetching library?