Michael Schaufelberger Posts

Michael Schaufelberger

Explore posts from servers

TtRPC TTCTheo's Typesafe Cult XXata DTDrizzle Team

•Created by Michael Schaufelberger on 11/4/2024 in #help

FetcherError: Internal Error

Hi 👋 When trying to write to our non-postgres DB, we are currently getting the cryptic error:

FetcherError: Internal Error (Request ID: f347ce16-76ee-945f-81df-51cf8d1f6e59)

{
  status: 500,
  errors: [
    {
      message: 'Internal Error (Request ID: f347ce16-76ee-945f-81df-51cf8d1f6e59)',
      status: 500
    }
  ],
  requestId: 'f347ce16-76ee-945f-81df-51cf8d1f6e59',
  cause: undefined
}

FetcherError: Internal Error (Request ID: f347ce16-76ee-945f-81df-51cf8d1f6e59)

{
  status: 500,
  errors: [
    {
      message: 'Internal Error (Request ID: f347ce16-76ee-945f-81df-51cf8d1f6e59)',
      status: 500
    }
  ],
  requestId: 'f347ce16-76ee-945f-81df-51cf8d1f6e59',
  cause: undefined
}

How can we identify the cause? We currently cannot write to the database anymore (Edit: for some requests, some others still seem to work).

20 replies

XXata

•Created by Michael Schaufelberger on 9/3/2024 in #help

What is good practice with Xata's (Multi-version Schema) Migrations?

For now, we have used mainly Drizzle Kit for our migration needs. However, we are at a point where we would like to use Xata's Multi-Version schema feature. What's a good workflow that allows for good DX and safe migrations? With "normal" Xata migrations, every single schema change is a migration and results in its own file. This is suboptimal DX since during development you may have to add a field, rename it and delete it again until you have found the best schema for a feature and have many files in the end that are difficult to review. Do you have any experience/recommendations on how we could handle the experimentation vs. migration phase? Multi-version Schemas have a lifecycle. Is this something we can use to solve this problem? I need to be honest that I don't know how other tools handle it.

4 replies

XXata

•Created by Michael Schaufelberger on 8/30/2024 in #help

When are connections to the cluster available during the moving of a branch?

For several minutes the branches are moving to a newly provisioned cluster. When are new connections to the branch made to the cluster?

50 replies

DTDrizzle Team

•Created by Michael Schaufelberger on 8/29/2024 in #help

How can I create a `nulls not distinct` index in Postgres 15 for `push`?

Hi 👋 I'm having trouble to create such an index. This

  (table) => ({
    uniqueIdx: unique("mytable_uniq_idx_composite").on(table.foo, table.bar).nullsNotDistinct(),
  }),

  (table) => ({
    uniqueIdx: unique("mytable_uniq_idx_composite").on(table.foo, table.bar).nullsNotDistinct(),
  }),

somehow does not create an index that has the nulls not distinct constraint. Whereas a query like this would work.

(table) => ({
    uniqueIdx: uniqueIndex('mytable_uniq_idx_composite')
      .on(table.foo, table.bar)
      .where(sql`NULLS NOT DISTINCT`),
  }),

(table) => ({
    uniqueIdx: uniqueIndex('mytable_uniq_idx_composite')
      .on(table.foo, table.bar)
      .where(sql`NULLS NOT DISTINCT`),
  }),

However, there's a WHERE clause that breaks it.

CREATE UNIQUE INDEX IF NOT EXISTS "mytable_uniq_idx_composite" ON "mytable" USING btree ("foo","bar") WHERE NULLS NOT DISTINCT;

CREATE UNIQUE INDEX IF NOT EXISTS "mytable_uniq_idx_composite" ON "mytable" USING btree ("foo","bar") WHERE NULLS NOT DISTINCT;

2 replies

TtRPC

•Created by Michael Schaufelberger on 7/25/2024 in #❓-help

Is there a way to call procedures more directly without a router?

I read the blog post about server actions and wondered if there's a way to use a query or mutation more directly. Does it even make sense to use a procedure more directly? I was just wondering how to more closely colocate a procedures usage near a component. About 80% of the time only a single component will need a mutation (e.g. specific form or a certain data table with component specific data).

2 replies

XXata

•Created by Michael Schaufelberger on 7/17/2024 in #help

Issues connecting to the database

Hi We are currently experiencing issues when connecting to the Postgres database on eu-central. Mainly, schema changes are not working. But we also had disconnects when reading data or rather when trying to connect to read data.

XATA_BRANCH dev
Using 'pg' driver for database querying
[✓] Pulling schema from database...

 Warning  You are about to execute current statements:

DROP INDEX IF EXISTS "fields_unique_idx";
DROP INDEX IF EXISTS "tags_unique_idx";
CREATE UNIQUE INDEX IF NOT EXISTS "fields_unique_idx" ON "fields" USING btree (LOWER(label),"eventSeriesId");
CREATE UNIQUE INDEX IF NOT EXISTS "tags_unique_idx" ON "tags" USING btree (LOWER(label),"owner","eventSeriesId");

Error: Connection terminated unexpectedly

XATA_BRANCH dev
Using 'pg' driver for database querying
[✓] Pulling schema from database...

 Warning  You are about to execute current statements:

DROP INDEX IF EXISTS "fields_unique_idx";
DROP INDEX IF EXISTS "tags_unique_idx";
CREATE UNIQUE INDEX IF NOT EXISTS "fields_unique_idx" ON "fields" USING btree (LOWER(label),"eventSeriesId");
CREATE UNIQUE INDEX IF NOT EXISTS "tags_unique_idx" ON "tags" USING btree (LOWER(label),"owner","eventSeriesId");

Error: Connection terminated unexpectedly

Is there currently something unstable? Username: 5713tr

27 replies

XXata

•Created by Michael Schaufelberger on 7/13/2024 in #help

Vercel connections timeout when using a transaction

Hi We are using Drizzle together with the TCP Pool pg client.

import { drizzle as drizzlePg } from 'drizzle-orm/node-postgres';
import { Pool } from 'pg';

import * as schema from './schema';
import { getXataClient } from './xata';

export const xata = getXataClient();

const pool = new Pool({
  connectionString: xata.sql.connectionString,
  max: 1,
});

export const db = drizzlePg(pool, {
  schema,
});

import { drizzle as drizzlePg } from 'drizzle-orm/node-postgres';
import { Pool } from 'pg';

import * as schema from './schema';
import { getXataClient } from './xata';

export const xata = getXataClient();

const pool = new Pool({
  connectionString: xata.sql.connectionString,
  max: 1,
});

export const db = drizzlePg(pool, {
  schema,
});

I have also tried

const pool = new Pool({
  connectionString: xata.sql.connectionString,
  max: 1,
  maxUses: 1,
  connectionTimeoutMillis: 5000,
  idleTimeoutMillis: 1000,
  allowExitOnIdle: true,
});

const pool = new Pool({
  connectionString: xata.sql.connectionString,
  max: 1,
  maxUses: 1,
  connectionTimeoutMillis: 5000,
  idleTimeoutMillis: 1000,
  allowExitOnIdle: true,
});

which just makes at least the serverless function not run until its own timeout is reached. However, the serverless function times out when we want to use a transaction. This only happens when deployed to Vercel. I tried to check how many connections we have using pgAdmin. But there are none, which I think we are just not allowed to see, because my current one should show up. How can I use transactions on Vercel using Drizzle? This is currently a major blocker. I read some threads here of a 40 connection limit. When reading stuff like this https://neon.tech/docs/connect/connection-pooling#connection-pooling it makes me question if Xata can even handle TCP in serverless functions. Would a dedicated Cluster solve this issue?

7 replies

DTDrizzle Team

•Created by Michael Schaufelberger on 7/10/2024 in #help

RQB: Why is a where clause inside a many-to-many relation not allowed?

Say we have posts and tags. Every post can have multiple tags, but a tag can have many posts. So a classical many-to-many relation. Note: we have a junction table to store the relation posts 1-n postsToTags n-1 tag Now we want to fetch a post with all the tags. We can just do this:

db.query.postsTbl.findMany({
  where: (postsTbl, { eq }) => {
    return eq(postsTbl.owner, authId)
  },
  with: {
    postsToTags: {
      with: {
        tag: {
          columns: {
            id: true,
            label: true,
          },
        },
      },
    },
  },
});

db.query.postsTbl.findMany({
  where: (postsTbl, { eq }) => {
    return eq(postsTbl.owner, authId)
  },
  with: {
    postsToTags: {
      with: {
        tag: {
          columns: {
            id: true,
            label: true,
          },
        },
      },
    },
  },
});

But say we only want the tags that contain "hello" in the label:

db.query.postsTbl.findMany({
  where: (postsTbl, { eq }) => {
    return eq(postsTbl.owner, authId);
  },
  with: {
    postsToTags: {
      with: {
        tag: {
          // @ts-ignore
          where: (_, { like }) => like(tagsTbl.label, "%hello%"),
          columns: {
            id: true,
            label: true,
          },
        },
      },
    },
  },
});

db.query.postsTbl.findMany({
  where: (postsTbl, { eq }) => {
    return eq(postsTbl.owner, authId);
  },
  with: {
    postsToTags: {
      with: {
        tag: {
          // @ts-ignore
          where: (_, { like }) => like(tagsTbl.label, "%hello%"),
          columns: {
            id: true,
            label: true,
          },
        },
      },
    },
  },
});

Why is it not allowed in the types? Is there something that could easily break if we add a where clause to the subquery?

1 replies

XXata

•Created by Michael Schaufelberger on 7/5/2024 in #help

How can I restrict the API access to a specific branch?

Is this even possible? I can't find anything in the docs.

8 replies

TtRPC

•Created by Michael Schaufelberger on 6/21/2024 in #❓-help

How do I use the rsc-rq-prefetch example with a protected procedure?

Thank you for this implementation! It looks sooo promising 😀 However, I ran into an issue when trying it out: https://github.com/trpc/trpc/blob/next/examples/.experimental/next-app-dir/src/app/rsc-rq-prefetch/page.tsx If we have a protected procedure instead of a public one, the server is not authenticated during SSR and an error is thrown. How can we solve this? Can we just ignore the error and will it still work?

24 replies

TtRPC

•Created by Michael Schaufelberger on 6/12/2024 in #❓-help

Can I use the "Streaming with Server Components" strategy with tRPC?

https://tanstack.com/query/latest/docs/framework/react/guides/advanced-ssr#streaming-with-server-components has a neat example on how to prefetch queries without having to await them. Is this something we can use with tRPC?

68 replies

XXata

•Created by Michael Schaufelberger on 6/11/2024 in #help

`xata pull <branch>` results in a high number of migration files - Safe to ignore?

This happens after running migrations or pushes with Drizzle. I have been testing different data models and now have over 200 migration files. Is it safe to ignore the folder since we are planning on running all migrations with Drizzle anyways?

4 replies

XXata

•Created by Michael Schaufelberger on 6/10/2024 in #help

Re-executing Drizzle Kit push fails after successful push

I'm getting the following error: PL/pgSQL function "xata.pg_catalog".regnamespace(text) line 14 at RAISE

> pnpx drizzle-kit push
drizzle-kit: v0.21.4
drizzle-orm: v0.30.10

No config path provided, using default path
Reading config file '/redacted/apps/partner-tool/app/drizzle.config.ts'
Using 'pg' driver for database querying
[⣟] Pulling schema from database.../redacted/node_modules/.pnpm/drizzle-kit@0.21.4_@esbuild-kit+esm-loader@2.6.5_commander@10.0.1_drizzle-orm@0.30.10_@neonda_jne73tzufwvw4dok65xkag4jqi/node_modules/drizzle-kit/bin.cjs:69429
        Error.captureStackTrace(err2);
              ^

error: schema "publics" does not exist
    at /redacted/node_modules/.pnpm/drizzle-kit@0.21.4_@esbuild-kit+esm-loader@2.6.5_commander@10.0.1_drizzle-orm@0.30.10_@neonda_jne73tzufwvw4dok65xkag4jqi/node_modules/drizzle-kit/bin.cjs:69429:15
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async Object.query (/redacted/node_modules/.pnpm/drizzle-kit@0.21.4_@esbuild-kit+esm-loader@2.6.5_commander@10.0.1_drizzle-orm@0.30.10_@neonda_jne73tzufwvw4dok65xkag4jqi/node_modules/drizzle-kit/bin.cjs:111366:26)
    at async /redacted/node_modules/.pnpm/drizzle-kit@0.21.4_@esbuild-kit+esm-loader@2.6.5_commander@10.0.1_drizzle-orm@0.30.10_@neonda_jne73tzufwvw4dok65xkag4jqi/node_modules/drizzle-kit/bin.cjs:15979:46 {
  length: 166,
  severity: 'ERROR',
  code: 'P0001',
  detail: undefined,
  hint: undefined,
  position: undefined,
  internalPosition: undefined,
  internalQuery: undefined,
  where: 'PL/pgSQL function "xata.pg_catalog".regnamespace(text) line 14 at RAISE',
  schema: undefined,
  table: undefined,
  column: undefined,
  dataType: undefined,
  constraint: undefined,
  file: 'pl_exec.c',
  line: '3923',
  routine: 'exec_stmt_raise'
}

> pnpx drizzle-kit push
drizzle-kit: v0.21.4
drizzle-orm: v0.30.10

No config path provided, using default path
Reading config file '/redacted/apps/partner-tool/app/drizzle.config.ts'
Using 'pg' driver for database querying
[⣟] Pulling schema from database.../redacted/node_modules/.pnpm/drizzle-kit@0.21.4_@esbuild-kit+esm-loader@2.6.5_commander@10.0.1_drizzle-orm@0.30.10_@neonda_jne73tzufwvw4dok65xkag4jqi/node_modules/drizzle-kit/bin.cjs:69429
        Error.captureStackTrace(err2);
              ^

error: schema "publics" does not exist
    at /redacted/node_modules/.pnpm/drizzle-kit@0.21.4_@esbuild-kit+esm-loader@2.6.5_commander@10.0.1_drizzle-orm@0.30.10_@neonda_jne73tzufwvw4dok65xkag4jqi/node_modules/drizzle-kit/bin.cjs:69429:15
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async Object.query (/redacted/node_modules/.pnpm/drizzle-kit@0.21.4_@esbuild-kit+esm-loader@2.6.5_commander@10.0.1_drizzle-orm@0.30.10_@neonda_jne73tzufwvw4dok65xkag4jqi/node_modules/drizzle-kit/bin.cjs:111366:26)
    at async /redacted/node_modules/.pnpm/drizzle-kit@0.21.4_@esbuild-kit+esm-loader@2.6.5_commander@10.0.1_drizzle-orm@0.30.10_@neonda_jne73tzufwvw4dok65xkag4jqi/node_modules/drizzle-kit/bin.cjs:15979:46 {
  length: 166,
  severity: 'ERROR',
  code: 'P0001',
  detail: undefined,
  hint: undefined,
  position: undefined,
  internalPosition: undefined,
  internalQuery: undefined,
  where: 'PL/pgSQL function "xata.pg_catalog".regnamespace(text) line 14 at RAISE',
  schema: undefined,
  table: undefined,
  column: undefined,
  dataType: undefined,
  constraint: undefined,
  file: 'pl_exec.c',
  line: '3923',
  routine: 'exec_stmt_raise'
}

Is this an error with Xata or Drizzle? It loks like Xata. Also, I'm not referencing publics anywhere. Which seems like a typo?

15 replies

XXata

•Created by Michael Schaufelberger on 6/7/2024 in #help

Can we use ULIDs without a dedicated cluster?

We are looking for some solution between a Postgres serial and a long UUID. So ULIDs are a natural choice. I would use something like https://github.com/geckoboard/pgulid or https://github.com/pksunkara/pgx_ulid. But without a dedicated cluster we cannot create types/functions. So is there another way to use ULID-like IDs? I've seen that select ('rec_'::text || (xata_private.xid())::text) produces a similar result. Can we use this for the foreseeable future for our own ID fields?

5 replies

TtRPC

•Created by Michael Schaufelberger on 5/28/2024 in #❓-help

How to access the request body in the onError callback?

Hi 👋 I'm having trouble getting the request's body (probably plain text) when the following error occurs in the fetchRequestHandler:

TRPCError: "input" needs to be an object when doing a batch call

TRPCError: "input" needs to be an object when doing a batch call

I've tried to get it with req.text() but as it is already consumed, I cannot access it. The code in question:

fetchRequestHandler({
    endpoint: '/api/trpc',
    req,
    router: appRouter,
    createContext: () => ({}),
    onError: async ({ error, req, input }) => {
      console.log('TRPC route error', error);

      let body = null;

      try {
        if (req.method === 'POST') {
          const isJsonContentType =
            !!req.headers.get('Content-Type')?.includes('application/json') ??
            false;
  
          body = isJsonContentType ? await req.json() : await req.text();
        }
      } catch () {
        // here it would throw that the body is unusable
      }
       
      console.log(
        req.method,
        req.headers.get('Content-Type'),
        { body, input },
       // here: both, body and input are empty 
      );

      // ...
    }
}

fetchRequestHandler({
    endpoint: '/api/trpc',
    req,
    router: appRouter,
    createContext: () => ({}),
    onError: async ({ error, req, input }) => {
      console.log('TRPC route error', error);

      let body = null;

      try {
        if (req.method === 'POST') {
          const isJsonContentType =
            !!req.headers.get('Content-Type')?.includes('application/json') ??
            false;
  
          body = isJsonContentType ? await req.json() : await req.text();
        }
      } catch () {
        // here it would throw that the body is unusable
      }
       
      console.log(
        req.method,
        req.headers.get('Content-Type'),
        { body, input },
       // here: both, body and input are empty 
      );

      // ...
    }
}

I would very much like to know what was sent in order to debug - in any case. The body has to be forwarded to the callback by tRPC, correct? Or is there another way?

2 replies

XXata

•Created by Michael Schaufelberger on 5/27/2024 in #help

Migrations with Drizzle

Hi What is the current best practice regarding migrations when I'm using Drizzle with Xata? With Planetscale I just used Drizzle Kit's db push on db feature branches because the actual migration to prod was handled through the Planetscale dashboard.

35 replies

TTCTheo's Typesafe Cult

•Created by Michael Schaufelberger on 8/9/2023 in #questions

"Body is unusable" or "fetch failed" when using server to upload Zip files

Hi I'm getting errors like

- error Error: [object Promise]
    at getProperError (/path-to-project/node_modules/next/dist/lib/is-error.js:41:12)
    at DevServer.run (/path-to-project/node_modules/next/dist/server/dev/next-dev-server.js:924:53)
    at processTicksAndRejections (node:internal/process/task_queues:96:5)
    at async DevServer.handleRequestImpl (/path-to-project/node_modules/next/dist/server/base-server.js:533:20) {
  digest: undefined
}
- error unhandledRejection: Error [TypeError]: Body is unusable
    at specConsumeBody (/path-to-project/node_modules/next/dist/compiled/undici/index.js:2:4358)
    at Response.json (/path-to-project/node_modules/next/dist/compiled/undici/index.js:2:2573)
    at Function.fromResponse (file:///path-to-project/node_modules/@uploadthing/shared/dist/index.mjs:198:33)
    at uploadFilesInternal (file:///path-to-project/node_modules/uploadthing/dist/server.mjs:53:28)
    at processTicksAndRejections (node:internal/process/task_queues:96:5)
    at async Object.uploadFiles (file:///path-to-project/node_modules/uploadthing/dist/server.mjs:126:19)

- error Error: [object Promise]
    at getProperError (/path-to-project/node_modules/next/dist/lib/is-error.js:41:12)
    at DevServer.run (/path-to-project/node_modules/next/dist/server/dev/next-dev-server.js:924:53)
    at processTicksAndRejections (node:internal/process/task_queues:96:5)
    at async DevServer.handleRequestImpl (/path-to-project/node_modules/next/dist/server/base-server.js:533:20) {
  digest: undefined
}
- error unhandledRejection: Error [TypeError]: Body is unusable
    at specConsumeBody (/path-to-project/node_modules/next/dist/compiled/undici/index.js:2:4358)
    at Response.json (/path-to-project/node_modules/next/dist/compiled/undici/index.js:2:2573)
    at Function.fromResponse (file:///path-to-project/node_modules/@uploadthing/shared/dist/index.mjs:198:33)
    at uploadFilesInternal (file:///path-to-project/node_modules/uploadthing/dist/server.mjs:53:28)
    at processTicksAndRejections (node:internal/process/task_queues:96:5)
    at async Object.uploadFiles (file:///path-to-project/node_modules/uploadthing/dist/server.mjs:126:19)

if I don't set the (readonly) property name on myBlob or

{
  data: null,
  error: { code: undefined, message: 'fetch failed', data: undefined }
}

{
  data: null,
  error: { code: undefined, message: 'fetch failed', data: undefined }
}

when trying to upload files using utapi.uploadFiles(myBlob). The file is a ZIP file and I can perfectly serve it later in the code using

res.setHeader("Content-Type", "application/zip");
res.setHeader("Content-Disposition", "attachment; filename=foo.zip");
return res.status(200).send(Buffer.from(await myBlob.arrayBuffer()));

res.setHeader("Content-Type", "application/zip");
res.setHeader("Content-Disposition", "attachment; filename=foo.zip");
return res.status(200).send(Buffer.from(await myBlob.arrayBuffer()));

What am I doing wrong?

5 replies

TTCTheo's Typesafe Cult

•Created by Michael Schaufelberger on 8/1/2023 in #questions

How do you handle global state with RSCs?

Hi Im having a hard time to understand how you can leverage RSCs for data fetching if the data you want to fetch is dependent on user input. Case Example For example. I have rooms in a building you can book. But some rooms can only be accessed by being a manager. Now, a user can see if a room is booked and who has booked it (booking status field). But for rooms which can only be accessed by a manager this info us hidden. Except if the user is a manager and is in the "manager" mode (global state). Question How do you handle such a case where you want to show the information of a room based on the user's role and mode? Imagine you need this booking status info in the overview list and then in the whole tree when you have chosen a room to book (and proceed to the next steps). Also, switching manager mode should be instant. Pre RSCs, I would fetch the whole list of rooms with all the booking state info the user is allowed to see and store it in a context. Then I would render the big component tree, making use of useContext. If a manager turns on the manager mode, I could just switch to show the previously hidden data, without having to refetch. Is there a way to leverage the power of RSCs for that? Does it even make sense to use RSCs? I enjoy querying my DB directly...

2 replies

TtRPC

•Created by Michael Schaufelberger on 7/21/2023 in #❓-help

Example for tRPC, stable version, with RSC

Hi This may be a stupid question... The readme of the experimental example mentions

Note You can already use tRPC with app directory, by: using @trpc/client directly in components (both RSC and non-RSC) use @trpc/next for client components

What do you mean by this? Do you have an example for that using the current stable version of tRPC? Especially the part for the RSC components. I currently wouldn't mind to use tRPC in a non-optimal RSC way. But I would like to use Next's app directory to colocate files (and not having to have code in a completely different pages directory).

6 replies

TtRPC

•Created by Michael Schaufelberger on 6/13/2023 in #❓-help

What's the benefit of using the context instead of a direct import for the database connection?

I've wondered why I should use the tRPC context instead of just importing my database singleton from a module. Is there any benefit to it? Because of TS performance issues, I'm creating multiple functions for bigger procedures. And passing the db connection around may be an unnecessary overhead.

12 replies

Gaming

Programming