Michael Schaufelberger Posts

Michael Schaufelberger

Explore posts from servers

TtRPC TTCTheo's Typesafe Cult XXata DTDrizzle Team

•Created by Michael Schaufelberger on 11/19/2024 in #help

How can I get data from a different table when adding a column? - Multi Schema Migrations

Hi 👋 Say I want to store a user's full name inside a post. For this, I would add a column to the posts table userFullName using pgroll:

[
  {
    "add_column": {
      "table": "posts",
      "column": {
        "name": "userFullName",
        "type": "text",
        "nullable": false
      },
      "up": "select \"fullName\" from users where users.id = ???"
    }
  }
]

[
  {
    "add_column": {
      "table": "posts",
      "column": {
        "name": "userFullName",
        "type": "text",
        "nullable": false
      },
      "up": "select \"fullName\" from users where users.id = ???"
    }
  }
]

How can I make sure pgroll fills the new column with the data of the user? Can I do something like this?

select "fullName" from users where users.id = "userId"
-- meaning, match the user with the post's foreign key userId

select "fullName" from users where users.id = "userId"
-- meaning, match the user with the post's foreign key userId

Edit: I'm getting an error if I try this: unable to execute start operation: failed to create trigger: pq: syntax error at or near "SELECT"

8 replies

XXata

•Created by Michael Schaufelberger on 11/4/2024 in #help

FetcherError: Internal Error

Hi 👋 When trying to write to our non-postgres DB, we are currently getting the cryptic error:

FetcherError: Internal Error (Request ID: f347ce16-76ee-945f-81df-51cf8d1f6e59)

{
  status: 500,
  errors: [
    {
      message: 'Internal Error (Request ID: f347ce16-76ee-945f-81df-51cf8d1f6e59)',
      status: 500
    }
  ],
  requestId: 'f347ce16-76ee-945f-81df-51cf8d1f6e59',
  cause: undefined
}

FetcherError: Internal Error (Request ID: f347ce16-76ee-945f-81df-51cf8d1f6e59)

{
  status: 500,
  errors: [
    {
      message: 'Internal Error (Request ID: f347ce16-76ee-945f-81df-51cf8d1f6e59)',
      status: 500
    }
  ],
  requestId: 'f347ce16-76ee-945f-81df-51cf8d1f6e59',
  cause: undefined
}

How can we identify the cause? We currently cannot write to the database anymore (Edit: for some requests, some others still seem to work).

20 replies

XXata

•Created by Michael Schaufelberger on 9/3/2024 in #help

What is good practice with Xata's (Multi-version Schema) Migrations?

For now, we have used mainly Drizzle Kit for our migration needs. However, we are at a point where we would like to use Xata's Multi-Version schema feature. What's a good workflow that allows for good DX and safe migrations? With "normal" Xata migrations, every single schema change is a migration and results in its own file. This is suboptimal DX since during development you may have to add a field, rename it and delete it again until you have found the best schema for a feature and have many files in the end that are difficult to review. Do you have any experience/recommendations on how we could handle the experimentation vs. migration phase? Multi-version Schemas have a lifecycle. Is this something we can use to solve this problem? I need to be honest that I don't know how other tools handle it.

4 replies

XXata

•Created by Michael Schaufelberger on 8/30/2024 in #help

When are connections to the cluster available during the moving of a branch?

For several minutes the branches are moving to a newly provisioned cluster. When are new connections to the branch made to the cluster?

50 replies

XXata

•Created by Michael Schaufelberger on 7/17/2024 in #help

Issues connecting to the database

Hi We are currently experiencing issues when connecting to the Postgres database on eu-central. Mainly, schema changes are not working. But we also had disconnects when reading data or rather when trying to connect to read data.

XATA_BRANCH dev
Using 'pg' driver for database querying
[✓] Pulling schema from database...

 Warning  You are about to execute current statements:

DROP INDEX IF EXISTS "fields_unique_idx";
DROP INDEX IF EXISTS "tags_unique_idx";
CREATE UNIQUE INDEX IF NOT EXISTS "fields_unique_idx" ON "fields" USING btree (LOWER(label),"eventSeriesId");
CREATE UNIQUE INDEX IF NOT EXISTS "tags_unique_idx" ON "tags" USING btree (LOWER(label),"owner","eventSeriesId");

Error: Connection terminated unexpectedly

XATA_BRANCH dev
Using 'pg' driver for database querying
[✓] Pulling schema from database...

 Warning  You are about to execute current statements:

DROP INDEX IF EXISTS "fields_unique_idx";
DROP INDEX IF EXISTS "tags_unique_idx";
CREATE UNIQUE INDEX IF NOT EXISTS "fields_unique_idx" ON "fields" USING btree (LOWER(label),"eventSeriesId");
CREATE UNIQUE INDEX IF NOT EXISTS "tags_unique_idx" ON "tags" USING btree (LOWER(label),"owner","eventSeriesId");

Error: Connection terminated unexpectedly

Is there currently something unstable? Username: 5713tr

27 replies

XXata

•Created by Michael Schaufelberger on 7/13/2024 in #help

Vercel connections timeout when using a transaction

Hi We are using Drizzle together with the TCP Pool pg client.

import { drizzle as drizzlePg } from 'drizzle-orm/node-postgres';
import { Pool } from 'pg';

import * as schema from './schema';
import { getXataClient } from './xata';

export const xata = getXataClient();

const pool = new Pool({
  connectionString: xata.sql.connectionString,
  max: 1,
});

export const db = drizzlePg(pool, {
  schema,
});

import { drizzle as drizzlePg } from 'drizzle-orm/node-postgres';
import { Pool } from 'pg';

import * as schema from './schema';
import { getXataClient } from './xata';

export const xata = getXataClient();

const pool = new Pool({
  connectionString: xata.sql.connectionString,
  max: 1,
});

export const db = drizzlePg(pool, {
  schema,
});

I have also tried

const pool = new Pool({
  connectionString: xata.sql.connectionString,
  max: 1,
  maxUses: 1,
  connectionTimeoutMillis: 5000,
  idleTimeoutMillis: 1000,
  allowExitOnIdle: true,
});

const pool = new Pool({
  connectionString: xata.sql.connectionString,
  max: 1,
  maxUses: 1,
  connectionTimeoutMillis: 5000,
  idleTimeoutMillis: 1000,
  allowExitOnIdle: true,
});

which just makes at least the serverless function not run until its own timeout is reached. However, the serverless function times out when we want to use a transaction. This only happens when deployed to Vercel. I tried to check how many connections we have using pgAdmin. But there are none, which I think we are just not allowed to see, because my current one should show up. How can I use transactions on Vercel using Drizzle? This is currently a major blocker. I read some threads here of a 40 connection limit. When reading stuff like this https://neon.tech/docs/connect/connection-pooling#connection-pooling it makes me question if Xata can even handle TCP in serverless functions. Would a dedicated Cluster solve this issue?

7 replies

XXata

•Created by Michael Schaufelberger on 7/5/2024 in #help

How can I restrict the API access to a specific branch?

Is this even possible? I can't find anything in the docs.

8 replies

XXata

•Created by Michael Schaufelberger on 6/11/2024 in #help

`xata pull <branch>` results in a high number of migration files - Safe to ignore?

This happens after running migrations or pushes with Drizzle. I have been testing different data models and now have over 200 migration files. Is it safe to ignore the folder since we are planning on running all migrations with Drizzle anyways?

4 replies

XXata

•Created by Michael Schaufelberger on 6/10/2024 in #help

Re-executing Drizzle Kit push fails after successful push

I'm getting the following error: PL/pgSQL function "xata.pg_catalog".regnamespace(text) line 14 at RAISE

> pnpx drizzle-kit push
drizzle-kit: v0.21.4
drizzle-orm: v0.30.10

No config path provided, using default path
Reading config file '/redacted/apps/partner-tool/app/drizzle.config.ts'
Using 'pg' driver for database querying
[⣟] Pulling schema from database.../redacted/node_modules/.pnpm/drizzle-kit@0.21.4_@esbuild-kit+esm-loader@2.6.5_commander@10.0.1_drizzle-orm@0.30.10_@neonda_jne73tzufwvw4dok65xkag4jqi/node_modules/drizzle-kit/bin.cjs:69429
        Error.captureStackTrace(err2);
              ^

error: schema "publics" does not exist
    at /redacted/node_modules/.pnpm/drizzle-kit@0.21.4_@esbuild-kit+esm-loader@2.6.5_commander@10.0.1_drizzle-orm@0.30.10_@neonda_jne73tzufwvw4dok65xkag4jqi/node_modules/drizzle-kit/bin.cjs:69429:15
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async Object.query (/redacted/node_modules/.pnpm/drizzle-kit@0.21.4_@esbuild-kit+esm-loader@2.6.5_commander@10.0.1_drizzle-orm@0.30.10_@neonda_jne73tzufwvw4dok65xkag4jqi/node_modules/drizzle-kit/bin.cjs:111366:26)
    at async /redacted/node_modules/.pnpm/drizzle-kit@0.21.4_@esbuild-kit+esm-loader@2.6.5_commander@10.0.1_drizzle-orm@0.30.10_@neonda_jne73tzufwvw4dok65xkag4jqi/node_modules/drizzle-kit/bin.cjs:15979:46 {
  length: 166,
  severity: 'ERROR',
  code: 'P0001',
  detail: undefined,
  hint: undefined,
  position: undefined,
  internalPosition: undefined,
  internalQuery: undefined,
  where: 'PL/pgSQL function "xata.pg_catalog".regnamespace(text) line 14 at RAISE',
  schema: undefined,
  table: undefined,
  column: undefined,
  dataType: undefined,
  constraint: undefined,
  file: 'pl_exec.c',
  line: '3923',
  routine: 'exec_stmt_raise'
}

> pnpx drizzle-kit push
drizzle-kit: v0.21.4
drizzle-orm: v0.30.10

No config path provided, using default path
Reading config file '/redacted/apps/partner-tool/app/drizzle.config.ts'
Using 'pg' driver for database querying
[⣟] Pulling schema from database.../redacted/node_modules/.pnpm/drizzle-kit@0.21.4_@esbuild-kit+esm-loader@2.6.5_commander@10.0.1_drizzle-orm@0.30.10_@neonda_jne73tzufwvw4dok65xkag4jqi/node_modules/drizzle-kit/bin.cjs:69429
        Error.captureStackTrace(err2);
              ^

error: schema "publics" does not exist
    at /redacted/node_modules/.pnpm/drizzle-kit@0.21.4_@esbuild-kit+esm-loader@2.6.5_commander@10.0.1_drizzle-orm@0.30.10_@neonda_jne73tzufwvw4dok65xkag4jqi/node_modules/drizzle-kit/bin.cjs:69429:15
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async Object.query (/redacted/node_modules/.pnpm/drizzle-kit@0.21.4_@esbuild-kit+esm-loader@2.6.5_commander@10.0.1_drizzle-orm@0.30.10_@neonda_jne73tzufwvw4dok65xkag4jqi/node_modules/drizzle-kit/bin.cjs:111366:26)
    at async /redacted/node_modules/.pnpm/drizzle-kit@0.21.4_@esbuild-kit+esm-loader@2.6.5_commander@10.0.1_drizzle-orm@0.30.10_@neonda_jne73tzufwvw4dok65xkag4jqi/node_modules/drizzle-kit/bin.cjs:15979:46 {
  length: 166,
  severity: 'ERROR',
  code: 'P0001',
  detail: undefined,
  hint: undefined,
  position: undefined,
  internalPosition: undefined,
  internalQuery: undefined,
  where: 'PL/pgSQL function "xata.pg_catalog".regnamespace(text) line 14 at RAISE',
  schema: undefined,
  table: undefined,
  column: undefined,
  dataType: undefined,
  constraint: undefined,
  file: 'pl_exec.c',
  line: '3923',
  routine: 'exec_stmt_raise'
}

Is this an error with Xata or Drizzle? It loks like Xata. Also, I'm not referencing publics anywhere. Which seems like a typo?

15 replies

XXata

•Created by Michael Schaufelberger on 6/7/2024 in #help

Can we use ULIDs without a dedicated cluster?

We are looking for some solution between a Postgres serial and a long UUID. So ULIDs are a natural choice. I would use something like https://github.com/geckoboard/pgulid or https://github.com/pksunkara/pgx_ulid. But without a dedicated cluster we cannot create types/functions. So is there another way to use ULID-like IDs? I've seen that select ('rec_'::text || (xata_private.xid())::text) produces a similar result. Can we use this for the foreseeable future for our own ID fields?

5 replies

XXata

•Created by Michael Schaufelberger on 5/27/2024 in #help

Migrations with Drizzle

Hi What is the current best practice regarding migrations when I'm using Drizzle with Xata? With Planetscale I just used Drizzle Kit's db push on db feature branches because the actual migration to prod was handled through the Planetscale dashboard.

35 replies

Gaming

Programming