christrading Comments - Answer Overflow

Topics

christrading

Explore posts from servers

TTCTheo's Typesafe Cult DTDrizzle Team

•Created by christrading on 11/20/2024 in #help

Transaction Query Statements

Okay. I recognize that transactions lock the database so other queries or transactions outside of that transaction aren't able to connect at the same time; I need the numerous queries wrapped in a transaction so that if one fails they all fail for the requirement of atomic strongly consistent serialization needed for Replicache. I do have .prepare() on my queries. Is this the equivalent of a stored procedure or do I need to write the procedure in raw sql? I didn't see too much on Drizzle stored procedures apart from these github threads: https://github.com/drizzle-team/drizzle-orm/discussions/2434

14 replies

•Created by christrading on 11/20/2024 in #help

Transaction Query Statements

Multiple queries within a transaction still cause multiple round-trips, a round-trip per query. Is there any way to optimize this?

14 replies

•Created by christrading on 11/20/2024 in #help

Transaction Query Statements

How about within my transaction rather than outside? Within my transaction I have a number of these queries that aren’t dependent of each other that I would have liked to batch together to save on round trips and latency. I dont have anything outside the transaction Id like to batch together with

14 replies

•Created by christrading on 11/20/2024 in #help

Transaction Query Statements

async function searchClients(tx: Transaction, clientGroupID: string, sinceClientVersion: number) {
  const clientRowStatementQuery = tx
    .select({
      id: replicacheClient.id,
      lastMutationID: replicacheClient.lastMutationID,
      clientVersion: replicacheClient.clientVersion,
    })
    .from(replicacheClient)
    .where(and(
      eq(replicacheClient.clientGroupID, clientGroupID),
      gt(replicacheClient.clientVersion, sinceClientVersion),
    ))
    .prepare()

  const clientRows = await clientRowStatementQuery.all()

  const clients = clientRows.map((row) => {
    const client: ClientRecord = {
      id: row.id,
      clientGroupID,
      lastMutationID: row.lastMutationID,
      clientVersion: row.clientVersion,
    }
    return client
  })

  return clients
}

export async function searchLists(
  tx: Transaction,
  accessibleByUserID: string,
): Promise<SearchResult[]> {
  const admin = await isAdmin(tx, accessibleByUserID)
  if (admin) {
    const listRowStatementQuery = tx
      .select({
        id: list.id,
        rowVersion: list.rowVersion,
      })
      .from(list)
      .prepare()

    const listRows = await listRowStatementQuery.all()
    return listRows
  }

  const shareRowStatementSubquery = tx
    .select({
      id: share.listID,
    })
    .from(share)
    .where(eq(share.userID, accessibleByUserID))

  const listRowStatementQuery = tx
    .select({
      id: list.id,
      rowVersion: list.rowVersion,
    })
    .from(list)
    .where(
      or(
        eq(list.ownerID, accessibleByUserID),
        inArray(list.id, shareRowStatementSubquery),
      ),
    )
    .prepare()

  const listRows = await listRowStatementQuery.all()

  return listRows
}

async function searchClients(tx: Transaction, clientGroupID: string, sinceClientVersion: number) {
  const clientRowStatementQuery = tx
    .select({
      id: replicacheClient.id,
      lastMutationID: replicacheClient.lastMutationID,
      clientVersion: replicacheClient.clientVersion,
    })
    .from(replicacheClient)
    .where(and(
      eq(replicacheClient.clientGroupID, clientGroupID),
      gt(replicacheClient.clientVersion, sinceClientVersion),
    ))
    .prepare()

  const clientRows = await clientRowStatementQuery.all()

  const clients = clientRows.map((row) => {
    const client: ClientRecord = {
      id: row.id,
      clientGroupID,
      lastMutationID: row.lastMutationID,
      clientVersion: row.clientVersion,
    }
    return client
  })

  return clients
}

export async function searchLists(
  tx: Transaction,
  accessibleByUserID: string,
): Promise<SearchResult[]> {
  const admin = await isAdmin(tx, accessibleByUserID)
  if (admin) {
    const listRowStatementQuery = tx
      .select({
        id: list.id,
        rowVersion: list.rowVersion,
      })
      .from(list)
      .prepare()

    const listRows = await listRowStatementQuery.all()
    return listRows
  }

  const shareRowStatementSubquery = tx
    .select({
      id: share.listID,
    })
    .from(share)
    .where(eq(share.userID, accessibleByUserID))

  const listRowStatementQuery = tx
    .select({
      id: list.id,
      rowVersion: list.rowVersion,
    })
    .from(list)
    .where(
      or(
        eq(list.ownerID, accessibleByUserID),
        inArray(list.id, shareRowStatementSubquery),
      ),
    )
    .prepare()

  const listRows = await listRowStatementQuery.all()

  return listRows
}

14 replies

•Created by christrading on 11/20/2024 in #help

Transaction Query Statements

  const baseClientGroupRecord = await getClientGroupForUpdate(tx, clientGroupID)
  const clientChanges = await searchClients(tx, clientGroupID, baseCVR.clientVersion)
  const listMeta = await searchLists(tx, userID)

export async function getClientGroupForUpdate(
  tx: Transaction,
  clientGroupID: string,
): Promise<ClientGroupRecord> {
  const previousClientGroup = await getClientGroup(tx, clientGroupID)
  return {
    id: clientGroupID,
    clientGroupVersion: previousClientGroup.clientGroupVersion,
    cvrVersion: previousClientGroup.cvrVersion,
  }
}

async function getClientGroup(
  tx: Transaction,
  clientGroupID: string,
): Promise<Omit<ClientGroupRecord, 'id'>> {
  const clientGroupRowStatementQuery = tx
    .select({
      cvrVersion: replicacheClientGroup.cvrVersion,
      clientGroupVersion: replicacheClientGroup.clientGroupVersion,
    })
    .from(replicacheClientGroup)
    .where(eq(replicacheClientGroup.id, clientGroupID))
    .prepare()

  const clientGroupRow = await clientGroupRowStatementQuery.get() || {
    clientGroupVersion: 0,
    cvrVersion: null,
  }

  return clientGroupRow
}

  const baseClientGroupRecord = await getClientGroupForUpdate(tx, clientGroupID)
  const clientChanges = await searchClients(tx, clientGroupID, baseCVR.clientVersion)
  const listMeta = await searchLists(tx, userID)

export async function getClientGroupForUpdate(
  tx: Transaction,
  clientGroupID: string,
): Promise<ClientGroupRecord> {
  const previousClientGroup = await getClientGroup(tx, clientGroupID)
  return {
    id: clientGroupID,
    clientGroupVersion: previousClientGroup.clientGroupVersion,
    cvrVersion: previousClientGroup.cvrVersion,
  }
}

async function getClientGroup(
  tx: Transaction,
  clientGroupID: string,
): Promise<Omit<ClientGroupRecord, 'id'>> {
  const clientGroupRowStatementQuery = tx
    .select({
      cvrVersion: replicacheClientGroup.cvrVersion,
      clientGroupVersion: replicacheClientGroup.clientGroupVersion,
    })
    .from(replicacheClientGroup)
    .where(eq(replicacheClientGroup.id, clientGroupID))
    .prepare()

  const clientGroupRow = await clientGroupRowStatementQuery.get() || {
    clientGroupVersion: 0,
    cvrVersion: null,
  }

  return clientGroupRow
}

14 replies

•Created by christrading on 5/6/2024 in #help

Drizzle Migrate with Turso

Okay. I've been trying to figure this out for the past while. I think I got it fixed unless it just totally breaks later down the road. What migration the pnpm migrate script thinks has or should be applied pertains to the _journal.json. So I removed the "ghost" migration from there, and pnpm generate; pnpm migrate applied a second migration which is dropping columns I added to get the database back to what it was before. I then committed the removal of those two sets migration changes of add columns and drop columns with the _journal.json, xxxx_snapshot.json, and xxxx_yyyy_zzzz.sql files. Then I removed those two most recent __drizzle_migrations rows with the SQL runner with

DELETE FROM "__drizzle_migrations"
WHERE created_at = (
    SELECT MAX(created_at)
    FROM "__drizzle_migrations"
);

DELETE FROM "__drizzle_migrations"
WHERE created_at = (
    SELECT MAX(created_at)
    FROM "__drizzle_migrations"
);

51 replies

•Created by christrading on 5/6/2024 in #help

Drizzle Migrate with Turso

@Angelelz would I be able to expect this fix which so far works to be reliable and stable? https://discord.com/channels/933071162680958986/1281003002479509636/1281752596876496979

51 replies

•Created by christrading on 5/6/2024 in #help

Drizzle Migrate with Turso

do you know a way to modify/delete that one row? when i try to do anything it either errors out or modifies all rows, which is not the behaviour id expect

51 replies

•Created by christrading on 5/6/2024 in #help

Drizzle Migrate with Turso

No description

51 replies

•Created by christrading on 5/6/2024 in #help

Drizzle Migrate with Turso

i also found and tried adding in the exact same file changes from the original migration commit I undid, but its considering it as a second migration instead of the same one

51 replies

•Created by christrading on 5/6/2024 in #help

Drizzle Migrate with Turso

@Angelelz is there anyway to apply/unapply database changes from migration files at this point? the last migration file/change was applied, i removed the files with incorrect understanding trying to undo the database changes, so now it's out of sync, when i do any other migration change to remedy it

51 replies

•Created by christrading on 5/6/2024 in #help

Drizzle Migrate with Turso

i cant, i just "delete 1 record" and it deletes everything D:

51 replies

•Created by christrading on 5/6/2024 in #help

Drizzle Migrate with Turso

can i just manually delete the most recent row in the __drizzle_migrations folder?

51 replies

•Created by christrading on 5/6/2024 in #help

Drizzle Migrate with Turso

Okay I guess Ill ask the Turso team then because yeah drizzle-kit drop seems to do just the same thing as what I did with git reset and force push the last migration change away. the __drizzle_migrations folder in Turso didnt remove the last migration upon doing so

51 replies

•Created by christrading on 5/6/2024 in #help

Drizzle Migrate with Turso

"let you select the one you want to delete" so it only works in an interactive console? I wont be able to use drizzle-kit drop in my pre-build step of my deployment pipeline? that's where i do my drizzle-kit migrate. or are there any flags available for it to just drop the one most recent migration?

51 replies

•Created by christrading on 5/6/2024 in #help

Drizzle Migrate with Turso

Given that I removed my schema and migration changes with git reset and force push, and thus unfortunately can't really get that back, should I do anything first before applying drizzle-kit drop? As the docs say Please don’t delete any files in migrations folder manually, it might break drizzle-kit https://orm.drizzle.team/kit-docs/commands#drop-migration ?

51 replies

•Created by christrading on 5/6/2024 in #help

Drizzle Migrate with Turso

How do I run the migration in a transaction? I just run drizzle-kit migrate

51 replies

•Created by christrading on 5/6/2024 in #help

Drizzle Migrate with Turso

@Angelelz How do you go back to a previous migration point in history? I had a migration that failed but that still made changes to the database table. I did git reset HEAD~2 and git push -f to before the schema and migration changes where I added the couple field rows in a table, but when I check my table with Drizzle, the rows are still there I wouldn't expect a failed migration to still make the changes (halfway or incorrectly) so now I'm simply trying to revert back before the failed migration so I may re-try to correctly apply the intended changes

51 replies

•Created by shikishikichangchang on 11/24/2023 in #help

Transaction doesn't support Promise.all

await Promise.all(insertStatements.map((query) => query.run()))

await Promise.all(insertStatements.map((query) => query.run()))

should it be .run() or .execute() to have these queries done in parallel?

82 replies

•Created by shikishikichangchang on 11/24/2023 in #help

Transaction doesn't support Promise.all

How do we have the .run() or .execute() of our queries run in the Promise.all and not when we are defining the queries?

82 replies