Theo Gravity Comments - Answer Overflow

Theo Gravity

Explore posts from servers

•Created by Theo Gravity on 6/12/2023 in #❓-help

How to use the errorFormatter when using an adapter?

Ah, it was a dumb question. It has nothing to do with adapters.

const t = initTRPC.context<OuterContext>().create({
  errorFormatter({ shape }) {
    
  }
});

export const router = t.router;
export const middleware = t.middleware;
export const mergeRouters = t.mergeRouters;
export const procedure = t.procedure;

const t = initTRPC.context<OuterContext>().create({
  errorFormatter({ shape }) {
    
  }
});

export const router = t.router;
export const middleware = t.middleware;
export const mergeRouters = t.mergeRouters;
export const procedure = t.procedure;

4 replies

TtRPC

•Created by Theo Gravity on 6/10/2023 in #❓-help

How do I log thrown errors from the query/ mutation on the server side globally?

Typing on did help in the autocomplete for the adapter, thanks

    trpcExpress.createExpressMiddleware({
      router: appRouter,
      createContext,
      onError({ error, ctx }) {

      },

    trpcExpress.createExpressMiddleware({
      router: appRouter,
      createContext,
      onError({ error, ctx }) {

      },

9 replies

TtRPC

•Created by Theo Gravity on 6/10/2023 in #❓-help

How do I log thrown errors from the query/ mutation on the server side globally?

Same with the source for it: https://github.com/trpc/trpc/blob/main/examples/express-server/src/server.ts

9 replies

TtRPC

•Created by Theo Gravity on 6/10/2023 in #❓-help

How do I log thrown errors from the query/ mutation on the server side globally?

It doesn't offer anything around that: https://trpc.io/docs/server/adapters/express

9 replies

KKysely

•Created by Theo Gravity on 5/26/2023 in #help

How to translate the WITH keyword in postgres when SELECT does not have FROM?

Linking the github issue thread so it can be traced: https://github.com/kysely-org/kysely/issues/517

9 replies

KKysely

•Created by Theo Gravity on 5/26/2023 in #help

How to translate the WITH keyword in postgres when SELECT does not have FROM?

Thanks Igal. I've responded. It's close but not the solution unfortunately

9 replies

KKysely

•Created by Theo Gravity on 5/26/2023 in #help

How to translate the WITH keyword in postgres when SELECT does not have FROM?

Got it to all work, it feels like a huge hack though:

export async function similaritySearch({
  db,
  text,
  model,
}: {
  db: Kysely<SbDatabase>;
  text: string;
  model?: string;
}): Promise<SimilaritySearchRow[]> {
  return (
    db
      .with(
        'query',
        () =>
          // Ignoring since kysely doesn't know how to work with selects without FROM
          // You can see what this query outputs to here: https://kyse.link/?p=s&i=hrcwDweUyG01KMCYp34j
          // eslint-disable-next-line @typescript-eslint/ban-ts-comment
          //@ts-ignore
          sql`(SELECT pgml.embed(${sql.val(model)}, ${sql.val(text)}) as embedding)`,
      )
      .selectFrom('document_chunk_embedding')
      // eslint-disable-next-line @typescript-eslint/ban-ts-comment
      //@ts-ignore
      .innerJoin('document_chunk', 'document_chunk_embedding.document_chunk_id', sql.raw('document_chunk.id, query'))
      .select([
        'document_chunk_embedding.document_chunk_id',
        'document_chunk.text as similarity',
        // eslint-disable-next-line @typescript-eslint/ban-ts-comment
        //@ts-ignore
        db.fn('pgml.cosine_similarity', ['document_chunk_embedding.embedding', 'query.embedding']),
      ])
      .orderBy('similarity', 'desc')
      .limit(5)
      .execute() as Promise<SimilaritySearchRow[]>
  );
}

export async function similaritySearch({
  db,
  text,
  model,
}: {
  db: Kysely<SbDatabase>;
  text: string;
  model?: string;
}): Promise<SimilaritySearchRow[]> {
  return (
    db
      .with(
        'query',
        () =>
          // Ignoring since kysely doesn't know how to work with selects without FROM
          // You can see what this query outputs to here: https://kyse.link/?p=s&i=hrcwDweUyG01KMCYp34j
          // eslint-disable-next-line @typescript-eslint/ban-ts-comment
          //@ts-ignore
          sql`(SELECT pgml.embed(${sql.val(model)}, ${sql.val(text)}) as embedding)`,
      )
      .selectFrom('document_chunk_embedding')
      // eslint-disable-next-line @typescript-eslint/ban-ts-comment
      //@ts-ignore
      .innerJoin('document_chunk', 'document_chunk_embedding.document_chunk_id', sql.raw('document_chunk.id, query'))
      .select([
        'document_chunk_embedding.document_chunk_id',
        'document_chunk.text as similarity',
        // eslint-disable-next-line @typescript-eslint/ban-ts-comment
        //@ts-ignore
        db.fn('pgml.cosine_similarity', ['document_chunk_embedding.embedding', 'query.embedding']),
      ])
      .orderBy('similarity', 'desc')
      .limit(5)
      .execute() as Promise<SimilaritySearchRow[]>
  );
}

9 replies

KKysely

•Created by Theo Gravity on 5/26/2023 in #help

How to translate the WITH keyword in postgres when SELECT does not have FROM?

Not sure how to do that

9 replies

KKysely

•Created by Theo Gravity on 5/26/2023 in #help

How to translate the WITH keyword in postgres when SELECT does not have FROM?

It seems query needs to be at the end of the join and can't be part of SELECT FROM

9 replies

KKysely

•Created by Theo Gravity on 5/26/2023 in #help

How to translate the WITH keyword in postgres when SELECT does not have FROM?

So far I have:

  return db
    .with('query', (qb) => qb.selectFrom('??').select([db.fn('pgml.embed', [sql.val(model), sql.val(text)]).as('embedding')]))
    .selectFrom(['document_chunk_embedding', 'query'])
    .innerJoin('document_chunk', 'document_chunk_embedding.document_chunk_id', 'document_chunk.id')
    .select(['document_chunk_embedding.document_chunk_id', 'document_chunk.text as similarity', db.fn('pgml.cosine_similarity', ['document_chunk_embedding.embedding', 'query.embedding'])])
    .orderBy('similarity', 'desc')
    .limit(5)
    .returningAll()
    .execute();

  return db
    .with('query', (qb) => qb.selectFrom('??').select([db.fn('pgml.embed', [sql.val(model), sql.val(text)]).as('embedding')]))
    .selectFrom(['document_chunk_embedding', 'query'])
    .innerJoin('document_chunk', 'document_chunk_embedding.document_chunk_id', 'document_chunk.id')
    .select(['document_chunk_embedding.document_chunk_id', 'document_chunk.text as similarity', db.fn('pgml.cosine_similarity', ['document_chunk_embedding.embedding', 'query.embedding'])])
    .orderBy('similarity', 'desc')
    .limit(5)
    .returningAll()
    .execute();

I think the problem is that the SELECT in question doesn't have a FROM in the WITH

9 replies

KKysely

•Created by Theo Gravity on 5/25/2023 in #help

Using Postgres function in an insert with a field as a select

Agreed! Thanks! Here's the final result:

export async function generateTextDocumentChunks(
  db: Kysely<SbDatabase>,
  document: TextDocumentRow,
): Promise<TextDocumentChunkRow[]> {
  return db
    .insertInto('text_document_chunk')
    .columns(['text_document_id', 'text'])
    .expression((eb) =>
      eb
        .selectFrom(
          eb
            .fn<{ sentence: string }>('unnest', [
              eb.fn('string_to_array', [sql.val(document.text_content), sql.val('. ')]),
            ])
            .as('sentence'),
        )
        .where('sentence', '~', '[[:alnum:]]')
        .select([sql.val(document.id).as('text_document_id'), 'sentence']),
    )
    .returningAll()
    .execute();
}

export async function generateTextDocumentChunks(
  db: Kysely<SbDatabase>,
  document: TextDocumentRow,
): Promise<TextDocumentChunkRow[]> {
  return db
    .insertInto('text_document_chunk')
    .columns(['text_document_id', 'text'])
    .expression((eb) =>
      eb
        .selectFrom(
          eb
            .fn<{ sentence: string }>('unnest', [
              eb.fn('string_to_array', [sql.val(document.text_content), sql.val('. ')]),
            ])
            .as('sentence'),
        )
        .where('sentence', '~', '[[:alnum:]]')
        .select([sql.val(document.id).as('text_document_id'), 'sentence']),
    )
    .returningAll()
    .execute();
}

20 replies

KKysely

•Created by Theo Gravity on 5/25/2023 in #help

Using Postgres function in an insert with a field as a select

Reading the doc, doing:

sql`${value}`

sql`${value}`

should be the equiv? This does work for me

export async function generateTextDocumentChunks(
  db: Kysely<SbDatabase>,
  document: TextDocumentRow,
): Promise<TextDocumentChunkRow[]> {
  return db
    .insertInto('text_document_chunk')
    .columns(['text_document_id', 'text'])
    .expression((eb) =>
      eb
        .selectFrom(
          eb
            .fn<{ sentence: string }>('unnest', [
              eb.fn('string_to_array', [sql`${document.text_content}`, sql.lit('. ')]),
            ])
            .as('sentence'),
        )
        .where('sentence', '~', '[[:alnum:]]')
        .select([sql`${document.id}`.as('text_document_id'), 'sentence']),
    )
    .returningAll()
    .execute();
}

export async function generateTextDocumentChunks(
  db: Kysely<SbDatabase>,
  document: TextDocumentRow,
): Promise<TextDocumentChunkRow[]> {
  return db
    .insertInto('text_document_chunk')
    .columns(['text_document_id', 'text'])
    .expression((eb) =>
      eb
        .selectFrom(
          eb
            .fn<{ sentence: string }>('unnest', [
              eb.fn('string_to_array', [sql`${document.text_content}`, sql.lit('. ')]),
            ])
            .as('sentence'),
        )
        .where('sentence', '~', '[[:alnum:]]')
        .select([sql`${document.id}`.as('text_document_id'), 'sentence']),
    )
    .returningAll()
    .execute();
}

20 replies

KKysely

•Created by Theo Gravity on 5/25/2023 in #help

Using Postgres function in an insert with a field as a select

This works, but it's not ideal as sql.lit says it's susceptible to SQL injection. Is there a version that I can use that would be safe (eg parameter binding)? The document.text_content variable would come from user input, and the contents of that variable is whatever they wrote. We may or may not have sanitized it by the time it gets called here.

export async function generateTextDocumentChunks(
  db: Kysely<SbDatabase>,
  document: TextDocumentRow,
): Promise<TextDocumentChunkRow[]> {
  return db
    .insertInto('text_document_chunk')
    .columns(['text_document_id', 'text'])
    .expression((eb) =>
      eb
        .selectFrom(
          eb
            .fn<{ sentence: string }>('unnest', [
              eb.fn('string_to_array', [sql.lit(document.text_content), sql.lit('. ')]),
            ])
            .as('sentence'),
        )
        .where('sentence', '~', '[[:alnum:]]')
        .select([sql.lit(document.id).as('text_document_id'), 'sentence']),
    )
    .returningAll()
    .execute();
}

export async function generateTextDocumentChunks(
  db: Kysely<SbDatabase>,
  document: TextDocumentRow,
): Promise<TextDocumentChunkRow[]> {
  return db
    .insertInto('text_document_chunk')
    .columns(['text_document_id', 'text'])
    .expression((eb) =>
      eb
        .selectFrom(
          eb
            .fn<{ sentence: string }>('unnest', [
              eb.fn('string_to_array', [sql.lit(document.text_content), sql.lit('. ')]),
            ])
            .as('sentence'),
        )
        .where('sentence', '~', '[[:alnum:]]')
        .select([sql.lit(document.id).as('text_document_id'), 'sentence']),
    )
    .returningAll()
    .execute();
}

20 replies

KKysely

•Created by Theo Gravity on 5/25/2023 in #help

Using Postgres function in an insert with a field as a select

really helped!

20 replies

KKysely

•Created by Theo Gravity on 5/25/2023 in #help

Using Postgres function in an insert with a field as a select

thank you for your time Igal

20 replies

KKysely

•Created by Theo Gravity on 5/25/2023 in #help

Using Postgres function in an insert with a field as a select

i think i understand the mismatch, in the original fn, NEW.text_content is an actual column, whereas here I'm feeding in the content, which I don't think is allowed?

20 replies

KKysely

•Created by Theo Gravity on 5/25/2023 in #help

Using Postgres function in an insert with a field as a select

this is the contents of document.text_content

20 replies

KKysely

•Created by Theo Gravity on 5/25/2023 in #help

Using Postgres function in an insert with a field as a select

20 replies

KKysely

•Created by Theo Gravity on 5/25/2023 in #help

Using Postgres function in an insert with a field as a select

Thanks! In my translation, I'm not using NEW (trying to translate out of triggers / pg functions), and have document instead as the object with the values. So would it be sql.lit(document.text_content)?

20 replies

KKysely

•Created by Theo Gravity on 5/25/2023 in #help

Using Postgres function in an insert with a field as a select

Would this be the correct version? One concern I have is does this properly handle issues around SQL injection?

db.insertInto('text_document_chunk').values((eb) => ({
    text_document_id: document.id,
    text: db
      .selectFrom(
        eb.fn('unnest', [eb.fn('string_to_array', [sql.lit(document.text_content), sql.lit('. ')])]).as('sentence'),
      )
      .where(sql.lit('sentence'), '~', '[[:alnum:]]').execute()

db.insertInto('text_document_chunk').values((eb) => ({
    text_document_id: document.id,
    text: db
      .selectFrom(
        eb.fn('unnest', [eb.fn('string_to_array', [sql.lit(document.text_content), sql.lit('. ')])]).as('sentence'),
      )
      .where(sql.lit('sentence'), '~', '[[:alnum:]]').execute()

20 replies

Gaming

Programming