Drizzle Team•11mo ago

Front to back end type inference with parameters sent via request

Hey there, I've been exploring using drizzle in a TRPC environment. This is probably a pipe dream, but ideally, our developers would be able to dynamically pass a findMany input and have the query's results inferred all the way to the front end.

// Front end
const { data } = trpc.tablename.get.useQuery({
    where: { } // <== where clause options are dynamically inferred
    with: { nestedEntities: true } // <== with clause options are dynamically inferred
})

console.log(data?.[0]?.nestedEntities)
// ^ This is valid when the with clause is provided to the hook, but will not if it's not there

// Front end
const { data } = trpc.tablename.get.useQuery({
    where: { } // <== where clause options are dynamically inferred
    with: { nestedEntities: true } // <== with clause options are dynamically inferred
})

console.log(data?.[0]?.nestedEntities)
// ^ This is valid when the with clause is provided to the hook, but will not if it's not there

The first issue I encountered is that there needs to be a translation layer for the where statements, as we cannot send drizzle operator functions over a request. To handle this, I built out a recursive layer for where statements at the root and also in any nested with and it works functionally very well (though is only strongly typed at the root layer). Example:

where: {
    columnA: { equal_to: 123 },
    or: [
        columnB: { isNull: false },
        columnA: { likeInsensitive: '%TRPC%' }
    ]
}

where: {
    columnA: { equal_to: 123 },
    or: [
        columnB: { isNull: false },
        columnA: { likeInsensitive: '%TRPC%' }
    ]
}

The above's operators and column names all have type inference/intellisense based on the DB model. The above turns into this on the backend:

where: (table, { and, eq, or, isNull, ilike }) =>
    and(...[
        eq(table['columnA'], 123),
        or(...[
            isNull(table['columnB']),
            ilike(table['columnA'], '%TRPC%')
        ])
    )

where: (table, { and, eq, or, isNull, ilike }) =>
    and(...[
        eq(table['columnA'], 123),
        or(...[
            isNull(table['columnB']),
            ilike(table['columnA'], '%TRPC%')
        ])
    )

Is this a fine approach? Unfortunately, while I have it functionally working with nested where statements inside of with clauses, when I try to get the types to be inferred all the way down I end up with:

error TS7056: The inferred type of this node exceeds the maximum length the compiler will serialize. An explicit type annotation is needed.

My question is - has anyone tried full front to back end type inference? Or even just how to best handle findMany params via the request?

9 Replies

Angelelz•11mo ago

This is a lot easier, in my opinion, than what you are thinking Notice that the where clause that you have there doesn't change the return type of the findMany For the where statements inside the with clause, you'll need to create your own types to translate your front-end object to the types from drizzle

MiNiMAL•11mo ago

Nested with clauses aside, what about parameters like columns that should directly affect the return value?

export const getEntityFetchProcedures = <TTableName extends TableNames>(
    table: TTableName,
) => ({
    discordDemo: privateProcedure
        .input(
            columns =>
                ({ columns } as {
                    columns: NonNullable<FindFirstArg<TTableName>>['columns']
                }),
        )
        .query(async ({ input }) => {
            const queryRef = db.query[table]
            return await queryRef.findFirst(input)
        }),
})

export const getEntityFetchProcedures = <TTableName extends TableNames>(
    table: TTableName,
) => ({
    discordDemo: privateProcedure
        .input(
            columns =>
                ({ columns } as {
                    columns: NonNullable<FindFirstArg<TTableName>>['columns']
                }),
        )
        .query(async ({ input }) => {
            const queryRef = db.query[table]
            return await queryRef.findFirst(input)
        }),
})

MiNiMAL•11mo ago

In this case, I have a route that takes an input expecting to match just the columns field. However when I try this I get the below error

MiNiMAL•11mo ago

I definitely won't discount that I may be going about this the wrong way for what it's worth! But I'm super curious if this is feasible because it would perfect the developer experience here.

Angelelz•11mo ago

I would advise not to let typescript infer the type but build it yourself

MiNiMAL•11mo ago

Gotcha ok, I’ll look into doing that in the morning, thanks so much! Do you have any guidance on utility types I should be using to build it?

Angelelz•11mo ago

You just need to annotate the function you are writing, or the values from the DB. First a couple type helpers that will make everything super easy:

type Schema = typeof schema;
type DizzleTables = {
  [K in keyof Schema as Schema[K] extends AnyPgTable ? K : never]: Schema[K];
};
type TableNames = keyof DizzleTables;
type ReturnedColumns<TTableName extends TableNames, TColumns extends string> = {
  [K in TColumns]: K extends keyof DizzleTables[TTableName]["$inferSelect"]
    ? DizzleTables[TTableName]["$inferSelect"][K]
    : never;
};
type PartialColumnSelects<TTableName extends TableNames> = {
  columns: {
    [K in keyof ExtractTablesWithRelations<Schema>[TTableName]["columns"]]?: boolean;
  };
};

type Schema = typeof schema;
type DizzleTables = {
  [K in keyof Schema as Schema[K] extends AnyPgTable ? K : never]: Schema[K];
};
type TableNames = keyof DizzleTables;
type ReturnedColumns<TTableName extends TableNames, TColumns extends string> = {
  [K in TColumns]: K extends keyof DizzleTables[TTableName]["$inferSelect"]
    ? DizzleTables[TTableName]["$inferSelect"][K]
    : never;
};
type PartialColumnSelects<TTableName extends TableNames> = {
  columns: {
    [K in keyof ExtractTablesWithRelations<Schema>[TTableName]["columns"]]?: boolean;
  };
};

This will make sure your TableNames are correctly linked to the schema. So will be the columns you pass from the front end

const getEntityFetchProcedures = <
  TTableName extends TableNames,
  TColumns extends PartialColumnSelects<TTableName>,
>(
  table: TTableName,
) => ({
  discordDemo: privateProcedure
    .input((columns) => ({ columns }) as TColumns)
    .query(
      async ({
        input,
      }): Promise<
        | ReturnedColumns<TTableName, keyof TColumns["columns"] & string>
        | undefined
      > => {
        const queryRef = db.query[table];
        return (await queryRef.findFirst(input)) as any;
      },
    ),
});

const getEntityFetchProcedures = <
  TTableName extends TableNames,
  TColumns extends PartialColumnSelects<TTableName>,
>(
  table: TTableName,
) => ({
  discordDemo: privateProcedure
    .input((columns) => ({ columns }) as TColumns)
    .query(
      async ({
        input,
      }): Promise<
        | ReturnedColumns<TTableName, keyof TColumns["columns"] & string>
        | undefined
      > => {
        const queryRef = db.query[table];
        return (await queryRef.findFirst(input)) as any;
      },
    ),
});

I am not completely sure how to use TRPC properly, but the idea is to explicitly annotate the return type of the function BTW, this was a quick example, the idea is to point you in the right direction

MiNiMAL•11mo ago

This is incredibly helpful thank you and I really appreciate the effort in pointing me in the right direction. This has been such a fun exercise in improving my typescript skills so I greatly prefer guidance like this. I’ll play around over the next couple days and batch together any other questions I may have to save you time Really appreciate the time you’ve already taken for me - hope you have a great rest of your weekend

MiNiMAL•11mo ago

Just following up with this article, apparently I’ve been fighting against a tRPC limitation this whole time! https://dev.to/zenstack/limitation-of-trpcs-type-inference-and-how-we-improved-it-47fl

DEV Community

Limitation of TRPC's Type Inference and How We Improved It

TRPC, despite its short history, has gained much popularity in the Node.js/TypeScript community. One...

Gaming

Programming

Front to back end type inference with parameters sent via request