Jim
Jim
Explore posts from servers
CDCloudflare Developers
Created by Jim on 4/19/2024 in #general-help
Detecting bot activity by email servers
Hey all, I have emails that get sent out to clients for them to commit a decision (accept / decline). The links go to an endpoint which reads an encrypted url parameter and commits the decision to a database, then serves a html page... The problem: Some corporate emails use bots to determine whether or not the email is a security threat. Basically the bot is going to both the links and hitting the endpoints! Now an easy fix would be to add an extra step to the process and have clients click again once they reach to the page, but this is doubling the effort for users for the sake 1% of accounts that use a corporate emails with aggressive crawling. Is there some sort of cloudflare product that I could use to confirm if the link clicked is by a human? Any advice welcome Cheers!
6 replies
RRailway
Created by Jim on 4/1/2024 in #✋|help
Redeploying service doesnt use updated Source Repo URL
Hi im new to railway so maybe this is a misunderstanding. I have started a project using a template and want to change the github repo the service is reading from (the original github repo fails to build). I go to Source Repo and change, click redeploy, but the build step reading from the original URL. I guess "redeploying" is just going to use whatever deployment settings were set at the time. Pushing a commit to the repo doesnt seem to deploy anything so I'm not sure what else I can do.
14 replies
TTCTheo's Typesafe Cult
Created by Jim on 3/28/2024 in #questions
typescript globals: @types/web overrides fetch
Hey all I have a pnpm monorepo with a couple of packages and two next apps. In the next apps 'window' and 'document' arent read by typescript unless I install "@types/web" as a dependency. Works great. The problem is when it's installed my fetch definitions get nuked:
Property 'arrayBuffer' does not exist on type 'Response'.
Property 'arrayBuffer' does not exist on type 'Response'.
Property 'json' does not exist on type 'Response'.
Property 'json' does not exist on type 'Response'.
Anyone ran into this before? Could be related?? https://github.com/DefinitelyTyped/DefinitelyTyped/issues/60924
4 replies
DTDrizzle Team
Created by Jim on 3/19/2024 in #help
Defining websocket for NeonDB connection
Hey all, I'm having issues with using neondb on a playwright/test package in my monorepo. The connections work fine in my nextjs app but fail when running tests.
Error: All attempts to open a WebSocket to connect to the database failed. Please refer to https://github.com/neondatabase/serverless/blob/main/CONFIG.md#websocketconstructor-typeof-websocket--undefined. Details: fetch failed
Error: All attempts to open a WebSocket to connect to the database failed. Please refer to https://github.com/neondatabase/serverless/blob/main/CONFIG.md#websocketconstructor-typeof-websocket--undefined. Details: fetch failed
The error recommends defining a websocket constructor to the config of the neon client like so:
import { neonConfig } from '@neondatabase/serverless';
import ws from 'ws';
neonConfig.webSocketConstructor = ws;
import { neonConfig } from '@neondatabase/serverless';
import ws from 'ws';
neonConfig.webSocketConstructor = ws;
But currently I'm using the Pool as recommended by drizzle like so:
import { drizzle } from 'drizzle-orm/neon-serverless';
import { Pool } from '@neondatabase/serverless';
import * as schema from "./schema";
import dotenv from "dotenv";
dotenv.config({ path: ".env.local" });

const pool = new Pool({ connectionString: process.env.DB_URL });
export const db = drizzle(pool, { schema })
import { drizzle } from 'drizzle-orm/neon-serverless';
import { Pool } from '@neondatabase/serverless';
import * as schema from "./schema";
import dotenv from "dotenv";
dotenv.config({ path: ".env.local" });

const pool = new Pool({ connectionString: process.env.DB_URL });
export const db = drizzle(pool, { schema })
Pool doesn't accept a webSoctetConstructor, so I'm pretty stuck now lol. Any ideas? Thanks for reading!
4 replies
DTDrizzle Team
Created by Jim on 12/3/2023 in #help
avoiding conflicts in mySQL
Hi im trying to avoid conflicts in my app by checking the 'lastModified' column on the database against the client. But I also need to add successfully updated records to a status log table to record the history of changes. Because I'm using mySQL there's no way of returning specific rows effected. Is there a common approach for this?
export async function updateStatusByIds({
status,
records,
}: {
status: Statuses;
records: SelectRecordWithUserData[];
}) {

// assemble sql where clause to match id and current lastModified date
const sqlWhereChunks: SQL[] = [];

records.forEach((r, i) => {
sqlWhereChunks.push(
sql`(${recordTable.id} = ${r.id} AND ${recordTable.lastModified} = ${r.lastModified})`
);
if (i < records.length - 1) {
sqlWhereChunks.push(sql` OR `);
}
});

const sqlWhere = sql.join(sqlWhereChunks);

try {
await db.transaction(async (tx) => {
let update = await tx
.update(recordTable)
.set({ statusId: status })
.where(sqlWhere);

console.log("update:", update);

await tx.insert(statusLog).values(
records.map((r) => ({
recordId: r.id,
statusId: status,
message: "bulk",
}))
);
});
} catch (error) {
console.error(error);
throw new Error(
`Failed to update status ids: ${
error instanceof Error ? error.message : "Unknown error"
}`
);
}
}
export async function updateStatusByIds({
status,
records,
}: {
status: Statuses;
records: SelectRecordWithUserData[];
}) {

// assemble sql where clause to match id and current lastModified date
const sqlWhereChunks: SQL[] = [];

records.forEach((r, i) => {
sqlWhereChunks.push(
sql`(${recordTable.id} = ${r.id} AND ${recordTable.lastModified} = ${r.lastModified})`
);
if (i < records.length - 1) {
sqlWhereChunks.push(sql` OR `);
}
});

const sqlWhere = sql.join(sqlWhereChunks);

try {
await db.transaction(async (tx) => {
let update = await tx
.update(recordTable)
.set({ statusId: status })
.where(sqlWhere);

console.log("update:", update);

await tx.insert(statusLog).values(
records.map((r) => ({
recordId: r.id,
statusId: status,
message: "bulk",
}))
);
});
} catch (error) {
console.error(error);
throw new Error(
`Failed to update status ids: ${
error instanceof Error ? error.message : "Unknown error"
}`
);
}
}
33 replies
DTDrizzle Team
Created by Jim on 11/20/2023 in #help
json column type mismatch when using db.query
No description
6 replies
TTCTheo's Typesafe Cult
Created by Jim on 11/8/2023 in #questions
importing drizzle schema client side error [nextjs monorepo]
Hey all, I'm trying to import a drizzle schema to the client side and I'm getting a webpack loader error:
../../packages/db/utils/index.ts
Module parse failed: Unexpected token (11:1)
You may need an appropriate loader to handle this file type, currently no loaders are configured to process this file. See https://webpack.js.org/concepts#loaders
../../packages/db/utils/index.ts
Module parse failed: Unexpected token (11:1)
You may need an appropriate loader to handle this file type, currently no loaders are configured to process this file. See https://webpack.js.org/concepts#loaders
'use client'
import { formatStatus } from "db/utils"

const Test = () => {
return (
<div>
{formatStatus("accepted-paid")}
</div>
);
}
'use client'
import { formatStatus } from "db/utils"

const Test = () => {
return (
<div>
{formatStatus("accepted-paid")}
</div>
);
}
I saw this similar issue but exporting the function from the db/index.ts gives the same error https://github.com/drizzle-team/drizzle-orm/discussions/885 Any ideas? Thanks!
4 replies
TTCTheo's Typesafe Cult
Created by Jim on 11/7/2023 in #questions
enum status table schema
Hey all, using drizzle + pscale + next. I'm making a CRM which has a status field for all records which are predefined (e.g. "verified", "shipped", "complete", "rejected".etc). I'm having trouble getting my head around how to get my zod schema to match up with these possible statuses. In drizzle and mysql there is an ENUM column type, but I would prefer to keep the possible statuses in a table. Does anyone know a good approach for this? Thanks!
4 replies
DTDrizzle Team
Created by Jim on 11/7/2023 in #help
enumerating tables
I have a table of possible 'statuses' for a column in my records table and would like my zod schema to see what statuses are available to set. Is this possible? I'm aware we can create an ENUM column on my records, but it 'feels' better to have a separate table for this. I guess I'm looking for an 'in-between' an enum column and a separate table. Maybe it's possible to predefine/lock-in the statuses in the application layer, is it bad practise? I'm new to db stuff so unsure on the best approach.
6 replies
TTCTheo's Typesafe Cult
Created by Jim on 11/3/2023 in #questions
Workflow: use production database for preview builds?
Hey all, creating a CRM using planetscale and just adding some env variables to vercel and notice there are [development, preview, production] toggles for the keys. what is the common practise for a 'preview' environment, should i show my production database or the development database? Both (and determine env vars based on my own logic)?
5 replies
TTCTheo's Typesafe Cult
Created by Jim on 11/3/2023 in #questions
drizzle + planetscale branching workflow
finding it hard to find any repos that automate merging of planetscale development branches to production on git merges, that uses drizzle. any pointers? I've not used github actions before so it would be nice to see examples of this before diving in.
3 replies
DTDrizzle Team
Created by Jim on 9/26/2023 in #help
id on transaction conflict
hey all, using mysql (planetscale). when a transaction conflict happens over an insert with the same 'serial' id i believe that mysql will resolve this by changing one of the inserts to the next available. my concern is that within a transaction i want to reference the returned id to use as a reference. will this id match the 'new' id after the conflict has resolved, or will it reference the original? here is some code to better explain maybe:
async function test() {
await db.transaction(async (tx) => {
const insertRecord = await tx.insert(record).values({
foo: "bar",
});

await tx
.insert(user)
.values({
foo: "bar",
recordId: parseInt(insertRecord.insertId) // is insertId ok on conflict?
});
});
}
async function test() {
await db.transaction(async (tx) => {
const insertRecord = await tx.insert(record).values({
foo: "bar",
});

await tx
.insert(user)
.values({
foo: "bar",
recordId: parseInt(insertRecord.insertId) // is insertId ok on conflict?
});
});
}
This is my first time trying to make something production ready so want to make sure all situations are covered, any help is appreciated thank you!
5 replies
TTCTheo's Typesafe Cult
Created by Jim on 9/25/2023 in #questions
data branching planetscale
new to database stuff, im about to seed a database for a developer environment to test on. when im happy i was thinking i could then just merge the schema to production (without the data), looking at the docs there is a databranching feature https://planetscale.com/docs/concepts/data-branching but this is premium. I guess the usual approach is to make a separate database, but this is also another cost. Wondering how any of you planetscale users work in development?
4 replies
DTDrizzle Team
Created by Jim on 9/23/2023 in #help
Help understanding relations
Hey all, just trying to get my head around relations. In this example we have a one-to-many relationship:
export const users = mysqlTable("users", {
id: serial("id").primaryKey(),
username: varchar("username", { length: 120 }),
});

export const blocks = mysqlTable("blocks", {
id: serial("id").primaryKey(),
block_type: int("type"),
user_id: int("user_id"),
});

export const usersRelations = relations(users, ({ many }) => ({
pagebuilder: many(blocks),
}));

(async () => {
const user = await db.query.users.findFirst({
where: eq(users.username, "johnsmith"),
with: {
pagebuilder: true,
},
});
// user.pagebuilder[0].block_type ☑️
})();
export const users = mysqlTable("users", {
id: serial("id").primaryKey(),
username: varchar("username", { length: 120 }),
});

export const blocks = mysqlTable("blocks", {
id: serial("id").primaryKey(),
block_type: int("type"),
user_id: int("user_id"),
});

export const usersRelations = relations(users, ({ many }) => ({
pagebuilder: many(blocks),
}));

(async () => {
const user = await db.query.users.findFirst({
where: eq(users.username, "johnsmith"),
with: {
pagebuilder: true,
},
});
// user.pagebuilder[0].block_type ☑️
})();
How is the query 'joining' the tables when I havent defined which columns are related. Or in this case do 'all' block rows get attached?
3 replies
DTDrizzle Team
Created by Jim on 9/4/2023 in #help
Type error inserting 'new Date()' into 'time' type schema.
Driving me up the wall this a bit. my schema
import { mysqlTable, serial, varchar, float, time } from 'drizzle-orm/mysql-core';

export const metalPrices = mysqlTable('metals', {
id: serial('id'),
metal: varchar('metal', { length: 5 }).primaryKey(),
value: float('value'),
time: time('time'),
});
import { mysqlTable, serial, varchar, float, time } from 'drizzle-orm/mysql-core';

export const metalPrices = mysqlTable('metals', {
id: serial('id'),
metal: varchar('metal', { length: 5 }).primaryKey(),
value: float('value'),
time: time('time'),
});
my insert
await db.insert(metalPrices).values({
metal: "FOO",
time: Date.now(), // type error when adding this line
});
await db.insert(metalPrices).values({
metal: "FOO",
time: Date.now(), // type error when adding this line
});
type error:
The expected type comes from property 'time' which is declared here on type '{ metal: string | SQL<unknown> | Placeholder<string, any>; value?: number | SQL<unknown> | Placeholder<string, any> | null | undefined; id?: number | SQL<unknown> | Placeholder<...> | undefined; time?: string | ... 3 more ... | undefined; }'
The expected type comes from property 'time' which is declared here on type '{ metal: string | SQL<unknown> | Placeholder<string, any>; value?: number | SQL<unknown> | Placeholder<string, any> | null | undefined; id?: number | SQL<unknown> | Placeholder<...> | undefined; time?: string | ... 3 more ... | undefined; }'
🫠 Any ideas?
11 replies
TTCTheo's Typesafe Cult
Created by Jim on 8/20/2023 in #questions
Get image size using UploadThing
Hey all, trying to get the image size of a file when a client uploads. Any recommendations? I'm thinking it could be done client size, before the file gets uploaded then meta info passed to uploadthing. Not sure how to do this 'before' the upload when using the UploadThing API. Theres onUploadProgress onClientUploadComplete and onUploadError, but no sort of preflight callback where I can read the file's buffer. Another option could be server side, once the file is uploaded, call a server function which gets passed the uploadthing url and does a HTTP get to work out the size. The aim is to have the file size in a metadata object that goes along with the url when it's added to a database. So yeah, how can I grab the file buffer before uploading? Thanks!
3 replies
TTCTheo's Typesafe Cult
Created by Jim on 8/6/2023 in #questions
Tailwind Theming using CSS variables
Hey all, using shadcn for styling and they use the css variables like so, recommended by tailwind. https://tailwindcss.com/docs/customizing-colors#using-css-variables
@tailwind base;
@tailwind components;
@tailwind utilities;

@layer base {
:root {
--background: 0 0% 100%;
--foreground: 240 10% 3.9%;

--muted: 240 4.8% 95.9%;
--muted-foreground: 240 3.8% 46.1%;

--popover: 0 0% 100%;
--popover-foreground: 240 10% 3.9%;

--card: 0 0% 100%;
--card-foreground: 240 10% 3.9%;

[...]


@tailwind base;
@tailwind components;
@tailwind utilities;

@layer base {
:root {
--background: 0 0% 100%;
--foreground: 240 10% 3.9%;

--muted: 240 4.8% 95.9%;
--muted-foreground: 240 3.8% 46.1%;

--popover: 0 0% 100%;
--popover-foreground: 240 10% 3.9%;

--card: 0 0% 100%;
--card-foreground: 240 10% 3.9%;

[...]


I'm just wondering, is there a way to use already declared tailwind colours here? Or maybe a VScode plugin that would allow me to convert hex to hsl quickly? I just figured you can just punch in the rgb values too which probably will work ok for me as I can just pick out the tailwind colours I want from my browser I found this which is probably the same issue: https://github.com/shadcn-ui/ui/issues/199 But yeah in the time it's taken to look all that up I could have just converted the colours I need to HSL 😪 I put a lot of effort into being lazy
8 replies
DTDrizzle Team
Created by Jim on 6/26/2023 in #help
TypeError when adding subquery to `drizzle.insert.values`
Hey all, I'm a bit new to drizzle so not sure if this is even possible or if I'm just doing something wrong. I'm getting type errors when using a subquery to select for companyId based on a company name:
return await db.insert(widgetItem).values({
name: "New Widget",
companyId: db // using subquery
.select({ id: company.id })
.from(company)
.where(eq(
company.name,
userCompany
)),
typeId: widgetTypeId,
slug: generateSlug(),
});
return await db.insert(widgetItem).values({
name: "New Widget",
companyId: db // using subquery
.select({ id: company.id })
.from(company)
.where(eq(
company.name,
userCompany
)),
typeId: widgetTypeId,
slug: generateSlug(),
});
but this checks out ok:
return await db.insert(widgetItem).values({
name: "New Widget",
companyId: BigInt(2), // hardcoded
typeId: widgetTypeId,
slug: generateSlug(),
});
return await db.insert(widgetItem).values({
name: "New Widget",
companyId: BigInt(2), // hardcoded
typeId: widgetTypeId,
slug: generateSlug(),
});
wondering if this is possible or will I have to send multiple queries? Cheers
6 replies
TTCTheo's Typesafe Cult
Created by Jim on 6/23/2023 in #questions
Migrating next app from [client -> ts-query + clerk -> supabase] to planetscale
Hi all, I'm migrating an app which used supabase's RLS with clerk's JWT to directly connect clients to a database. I now would like to use planetscale but getting stun locked with the different options I have. Ideally I want to keep data management on the client side (with tanstack query) so would it be best to just create a bunch of Route Handlers? There are about 9 queries and mutations. Any input will be great thank you!
4 replies
TTCTheo's Typesafe Cult
Created by Jim on 6/6/2023 in #questions
Vercel Postgres Backup (pg_dump)
Hey all, looking to use vercel's postgres storage in production, but need to be able to backup the tables on a schedule somehow. First I'm unsure how to pg_dump from a serverless function, looks like ill have to use a process, then stream it to an s3. This looks promising but it's built specifically for aws lambda. https://github.com/jameshy/pgdump-aws-lambda Anyone done this already?
1 replies