need to change schema to reference another instance

137 Replies
nikivi
nikiviOP2y ago
assume you meant this was not sure
Andrii Sherman
oh if you are referncing the same table
nikivi
nikiviOP2y ago
yes its a topic a user has made im new to table design so maybe im going about it wrong way but i have users and i have topics users create topics each topic has user_id as owner user can have many topics topic can have a parent which is another topic last part lets you create hierarchies
export const users = sqliteTable("users", {
id: integer("id").primaryKey(),
name: text("name").notNull(),
email: text("email").notNull(),
createdAt: integer("created_at", { mode: "timestamp" }).default(
sql`(strftime('%s', 'now'))`
),
})
export const insertUserSchema = createInsertSchema(users)
export const selectUserSchema = createSelectSchema(users)

export const topics = sqliteTable("topics", {
id: integer("id").primaryKey(),
name: text("name").notNull(),
content: text("content").notNull(), // markdown of the topic TODO: make notes/links separate entity
parent: text("parent_id").references((): AnySQLiteTable => topics.id),
ownerId: integer("owner_id")
.notNull()
.references(() => users.id),
createdAt: integer("created_at", { mode: "timestamp" }).default(
sql`(strftime('%s', 'now'))`
),
updateAt: integer("updated_at", { mode: "timestamp" }).default(
sql`(strftime('%s', 'now'))`
),
})
export const users = sqliteTable("users", {
id: integer("id").primaryKey(),
name: text("name").notNull(),
email: text("email").notNull(),
createdAt: integer("created_at", { mode: "timestamp" }).default(
sql`(strftime('%s', 'now'))`
),
})
export const insertUserSchema = createInsertSchema(users)
export const selectUserSchema = createSelectSchema(users)

export const topics = sqliteTable("topics", {
id: integer("id").primaryKey(),
name: text("name").notNull(),
content: text("content").notNull(), // markdown of the topic TODO: make notes/links separate entity
parent: text("parent_id").references((): AnySQLiteTable => topics.id),
ownerId: integer("owner_id")
.notNull()
.references(() => users.id),
createdAt: integer("created_at", { mode: "timestamp" }).default(
sql`(strftime('%s', 'now'))`
),
updateAt: integer("updated_at", { mode: "timestamp" }).default(
sql`(strftime('%s', 'now'))`
),
})
what i have now
Andrii Sherman
sorry my bad change to AnySQLiteColumn instead of Table
nikivi
nikiviOP2y ago
nikivi
nikiviOP2y ago
gpt-4 said this ok nice no errors runnning migration
nikivi
nikiviOP2y ago
nikivi
nikiviOP2y ago
still hitting this issue i can get around it but its so strange
Andrii Sherman
you have old drizzle-kit
nikivi
nikiviOP2y ago
GitHub
learn-anything/db/package.json at main · learn-anything/learn-anyth...
Organize world's knowledge, explore connections and curate learning paths - learn-anything/db/package.json at main · learn-anything/learn-anything
nikivi
nikiviOP2y ago
but that cant be
Andrii Sherman
you need to upgrade it
nikivi
nikiviOP2y ago
its all latest modules
nikivi
nikiviOP2y ago
nikivi
nikiviOP2y ago
unless pnpm messes up
Andrii Sherman
no it's not latest
nikivi
nikiviOP2y ago
ah my bad i update with ncu -u and it did not see that
nikivi
nikiviOP2y ago
nikivi
nikiviOP2y ago
what was the command to db-push to turso or wait if i run server it would do it for me i think
nikivi
nikiviOP2y ago
nikivi
nikiviOP2y ago
GitHub
GitHub - learn-anything/learn-anything: Organize world's knowledge,...
Organize world's knowledge, explore connections and curate learning paths - GitHub - learn-anything/learn-anything: Organize world's knowledge, explore connections and curate learning paths
nikivi
nikiviOP2y ago
i dont get why it would run sql in node_modules is it the schema
Andrii Sherman
you didn;t fix a migration here
Andrii Sherman
It's all explained inside. What was not clear for you? Would love to improve it in a way it would be more clear
nikivi
nikiviOP2y ago
nikivi
nikiviOP2y ago
i would love if it said it here beware i couldn't make migration work do it yourself and show clear steps 1. 2.
Andrii Sherman
sure! great thanks we will add all of that in this output
nikivi
nikiviOP2y ago
does that mean i find query to make it work then edit migration manually ALTER TABLE topics ADD parent_id text REFERENCES topics(id);--> statement-breakpoint is it a hard thing to automate? reading on alter table now 🙂 in theory its adding a new column to db so there are no migrations of data even
Andrii Sherman
as you can see we have this statement generated 1 statements that was not generated - was adding FK to an existing column it's not possible in SQLite in 1 statement
nikivi
nikiviOP2y ago
can drizzle create 2 migrations and still automate it?
nikivi
nikiviOP2y ago
nikivi
nikiviOP2y ago
nikivi
nikiviOP2y ago
so i need to create new migration 0002_hot_rabbit_cooper.sql or something
nikivi
nikiviOP2y ago
nikivi
nikiviOP2y ago
edit this too i guess
Andrii Sherman
it;s not a valid syntax you can't add FK on existin column on SQLite Sure, it can. We do it in the push command. The case here is that you need to make more than two statements: You need to rename a table. Create a new table. Move all the data. Remove the old table. We don't want to be responsible for any of your data movements and manipulation. This is fully your responsibility for your production data. That's why we added links to blog posts for reference. As an improvement, we can generate those statements but still comment them out. In this case, you'll decide what to do with them
nikivi
nikiviOP2y ago
makes sense, ok will try make migration manually is there doc on how to create a migration of drizzle form with editing of _journal.json
nikivi
nikiviOP2y ago
only mentions generating migrations from changing schema it seems
Andrii Sherman
so you can modify generated sql if it wasn't applied to a database
nikivi
nikiviOP2y ago
oh
Andrii Sherman
the flow is 1. You generate an sql file 2. If you see any warnings inside - you can fix them inside this sql file 3. apply migrations If you still don't have any data inside your db you can remove all tables and migration folder and generate it from scratch and then apply
nikivi
nikiviOP2y ago
nikivi
nikiviOP2y ago
for some reason it says generated sql is wrong
Andrii Sherman
if you have data and didn't apply last migration - just modify existing sql file
nikivi
nikiviOP2y ago
feel like i do have to study sql in depth after all 😄
nikivi
nikiviOP2y ago
EdgeDB | The post-SQL era has arrived
EdgeDB is an open-source database designed as a spiritual successor to SQL and the relational paradigm.
nikivi
nikiviOP2y ago
wanted to avoid using it to keep data at edge but sql seems tricky
nikivi
nikiviOP2y ago
nikivi
nikiviOP2y ago
i thought i could do this extend generated code with adding of foreign key but you say thats wrong sql
Andrii Sherman
you can't add FK to an existing column
nikivi
nikiviOP2y ago
SQLite: Foreign Keys
This SQLite tutorial explains how to use Foreign Keys in SQLite with syntax and examples. A foreign key is a way to enforce referential integrity within your SQLite database. A foreign key means that values in one table must also appear in another table.
nikivi
nikiviOP2y ago
You need to rename a table.
Create a new table.
Move all the data.
Remove the old table.
You need to rename a table.
Create a new table.
Move all the data.
Remove the old table.
so i need to do this manually
nikivi
nikiviOP2y ago
or somehow encode it in sql and put in migration
Andrii Sherman
you need to write an sql script
nikivi
nikiviOP2y ago
oh i see
Andrii Sherman
inside generated sql file
nikivi
nikiviOP2y ago
ok so sql just without add foreign key ok thanks
Andrii Sherman
actually I would recommend you to avoid FK's in SQLite
nikivi
nikiviOP2y ago
how would i do the use case i need
Andrii Sherman
it super hard to work with them
nikivi
nikiviOP2y ago
i need
nikivi
nikiviOP2y ago
somehow encode who is parent of who
Andrii Sherman
you don't need to have FK to make joins work
nikivi
nikiviOP2y ago
i only have one table topics what join
Andrii Sherman
you can just store parent as integer without reference
nikivi
nikiviOP2y ago
the id
Andrii Sherman
it will work the same
nikivi
nikiviOP2y ago
ah smart
Andrii Sherman
and you don't need to handle such migrations just add index in this field and you are good to go
nikivi
nikiviOP2y ago
nikivi
nikiviOP2y ago
nikivi
nikiviOP2y ago
so strange im just modifying the turso drizzle example i just changed schema now its 502
Andrii Sherman
this error comes from turso do you have a data in your database?
nikivi
nikiviOP2y ago
i do should i ask on turso discord?
Andrii Sherman
as you can see
nikivi
nikiviOP2y ago
i guess i need to migrate data somehow
Andrii Sherman
you had a ref you can't remove a FK from an existin column in SQLite
nikivi
nikiviOP2y ago
ok
Andrii Sherman
and you can't add a new one
nikivi
nikiviOP2y ago
wiping migrations
Andrii Sherman
and you can't change a type is SQLite
nikivi
nikiviOP2y ago
fk are no go in sqlite
Andrii Sherman
SQLite can only create and drop tables and add columns evertyhing else need to drop a table a create a new one those are SQLite limitations
nikivi
nikiviOP2y ago
nikivi
nikiviOP2y ago
was reading about it like why have fk it seems they help with joins
Andrii Sherman
I would suggest to just remove all migrations and tables in you db and generate first one from scratch. It will work well you can have joins without fk just add index to a column fk constraints are good to check that you've added an id
nikivi
nikiviOP2y ago
did this and wiped migrations folder
nikivi
nikiviOP2y ago
nikivi
nikiviOP2y ago
nikivi
nikiviOP2y ago
it worked, amazing ah nice so its literally just a type checker thing under hood its an integer with id of a table item
Andrii Sherman
Sorry internet went down To check that you’ve added Id that exists in referenced table Yeah So fk constraint will block you if you want to add Id:15, having just 10 topics Everything else can work the same without fk
nikivi
nikiviOP2y ago
so now its on me to make it valid ok fair enough doesnt seem too bad
nikivi
nikiviOP2y ago
nikivi
nikiviOP2y ago
can i make zod somehow validate if i am passing wrong parent_id
app.post("/topics", zValidator("json", insertTopicSchema), async (ctx) => {
const data = ctx.req.valid("json")
const post = await db.insert(topics).values(data).returning().get()
return ctx.json(insertPostResponse.parse(post))
})
app.post("/topics", zValidator("json", insertTopicSchema), async (ctx) => {
const data = ctx.req.valid("json")
const post = await db.insert(topics).values(data).returning().get()
return ctx.json(insertPostResponse.parse(post))
})
im trying to modify this i guess so it adds a parent
app.post("/topics", zValidator("json", insertTopicSchema), async (ctx) => {
const data = ctx.req.valid("json")
app.post("/topics", zValidator("json", insertTopicSchema), async (ctx) => {
const data = ctx.req.valid("json")
i used hono briefly before i assume ctx.req.valid checks if json satisfies insertTopicSchema
nikivi
nikiviOP2y ago
nikivi
nikiviOP2y ago
db.insert(topics).values(data)
db.insert(topics).values(data)
im not sure if it wants camelCase for values or as they exist in db
nikivi
nikiviOP2y ago
it just uses name
nikivi
nikiviOP2y ago
before drizzle you used prisma I imagine?
Andrii Sherman
https://www.npmjs.com/package/drizzle-zod I tried to use prisma 2 yrs ago. Didn't like it. So we wrote Drizzle for ourselves and then made it publicly accessible
nikivi
nikiviOP2y ago
nikivi
nikiviOP2y ago
i tried both
nikivi
nikiviOP2y ago
nikivi
nikiviOP2y ago
based on schema the rest should have default values name, content, ownerId i fill parent should set to null if not passed or i guess it needs excplicit null
nikivi
nikiviOP2y ago
nikivi
nikiviOP2y ago
yea it should be this mm
Andrii Sherman
I'm not sure it can be optional it should let me check looking at you error it says that you've sent ownerId as string and not number
nikivi
nikiviOP2y ago
Andrii Sherman
nothing with drizzle
nikivi
nikiviOP2y ago
i dont get logs here
Andrii Sherman
you can add {logger:true}
nikivi
nikiviOP2y ago
Andrii Sherman
https://orm.drizzle.team/docs/goodies#logging It's not a drizzle error so you should check your code
nikivi
nikiviOP2y ago
nikivi
nikiviOP2y ago
ok yea it does not reach this ednpoint it seems will ask in hono discord its weird thanks ❤️ like it actually sees the route
nikivi
nikiviOP2y ago
nikivi
nikiviOP2y ago
app.post("/topics", zValidator("json", insertTopicSchema), async (ctx) => {
nikivi
nikiviOP2y ago
nikivi
nikiviOP2y ago
ohh ok solved it
nikivi
nikiviOP2y ago
nikivi
nikiviOP2y ago
nikivi
nikiviOP2y ago
whats the drizzle way of doing seeding of data i guess i can't directly run that file
nikivi
nikiviOP2y ago
nikivi
nikiviOP2y ago
i can do something like this and comment it on/off
nikivi
nikiviOP2y ago
strange
nikivi
nikiviOP2y ago
like it should have that row there no errors too its connected to prod turso
nikivi
nikiviOP2y ago
i guess as i dont see insert query here
nikivi
nikiviOP2y ago
it ignored it
async function main() {
await migrate(db, {
migrationsFolder: "./migrations",
})

await seedDb()
async function main() {
await migrate(db, {
migrationsFolder: "./migrations",
})

await seedDb()
tried with await too, does not apply yea it even runs the function, no idea ohh i have to commit
await db
.insert(users)
.values({ name: "Nikita", email: "[email protected]" })
.returning()
.get()
await db
.insert(users)
.values({ name: "Nikita", email: "[email protected]" })
.returning()
.get()
ok doing this as it was in example ok yea that worked
Want results from more Discord servers?
Add your server