Cloudflare Developers•3mo ago

Hi, I am having troubble setting up drizzle with the local d1 database, when running `bun drizzle-ki

Hi, I am having troubble setting up drizzle with the local d1 database, when running bun drizzle-kit migrate i receive the following error:

$ bun db:migrate
$ drizzle-kit migrate
No config path provided, using default 'drizzle.config.ts'
Reading config file 'G:\GitHub\BuzzTrip\apps\web\drizzle.config.ts'
Using G:\GitHub\BuzzTrip\apps\web\.wrangler\state\v3\d1\miniflare-D1DatabaseObject\097dd93c38860e13c35b3676a4f13f4950f08ed9c239ec99db5716131b0e2718.sqlite
[⣷] applying migrations...Error: Can't find meta/_journal.json file
    at readMigrationFiles (G:\GitHub\BuzzTrip\node_modules\src\migrator.ts:41:9)
    at migrate (G:\GitHub\BuzzTrip\node_modules\src\libsql\migrator.ts:10:21)
    at migrateFn (G:\GitHub\BuzzTrip\node_modules\drizzle-kit\bin.cjs:72211:18)
    at Object.handler (G:\GitHub\BuzzTrip\node_modules\drizzle-kit\bin.cjs:83593:11)
    at async run (G:\GitHub\BuzzTrip\node_modules\drizzle-kit\bin.cjs:82064:7)
error: script "db:migrate" exited with code 1

$ bun db:migrate
$ drizzle-kit migrate
No config path provided, using default 'drizzle.config.ts'
Reading config file 'G:\GitHub\BuzzTrip\apps\web\drizzle.config.ts'
Using G:\GitHub\BuzzTrip\apps\web\.wrangler\state\v3\d1\miniflare-D1DatabaseObject\097dd93c38860e13c35b3676a4f13f4950f08ed9c239ec99db5716131b0e2718.sqlite
[⣷] applying migrations...Error: Can't find meta/_journal.json file
    at readMigrationFiles (G:\GitHub\BuzzTrip\node_modules\src\migrator.ts:41:9)
    at migrate (G:\GitHub\BuzzTrip\node_modules\src\libsql\migrator.ts:10:21)
    at migrateFn (G:\GitHub\BuzzTrip\node_modules\drizzle-kit\bin.cjs:72211:18)
    at Object.handler (G:\GitHub\BuzzTrip\node_modules\drizzle-kit\bin.cjs:83593:11)
    at async run (G:\GitHub\BuzzTrip\node_modules\drizzle-kit\bin.cjs:82064:7)
error: script "db:migrate" exited with code 1

I attach my drizzle.config.ts and wrangler.toml below. Anyone have any ideas on how i could fix my error?

18 Replies

JacobOP•3mo ago

drizzle.config.ts

import { defineConfig } from "drizzle-kit";
import fs from "node:fs";
import path from "node:path";

function getLocalD1DB() {
  try {
    const basePath = path.resolve(".wrangler");
    const dbFile = fs
      .readdirSync(basePath, { encoding: "utf-8", recursive: true })
      .find((f) => f.endsWith(".sqlite"));

    if (!dbFile) {
      throw new Error(`.sqlite file not found in ${basePath}`);
    }

    const url = path.resolve(basePath, dbFile);
    console.log(`Using ${url}`);
    return `file:${url}`;
  } catch (err) {
    console.log(`Error  ${err}`);
  }
}

export default defineConfig({
  dialect: "sqlite",
  schema: "./app/server/db/schema.ts",
  out: "./drizzle",
  ...(process.env.NODE_ENV === "production"
    ? {
        driver: "d1-http",
        dbCredentials: {
          accountId: process.env.CLOUDFLARE_D1_ACCOUNT_ID,
          databaseId: process.env.DATABASE,
          token: process.env.CLOUDFLARE_D1_API_TOKEN,
        },
      }
    : {
        dbCredentials: {
          url: getLocalD1DB(),
        },
      }),
});

import { defineConfig } from "drizzle-kit";
import fs from "node:fs";
import path from "node:path";

function getLocalD1DB() {
  try {
    const basePath = path.resolve(".wrangler");
    const dbFile = fs
      .readdirSync(basePath, { encoding: "utf-8", recursive: true })
      .find((f) => f.endsWith(".sqlite"));

    if (!dbFile) {
      throw new Error(`.sqlite file not found in ${basePath}`);
    }

    const url = path.resolve(basePath, dbFile);
    console.log(`Using ${url}`);
    return `file:${url}`;
  } catch (err) {
    console.log(`Error  ${err}`);
  }
}

export default defineConfig({
  dialect: "sqlite",
  schema: "./app/server/db/schema.ts",
  out: "./drizzle",
  ...(process.env.NODE_ENV === "production"
    ? {
        driver: "d1-http",
        dbCredentials: {
          accountId: process.env.CLOUDFLARE_D1_ACCOUNT_ID,
          databaseId: process.env.DATABASE,
          token: process.env.CLOUDFLARE_D1_API_TOKEN,
        },
      }
    : {
        dbCredentials: {
          url: getLocalD1DB(),
        },
      }),
});

toml

#:schema node_modules/wrangler/config-schema.json
name = "buzztrip"
compatibility_date = "2024-08-21"
compatibility_flags = ["nodejs_compat"]
pages_build_output_dir = ".vercel/output/static"

[[d1_databases]]
binding = "BUZZTRIP_DATABASE"
database_name = "BuzzTrip-db"
database_id = "<database-id>"
migrations_dir = "drizzle/migrations"

#:schema node_modules/wrangler/config-schema.json
name = "buzztrip"
compatibility_date = "2024-08-21"
compatibility_flags = ["nodejs_compat"]
pages_build_output_dir = ".vercel/output/static"

[[d1_databases]]
binding = "BUZZTRIP_DATABASE"
database_name = "BuzzTrip-db"
database_id = "<database-id>"
migrations_dir = "drizzle/migrations"

Keavon•3mo ago

I want to write a backend accounts API using Workers and D1, but using Rust instead of TypeScript. I see some ORMs listed at https://developers.cloudflare.com/d1/reference/community-projects/ for TS, but I don't see anything Rust related. Are there any I can look into? I'm basically very unsure how to dive into building this tech stack. It would also be nice to take advantage of Rust's ecosystem with libraries like Axum, is that feasible?

James•3mo ago

For those hitting this, and other similar errors, we’ve escalated and are told the team are actively investigating. I am trying to get them to post something more official, but hopefully we’ll see a real resolution soon.

Unknown User•3mo ago

Message Not Public

Max (@rozenmd)•3mo ago

https://www.cloudflarestatus.com/incidents/s8p372y81z28

Cloudflare D1 Issues

lucasp•3mo ago

Hi, I wanted to ask you a question: should I push the database_id to the repository? If not, how do I put it as an environment variable? I can't find it in the documentation.

[[env.dev.d1_databases]]
binding = "DB"
database_name = "d-dev"
database_id = "67d41931-140c-418b-xxxx-xxxxx"

[[env.dev.d1_databases]]
binding = "DB"
database_name = "d-dev"
database_id = "67d41931-140c-418b-xxxx-xxxxx"

Cyb3r-Jak3•3mo ago

The ID is safe to commit

lucasp•3mo ago

ty i thought it was sesitive data

Cyb3r-Jak3•3mo ago

Everything in the wrangler.toml is safe to commit. The only thing would be environment variables which are sensitive. Those should be stored as secrets

jasperagrante•3mo ago

Hi there! I'm doing a usage counter with max cap. What would be the best way to increment the counter that I can detect if it has reached the threshold without going over?

Cyb3r-Jak3•3mo ago

I would look at durable objects

jasperagrante•3mo ago

Thanks! I was try to avoid durable objects because it will double the work (durable object logic, durable object management). I was able to to it with sqlite trigger with RAISE Unforunately I've experience this same error as well. Had to execute trigger creation manually via wrangler.

Max (@rozenmd)•3mo ago

We've closed the incident. Here's what happened: we rolled out a fix for rare cases where D1 DBs weren't booting up successfully last week. This fix uncovered a bug from an update we released months ago, and caused a small percentage of requests to fail (roughly 1 error for every 22.5k requests) and return an error of Internal error in D1 storage caused object to be reset. We have rolled back the update from months ago, and the errors have stopped as of 2024-09-04 21:55 UTC

Mozzy•3mo ago

🙏

xiaolong01•3mo ago

Hey Cloudflare team, I saw that the current limit is 10GB per database, but we need to store data for about 200 million users, which would require something like 200GB. With the 10GB limit, we can only fit data for around 10-20 million users. Are there any plans to increase the limit, or is there any way to upgrade to larger storage? Just trying to figure out the best approach for our project. Thanks in advance!🙏

Max (@rozenmd)•3mo ago

If you're set on storing that data in a single database, we'd recommend using Hyperdrive instead

Unknown User•3mo ago

Message Not Public

Max (@rozenmd)•3mo ago

D1 is built around Cloudflare's network, which is really efficient at loading lots of small files really quickly - we'd offer bigger DBs but the cold starts would be brutal

Gaming

Programming

Hi, I am having troubble setting up drizzle with the local d1 database, when running `bun drizzle-ki