Astro Comments - Answer Overflow

Astro

Posts Comments

CDCloudflare Developers

•Created by cloudkite on 2/11/2025 in #vectorize

Bump for visibility, also getting this.

*** important to note im still able to access other bindings without issue, such as D1

24 replies

CDCloudflare Developers

•Created by cloudkite on 2/11/2025 in #vectorize

Bump for visibility, also getting this.

anywhere i use vectorize i see this error

24 replies

CDCloudflare Developers

•Created by cloudkite on 2/11/2025 in #vectorize

Bump for visibility, also getting this.

Very weird behavior - but im now experiencing the 10000 error everywhere in my app in local dev mode, not just on that code.

24 replies

CDCloudflare Developers

•Created by cloudkite on 2/11/2025 in #vectorize

Bump for visibility, also getting this.

The line erroring here is

await vectorize.insert([{ id: `${videoId}`, values: combinedEmb }]);

await vectorize.insert([{ id: `${videoId}`, values: combinedEmb }]);

24 replies

CDCloudflare Developers

•Created by cloudkite on 2/11/2025 in #vectorize

Bump for visibility, also getting this.

And here are the console logs when executing this code (showing where in the process it errors out)

Action function started
ProxyStub { name: 'VectorizeIndexImpl', poisoned: false }
Initialized database and vectorize
Verified user
Got form data
No early returns needed
Got new embeddings
combined embeddings
Error: VECTOR_INSERT_ERROR (code = 10000): Authentication error
    at processTicksAndRejections (node:internal/process/task_queues:95:5)
    at action (/home/astro/Development/deployed/neuraly/app/routes/videos.upload.tsx:145:3)
    at Object.callRouteAction (/home/astro/Development/deployed/neuraly/node_modules/@remix-run/server-runtime/dist/data.js:36:16)
    at /home/astro/Development/deployed/neuraly/node_modules/@remix-run/router/router.ts:4929:19
    at callLoaderOrAction (/home/astro/Development/deployed/neuraly/node_modules/@remix-run/router/router.ts:4993:16)
    at async Promise.all (index 2)
    at /home/astro/Development/deployed/neuraly/node_modules/@remix-run/server-runtime/dist/single-fetch.js:44:19
    at callDataStrategyImpl (/home/astro/Development/deployed/neuraly/node_modules/@remix-run/router/router.ts:4865:17)
    at callDataStrategy (/home/astro/Development/deployed/neuraly/node_modules/@remix-run/router/router.ts:4022:19)
    at submit (/home/astro/Development/deployed/neuraly/node_modules/@remix-run/router/router.ts:3785:21) {
  [cause]: Error: Authentication error
      at VectorizeIndexImpl._send (cloudflare-internal:vectorize-api:167:28)
      at async VectorizeIndexImpl.insert (cloudflare-internal:vectorize-api:91:21)
      at async #fetch (file:///home/astro/Development/deployed/neuraly/node_modules/miniflare/dist/src/workers/core/entry.worker.js:828:18)
      at async ProxyServer.fetch (file:///home/astro/Development/deployed/neuraly/node_modules/miniflare/dist/src/workers/core/entry.worker.js:734:14) {
    [cause]: undefined
  }
}

Action function started
ProxyStub { name: 'VectorizeIndexImpl', poisoned: false }
Initialized database and vectorize
Verified user
Got form data
No early returns needed
Got new embeddings
combined embeddings
Error: VECTOR_INSERT_ERROR (code = 10000): Authentication error
    at processTicksAndRejections (node:internal/process/task_queues:95:5)
    at action (/home/astro/Development/deployed/neuraly/app/routes/videos.upload.tsx:145:3)
    at Object.callRouteAction (/home/astro/Development/deployed/neuraly/node_modules/@remix-run/server-runtime/dist/data.js:36:16)
    at /home/astro/Development/deployed/neuraly/node_modules/@remix-run/router/router.ts:4929:19
    at callLoaderOrAction (/home/astro/Development/deployed/neuraly/node_modules/@remix-run/router/router.ts:4993:16)
    at async Promise.all (index 2)
    at /home/astro/Development/deployed/neuraly/node_modules/@remix-run/server-runtime/dist/single-fetch.js:44:19
    at callDataStrategyImpl (/home/astro/Development/deployed/neuraly/node_modules/@remix-run/router/router.ts:4865:17)
    at callDataStrategy (/home/astro/Development/deployed/neuraly/node_modules/@remix-run/router/router.ts:4022:19)
    at submit (/home/astro/Development/deployed/neuraly/node_modules/@remix-run/router/router.ts:3785:21) {
  [cause]: Error: Authentication error
      at VectorizeIndexImpl._send (cloudflare-internal:vectorize-api:167:28)
      at async VectorizeIndexImpl.insert (cloudflare-internal:vectorize-api:91:21)
      at async #fetch (file:///home/astro/Development/deployed/neuraly/node_modules/miniflare/dist/src/workers/core/entry.worker.js:828:18)
      at async ProxyServer.fetch (file:///home/astro/Development/deployed/neuraly/node_modules/miniflare/dist/src/workers/core/entry.worker.js:734:14) {
    [cause]: undefined
  }
}

24 replies

CDCloudflare Developers

•Created by cloudkite on 2/11/2025 in #vectorize

Bump for visibility, also getting this.

Here's the code thats consistently causing this error

export const action: ActionFunction = async ({ request, context }) => {
  console.log("Action function started");

  const db = drizzle(context.cloudflare.env.database);
  const vectorize =
    context.cloudflare.env.environment == "local"
      ? context.cloudflare.env.localvectorize
      : context.cloudflare.env.vectorize;

  console.log(vectorize)

  console.log("Initialized database and vectorize");

  const { user, newToken } = await verifyUser(context, request);

  console.log("Verified user");

  const formData = await request.formData();
  const videoId = formData.get("videoId");

  console.log("Got form data");
  if (!videoId || !title || !thumbnail || !duration) {
    console.warn(`Missing required field(s)`);
    return AuthorizedResponse(
      { success: false, message: "Missing required field(s)" },
      newToken
    );
  }
  console.log("No early returns needed");

  // Embedding stuff

  const WEIGHTS = {
    thumbnails: 0.25,
    text: {
      title: 0.5,
      description: 0.25,
    },
  };

  const [titleEmb, descEmb, thumbnailEmb] = await Promise.all([...]);

  console.log("Got new embeddings");

  const combinedEmb = combineVideoEmbeddings(...);
  console.log("combined embeddings");

  await vectorize.insert([{ id: `${videoId}`, values: combinedEmb }]);

  console.log("inserted vector");
  await handleTopicVectorOperations(
    vectorize,
    db,
    null,
    topic as string | null,
    combinedEmb
  );
  console.log("handled vector operations");

  // Database stuff
  await db.insert(videos).values({...});

  console.log("inserted into db");
  // Video processor stuff
  await sendToSQS(context, `${videoId}`);

  console.log("sent to queue");
  await startEC2Instance(context);

  console.log("started ec2 instance");

  return AuthorizedResponse(
    { success: true, message: "Uploaded video" },
    newToken
  );
};

export const action: ActionFunction = async ({ request, context }) => {
  console.log("Action function started");

  const db = drizzle(context.cloudflare.env.database);
  const vectorize =
    context.cloudflare.env.environment == "local"
      ? context.cloudflare.env.localvectorize
      : context.cloudflare.env.vectorize;

  console.log(vectorize)

  console.log("Initialized database and vectorize");

  const { user, newToken } = await verifyUser(context, request);

  console.log("Verified user");

  const formData = await request.formData();
  const videoId = formData.get("videoId");

  console.log("Got form data");
  if (!videoId || !title || !thumbnail || !duration) {
    console.warn(`Missing required field(s)`);
    return AuthorizedResponse(
      { success: false, message: "Missing required field(s)" },
      newToken
    );
  }
  console.log("No early returns needed");

  // Embedding stuff

  const WEIGHTS = {
    thumbnails: 0.25,
    text: {
      title: 0.5,
      description: 0.25,
    },
  };

  const [titleEmb, descEmb, thumbnailEmb] = await Promise.all([...]);

  console.log("Got new embeddings");

  const combinedEmb = combineVideoEmbeddings(...);
  console.log("combined embeddings");

  await vectorize.insert([{ id: `${videoId}`, values: combinedEmb }]);

  console.log("inserted vector");
  await handleTopicVectorOperations(
    vectorize,
    db,
    null,
    topic as string | null,
    combinedEmb
  );
  console.log("handled vector operations");

  // Database stuff
  await db.insert(videos).values({...});

  console.log("inserted into db");
  // Video processor stuff
  await sendToSQS(context, `${videoId}`);

  console.log("sent to queue");
  await startEC2Instance(context);

  console.log("started ec2 instance");

  return AuthorizedResponse(
    { success: true, message: "Uploaded video" },
    newToken
  );
};

24 replies

CDCloudflare Developers

•Created by cloudkite on 2/11/2025 in #vectorize

Bump for visibility, also getting this.

Update on this behavior: a couple days ago i started seeing some address already in use error that would timeout my Remix app being caused by wrangler.. seeing this, i logged out via wrangler logout, then realized i couldnt actually log back in because i code via SSH, and clicking the oauth url to sign in will redirect to localhost, which is NOT where the CLI was initiated from.. I was able to login via a "CLOUDFLARE_API_TOKEN" env variable on the remote system (not ideal - but technically worked) but now i'm consistently getting the 10000 errors from vectorize for "Authentication error".

24 replies

CDCloudflare Developers

•Created by cloudkite on 2/11/2025 in #vectorize

Bump for visibility, also getting this.

I have a gut feeling it only happens when I first put a vector into the database. Like when i create a vector for the video, then I go to that video's page, which should query vectorize for similar videos (for recommendations) is when this happens. I'm testing my whole embedding flow right now so i'll give this a bit more testing to hopefully get to the bottom of it

24 replies

CDCloudflare Developers

•Created by cloudkite on 2/11/2025 in #vectorize

Bump for visibility, also getting this.

I see it in dev mode yes. I notice it mostly when Vite hot refreshes after I make a change (Remix app - so this will cause my loader function to refresh data)

24 replies

CDCloudflare Developers

•Created by cloudkite on 2/11/2025 in #vectorize

Bump for visibility, also getting this.

Mine is very similar, instead of querying for a search this is for recommendations. Same error message as cloudkite is seeing.

      // Grab the appropriate vectorize binding (one for local dev, one for production)
      const vectorize =
          context.cloudflare.env.environment == "local"
            ? context.cloudflare.env.localvectorize
            : context.cloudflare.env.vectorize;

      const videoVectors = await vectorize.getByIds([videoId]);
      const videoVector = videoVectors?.[0];

      const results = await vectorize.query(videoVector.values, {
        topK: 50,
      });

      // Grab the appropriate vectorize binding (one for local dev, one for production)
      const vectorize =
          context.cloudflare.env.environment == "local"
            ? context.cloudflare.env.localvectorize
            : context.cloudflare.env.vectorize;

      const videoVectors = await vectorize.getByIds([videoId]);
      const videoVector = videoVectors?.[0];

      const results = await vectorize.query(videoVector.values, {
        topK: 50,
      });

Wrangler.toml:

[[vectorize]]
binding = "localvectorize"
index_name = "neuraly-videos-local"

[[vectorize]]
binding = "vectorize"
index_name = "neuraly-videos"

[[vectorize]]
binding = "localvectorize"
index_name = "neuraly-videos-local"

[[vectorize]]
binding = "vectorize"
index_name = "neuraly-videos"

24 replies

CDCloudflare Developers

•Created by cloudkite on 2/11/2025 in #vectorize

Bump for visibility, also getting this.

Yes i am.. more common in local dev because thats where i spend most of my time.. Im unsure just how noticeable it is on production, but i've definitely seen it a few times

24 replies

CDCloudflare Developers

•Created by cloudkite on 2/11/2025 in #vectorize

Bump for visibility, also getting this.

I also experience this bug intermittently.. It's a hard one to debug. I'm using vectorize via bindings in a Pages/Workers app so auth shouldnt be a problem, although i am loggin in via the wrangler CLI. It seems to happen to me more when i navigate between pages that query vectorize frequently. Possibly a different edge case causing it, but nothing i've been able to debug myself.

24 replies

Gaming

Programming