Hi, I am having issue in inserting
Hi, I am having issue in inserting vector with correct vector format, anyone else experiencing same issue, there is no error but it is returning null. Any idea
5 Replies
Hi @Utkarsh Saxena. Thanks for reaching out! Could you please share some details of the issue you are facing?
@garvit
this is the code block:
const modelResp = await env.AI.run('@cf/baai/bge-small-en-v1.5', {
text: limitedHeadings.map(heading => heading.content),
});
//console.log("modelResp after inserting headings content for vectorizing : ", modelResp)
const vectors: Array<VectorizeVector> = []
if (modelResp && modelResp.data) {
modelResp.data.forEach((vector: number[], index: number) => {
if (headings[index] && vector) {
vectors.push({ id: headings[index].id, values: vector)
}
})
console.log("inserting vectors after processing modelResp : ", modelResp.data.length, " vectors length: ", vectors.length)
} else {
console.error("empty modelResp", modelResp, " headings sent to model: ", limitedHeadings);
}
//console.log("inserting vectors :", vectors)
let vectorsInserted: VectorInserted[] = [];
if (vectors.length > 0) {
console.log("vectors to be inserted : ", vectors)
const res = await env.VECTORIZE_INDEX_2.insert(vectors);
console.log("inserted vectors :", res.count, " vectors length : ", vectors.length)
if (res && res.ids) {
for (let i = 0; i < res.ids.length; i++) {
if (headings[i] && res.ids[i]) {
vectorsInserted.push({ content_id: headings[i].content_id, vid2: res.ids[i] })
}
}
}
}
in this I am getting proper vector dimension from model, and i also checked vector array which i am inserting attached in image, still getting null after insert method without any exception or error.
I was able to insert vectors by making some tweaks to the code you have provided (attached as file)
Summary of the changes:
1. Fixed the syntax of the
vectors.push statement: added the closing curly brace.
2. Replaced the usages of headings with limitedHeadings.
3. Created some mock data for limitedHeadings and deduced the type of VectorInserted based on the code.
4. Ensured that the index bound to the worker contains 384 dimensions to match the output vector size of the @cf/baai/bge-small-en-v1.5 model.Recommendations:
1. I noticed that you are using a Vectorize V1 index while analyzing the code. We recommend that you transition to a Vectorize V2 index: https://developers.cloudflare.com/vectorize/reference/transition-vectorize-legacy/ to receive the added scalability and performance benefits. V1 indexes are on a deprecation path.
2. We recommend the use of Worker logs: https://developers.cloudflare.com/workers/observability/logs/workers-logs/ for additional insights and debugging ability into any issues related to your worker's functionality.
Cloudflare Docs
Transition legacy Vectorize indexes · Vectorize
Legacy Vectorize (V1) indexes are on a deprecation path as of Aug 15, 2024. Your Vectorize index may be a legacy index if it fulfills any of the follwing crieria:
Cloudflare Docs
Workers Logs · Cloudflare Workers docs
Store, filter, and analyze log data emitted from Cloudflare Workers.
I tried the code, actually it was some typo in my last code which i sent here, i fixed all issue of code, still getting null after inserting vectors, vectors are well structured in log before insert step, still null.
