Quick question 1. Why do we need to create metadata index and even if we have to create, why only 1
Quick question
1. Why do we need to create metadata index and even if we have to create, why only 1 key at a time and upto 10 index is the limit? While upstash allows to filter on metadata normally on all metadata
2. Can we also expect glob and more type of searches operators along with eq and neq
Like : contains , IN, not in, glob, pattern match, and , or etc
3. can we also have hybrid search like a cosine similarity search with Full text search as well for better search in future?
16 Replies
Unknown User•3mo ago
Message Not Public
Sign In & Join Server To View
Vectorize now has a significantly improved query latency through REST API https://developers.cloudflare.com/vectorize/platform/changelog/#2024-10-28
Cloudflare Docs
Changelog | Vectorize
Hey! Whats the limit on insert/upsert.
for some reasons, my vector inserts/upsers have stoped at 128.04k vectors.
In dev, in another vectorize database, everything is working fine. Is there any hidden limit we dont know about???
@yevgen
Is there a way to dynamically add Vectorize bindings programatically?
ah i found it https://developers.cloudflare.com/vectorize/best-practices/create-indexes/
Hmmm, but these don't seem to be enough number for my use-case
Indexes per account 50,000 (Workers Paid)
Unknown User•3mo ago
Message Not Public
Sign In & Join Server To View
This issue is happening from : 2024-10-25T14:36:55.873Z
whatever improvement was made on this date, caused this to break.
Triggering it from binding or api, even tho the call return succees true, it actually does nothing on the index. No delete, no insert, no upset
Nothing should have caused such issues, let me check with the team, it could be delayed as most of the team is in US time zone.
Thanks for getting back! Okay thank you.
📢 Vectorize update:
2024-10-30: Vectorize has local mode support in Wrangler @3.84.0 https://github.com/cloudflare/workers-sdk/releases/tag/wrangler%403.84.0
GitHub
Release [email protected] · cloudflare/workers-sdk
Minor Changes
#6999 0111edb Thanks @garvit-gupta! - docs: Vectorize GA Announcement Banner
#6916 a33a133 Thanks @garrettgu10! - Local development now supports Vectorize bindings
#7004 15ef013...
Unknown User•3mo ago
Message Not Public
Sign In & Join Server To View
I am noticing a large number of
AiError: 3001: Unknown internal error s when doing some Vectorize queries. Is there any way to get more insight into what these are? I have no clue if it's not finding a vector by ID, if I'm getting rate limited, or if it's something I can't controlI've been browsing through messages in the past and documentations on vectorize, looks like now it's a great time to jump in, considering the beta release of workflow as well. though my concern is full RAG pipeline still needs a long way to go, so I'd prefer develop locally using llamaindex+chromadb(or equivalent)+litellm(embedding, inference, rerank, whisper, etc) first, before switching to cloudflare edge side completely, which I do need at a certain point because the product I am building requires global coverage.
may I ask guy who already had previous experiences here, that, is this a viable plan, or anything I should keep in mind for choosing tech stack or develop or doing gitops? thanks a million!
Hi, is Cloudflare Vectorize now support $in metadata filter ?
Unknown User•3mo ago
Message Not Public
Sign In & Join Server To View
📢 Vectorize update:
Added support for
$in and $nin metadata filters
https://developers.cloudflare.com/vectorize/platform/changelog/Cloudflare Docs
Changelog | Vectorize
Unknown User•3mo ago
Message Not Public
Sign In & Join Server To View