As I move from a proof of concept to a real application product,is there a way to migrate the data o
As I move from a proof of concept to a real application product,is there a way to migrate the data or rename an existing vector index? It would be very handy to be able to do this in the wrangler cli
15 Replies
Unknown User•3mo ago
Message Not Public
Sign In & Join Server To View
I can repro - seems like the dash only lists old (v1) indexes and not v2. I've let the team know so hopefully they'll look into this next week. Until then you can use
wrangler vectorize list to see your v2 indexesUnknown User•3mo ago
Message Not Public
Sign In & Join Server To View
The v2 docs indicate that querying indexed metadata can truncate long text values. I understand not wanting to define the exact value in the API, although is there a guideline for this truncation threshold? I would like to store the original text associated with the vector. In my use case, these text values are generally a few hundred words or less.
You can store up to 10KiB of metadata per vector; vectorize will either accept the data if it fits, or reject the upsert if not. The truncation happens in the metadata index at 64B, meaning you can filter indexed metadata properties on their first 64B. The original vector metadata is never truncated, and can be obtained verbatim on vector query by specifying the
"metadata": "all" option.Thank you for those details. That’s really helpful. So basically ~64 ASCII characters before truncation when using indexed metadata?
The challenge with metadata: all option for me is that it limits to topK 20 results. I’m hoping to return closer to 40 results, as I do a rerank and then trim the set before presenting to the LLM. Metadata: indexed would let me return topK 100 but the truncation cutoff makes it less useful.
Is the topK max of 20 anticipated to improve in the future with metadata: all? Otherwise I will need to completely re-architect and store metadata independently of vectors.
Hey guys! I'm aware that Vectorize is currently in beta, but I'm curious about its reliability for production use. My specific use case involves a chatbot that serves approximately 1 million users per month. If Vectorize isn't suitable, could you recommend any other alternatives? Pinecone? pgvector?
Any plans to support arrays in metadata filtering ?
Hey, quick question: its expected that on the web dashboard you don't see any of the vector indexes you've created (while in wrangler you do) ?
ah that has already been answered here https://discord.com/channels/595317990191398933/1279371665859674132/1280103851298394145 thanks!
Is there a limit to how many vector indexes we can create?
Edit: found the limits docs, all good
Unknown User•3mo ago
Message Not Public
Sign In & Join Server To View
Cool! Are there any early details on how the limits will increase? 😊
Is there any way to upgrade a v1 Vectorize index to v2, or do I need to create a completely new one?
Unknown User•3mo ago
Message Not Public
Sign In & Join Server To View
TIL that you can link a slash command in a message 😄
Unknown User•3mo ago
Message Not Public
Sign In & Join Server To View