Increase check limit on text
It seems like Xata is limiting "text" columns to 204800 when it's created through the ui. I'm judging this based on the schema that's generated through the CLI
Is there a specific reason for this, will search break if I increase the column size limit?
6 Replies
Ah, it seems like that's only allowed in a dedicated cluster? Can anyone confirm
The column length limits exist in shared environments to keep performance and resource utilization in check. Longer string or text limits are not supported in the shared environments.
Are you looking to index very large text column values? Could you provide more details?
You can create i.e. varchar columns with extra length and those will be replicated for Search but up to 1MB limit per record.
Is that 1MB limit per row or per column?
It's either of the two: the indexing pipeline that replicates data from Postgres to OpenSearch is limited to processing objects 1MB in size.
It is necessary for our service to have a reasonable limit in place to ensure performance at scale.
Over the REST API the limits are stricter than that: https://xata.io/docs/rest-api/limits#column-limits max column size id 200kb and max record size is 600kb.
Using Postgres over Wire these limits do not apply, but the 1MB safety limit kicks in when indexing data for search.
Gotcha, does the typescript Xata SDK work with the postgres "wire" connection?
I was using the HTTP databaseURL, and when I switch to postgres URL, I get
invalid workspace ID is this a configuration on my account error or an error in code?
The same postgres URL works in drizzle-kit studio (so the link seems to be working with other platforms). I've tried several permutations of the postgres URL with the Xata SDK but to no avail.Hey, the Xata SDK works only with the HTTP endpoint (REST API). We have plans to enable Postgres in it in the near future (next few months) and make all SDK calls also work over wire.
For now, you can use the SQL over HTTP endpoint of the Xata SDK https://xata.io/docs/sdk/sql/overview (xata.sql
STATEMENT) which has very close parity to the wire protocol connection, it just works over an HTTP proxy.SQL over HTTP
How to access Xata using SQL directly over HTTP