Database error: error parsing response from server: `S` field missing
Hello everyone, I'm trying to use Xata for an app (Umami) I'm trying to deploy to Vercel. Vercel seems to be able to connect to the database (according to logs), but then I get this error:
and I have no idea how to work around it. I don't have another PostgreSQL instance I can use to test if it's a specific database issue or an app problem. I have already reported the issue to Umami but I haven't got a reply yet. Do you know if this could be related to some Xata limitation? Thanks
47 Replies
Hi, the failed command is "prisma migrate deploy". Does that work when run locally?
Prisma migrate requires a shadow database, in addition to the target xata database. You can read more about it here: https://xata.io/docs/integrations/prisma#shadow-database-for-migrations
Have you set up the shadow database?
hi and thanks for your reply. I'm new to Vercel to be honest and very little experience with NodeJS stuff (I'm a Python backend dev as a daily job). Coming to your quesiton: I haven't tried
prisma migrate deploy locally because I don't run the app locally, I'm trying to deploy directly to Vercel, following these instructions https://umami.is/docs/guides/running-on-vercel I'm also not aware of what a "shadow" db is but I'm going to read that document in a moment...
going to create it now, hold on
ok, umami-migrations created and added the SHADOW_DATABASE url to Vercel. Retrying deploy...
I just don't get this step: configure this database's Postgres connection string as the shadowDatabaseUrl in the Prisma datasourceYou can see a more complete example in the Prisma docs here: https://www.prisma.io/docs/orm/prisma-migrate/understanding-prisma-migrate/shadow-database#cloud-hosted-shadow-databases-must-be-created-manually
Basically, you need the Postgres endpoint of the Xata database in the "url", and the Postgres endpoint of the additional shadow database (which is used by Prisma to store migration information) in "shadowDatabaseUrl"
I did set both env variables and rerun the deployment. But now I got a different error in Vercel:
and on Xata dashboard I see this:
uhm...no well the error is the same as before, sorry
The fact that it created a _prisma_migrations table means that it did pick up the shadow db which is good
but it was creating that table even before
and it created it on
umami not on umami-migrationsAh then it didn't pick it up
I even tried to apply this, previously
Adapting isn't going to help, this is to make the tables render in Xata
and it does some changes as it says... then if I re-run, a row is written to the table, but it cannot complete the migration. So I was deleting that row every time to make it clean
what is your workspace id so I can check logs on our end?
Andrea-Grandi-s-workspace-5jfka8
(and yes, I checked Allow support to view your workspace)
thanks 🙏🏻It is failing on this step:
CREATE EXTENSION IF NOT EXISTS "pgcrypto";
never used/heard
That extension is already available on the Free tier: https://xata.io/docs/postgres#extensions
Trying to create it, it gets an error from Xata because we don't allow managing extensions on the Free Tier. We only allow them on Dedicated Clusters.
ahhhh I see... so it's partially because of my config, but also a lack of handled exception on Umami side
So it is trying to run that statement, it gets a rejection and it fails
yeah
It should continue even though it got an error on that statement
ok.. it gets complicated then 🙂 I can try to report this as a bug and hope they can fix it. I doubt I can also provide a PR to them (never touched a single line of NodeJS)
but thanks for the info because this will surely be useful to understand what needs to be fixed
I am checking with our team if we could silently not return an error when trying to create an extension that is available, to avoid the client aborting like this. But this is going to take us a while even if acknowledged.
I found the line in their SQL script https://github.com/umami-software/umami/blob/b006747a457b8284b2679d6611ddd16378f7dfaa/db/postgresql/migrations/01_init/migration.sql#L2
I wonder if I could simply copy-paste the whole script and run it on Xata (so that it crates the tables) but I would need to also fake the row in
_prisma_migrations table...While running the script directly in Xata (over psql/wire) will work,I think it will conflict with the rest of the umami setup. It seems they use Prisma for all migrations, and introducing migrations (sql commands) "externally" will break its commit log
let me give it a try...
it's more plausible if you can clone the umami repo and edit out that line, and run the rest as-is
let's see if it resumes from the step 2
ok
ahhhh right
that's true. I already cloned it on my account 🤦♂️
but let's see if this works first... if it doesnt I can patch my fork
it went forward
still running...
The dev team confirmed we can suppress returning an error when the extension already exists
nice, that's some progress
this would fix the issue for other people running into the same issue ❤️
totally
it's up and running 🚀
That's great news! Well done 🙂
in the end it seems that the
umami-migrations (the shadow db) wasn't needed (it's currently empty). Should I delete it? Or it doesn't matter?"Normally" you do need this in order to use prisma migrations, which seems to be the underlying command issued in umami. Maybe check with Umami if they do support a shadow database, and if so, what is the process/env variable to set to configure it. Other than that, if things work 🤷♂️ you can certainly remove it
I think Umami does, by default, some aggressive data polling. The "realtime" data stopped working. I checked Vercel logs and I see a lot of these errors:
The Free plan has a limit of 20 concurrent connections, and that is the exact error returned when they're reached
The Pro plan bumps that up to 60, and the Dedicated Instances are unlimited (it goes as high as the instance can take)
probably not an issue on your side (you rightly limit the number of connections), but rather on their side. It's absurd that a single user application uses all the 20 connections
not even an issue of too many user on the website because the app was only recording 2-3 visits
I think their "real time" code is so badly written that I would hit the limit even on a pro plan
I cannot comment on the quality of the app as I am not familiar with it but doesn't sound scalable if it reached the 20 connections limit with minimal to no traffic. Unless it is designed to run against a dedicated Postgres node in which case the connection limit would be in the thousands, so maybe it's tuned to take advantage of as many parallel requests as possible.
well.. logs are quite clear: once you move away from the "Realtime" page, all the errors in Vercel logs disappear and the main dashboard shows data correctly
Then the realtime page is overly ambitious on the database bandwidth expected
I found this https://github.com/umami-software/umami/issues/87#issuecomment-684861864 they seem to "blame" Prisma but they haven't effectively fixed it
GitHub
Too many connections · Issue #87 · umami-software/umami
i am getting too many connections error when when loading the dashboard i am using aheroku postgresdb which can take up to 20 connections concurrently . is there a way to overcome this issue
using
?connection_limit=10 (in my case) seems to mitigate the issue. I left the tab open on Real Time for 30 minutes, no errorsSo it supports connection limits as a parameter? That's great to hear and good to know as a workaround in case someone else hits the same. Thanks for sharing!
So it supports connection limits as a parameter?it seems so! They haven't updated the ticket but apparently the workaround...works. Personally I would have provided a nice default and added an env variable but it's ok. I only hope that
10 connections means the driver/library is automatically distributing the incoming requests among those connections... and not that I can only have 10 visitors at the same time (not that I regularly have 10 visitors at the same time... but when you post a link somwhere you usually get a lot of requests in few seconds and then they slow down.. in that case I would miss the events)@kostas I wanted to thank you again for helping me a couple of weeks ago. I finally found the time to document the whole procedure I did and also mentioned your service https://www.andreagrandi.it/posts/self-hosting-analytics-at-zero-cost/ 🙂
Andrea Grandi
Self hosting a website analytics at zero cost
How to self host a website analytics solution at zero cost using Umami, Vercel, Xata and Cloudflare.