Keycloak unstable
Hello!
So, I'm a bit of a newcomer to both keycloak and railway.
I digged a little here in this group until I found a post by brody with some instructions on how to run keycloak on railway and eventually I managed to deploy my own instance and configure it.
It works fine and I can login on my frontend app using next-auth.
However, sometimes I just get a "application failed to respond" message but I can't find any errors if I look at the deploy logs at railway.
I'm a bit lost and I don't know where I could start looking for errors.
If it helps, here's the dockerfile I'm using:
https://github.com/leonardochappuis/keycloak-docker/blob/master/Dockerfile
GitHub
keycloak-docker/Dockerfile at master ยท leonardochappuis/keycloak-do...
Contribute to leonardochappuis/keycloak-docker development by creating an account on GitHub.
103 Replies
Project ID:
N/Aa91e3f70-1457-4ae5-b646-23465ac28091
i dont actually use keycloak myself, i only fixed some common issues with their dockerfile that prevented it from deploying, i think @ThallesComH does though, thalles, have any ideas?
i was having that issue and I just got back to hosting my own keycloak
maybe it's that postgres timeout thing? i didn't dig deep enough
looks like neither of us know, sorry ๐ฆ
I can try to deploy again with a higher timeout but I'm not sure that's the issue. it usually works for a few requests and then I get that "application failed to response" screen for a few seconds
I'd say it works like 6/10 times
yeah i was getting that too
it doesn't show any error in logs i presume?
do you know if there's a way to output more uh, verbose logs?
for me it was the same thing
no
i don't remeber if i could find a variable to do that, but it's possible i think
oh, just found out
ok, I'll try to deploy again with a higher timeout and with all logs
btw I'm also from soccer country huehue
obrigado pela ajuda!
well, I'm not sure why but it appears you were right
I increased the timeout by a high amount and I'm getting no more errors at least for now
10/10 requests going through
I'm not sure if it will stay like this but if it does, I'll try to make a template
yeah do template!!!
it'd be my first but I'm looking forward to it
railway is the best
i've a dockerfile for keycloak, if you want i can send to you
thank you both for the help! I have managed to do a lot of stuff just by searching here and looking at your previous answers brody
sure, that would help, thanks
Ok, I believe it should be working. At least couldn't get it to fail so far ๐
Far from perfect I guess but we now have a starting point
https://railway.app/template/mSwigX?referralCode=AkM2z4
https://github.com/leonardochappuis/keycloak-docker/blob/master/Dockerfile#L27
while the variable will probably never be fully deprecated, it would be best to use
RAILWAY_PUBLIC_DOMAIN instead of RAILWAY_STATIC_URL
instead of 1StrongAdminPassword you can use ${{secret()}} to have railway generate a random 32 character string for every template deploy, you can also leave the admin variable blank so it forces the user to make their ownAh, I didn't know that.
I have updated both the dockerfile and the template. Hopefully it's better now
thank you for the tips
awsome, thank you for the template!!
no worries, thank you for your help!
btw brody, last thing
do you think I should change the postgres on my template to the new template one or is it ok to stick with the plugin one?
great question, stick with the plugin version of postgres, as the template postgres uses tcp proxying and thats a beta feature as far as im aware, since ive seen non beta users have issues deploying templates that used beta features
aight, thank you again
it did work on my end (non beta user here) but I guess it's better to be on the safe side ๐
for sure!
unfortunately there's still something off
really wanted to fix this
kuma!
and theres nothing bad in the logs when you get a 503?
no
that's what's weird
yeah, kuma is pretty cool
I uploaded what's basically the same dockerfile to fly.io
provisioned a high availability postgres, scaled the machine to 1gb ram
and there it's sitting like
I really wish I could host it at railway tho
there's gotta be something we are doing wrong, while railways postgres isn't highly available, they aren't high unavailable like your tests have shown
if you could diagnose this further would help a lot.
can you try hosting the keycloak at railway and the postgres database at another provider?
actually, nvm, i'll try it myself.
so curious to see what's going on
๐
i spinned up a Caddy webserver to proxy the requests to Keycloak through internal network and it's working great
I've no idea tho what's happening, maybe it's related to ipv4 or ipv6, i've no idea
@beuz if you want a workaround for now, spin up Caddy and use this
Caddyfile
hmmm what's the caddy for?
as i said, no idea, it's just working
interesting, though I don't think I'm happy with that solution
made some changes, will keep you both posted!
I got a 503 too, I am not built different
bruh
414+ messages of keycloak down, followed by keycloak up
are you using caddy? kinda confused
no ofc not
ah ok
and btw, it's still going strong
ignore the first ones, configuration file skill issue
Hi guys, I'll try this as soon as possible
Currently visiting my gf family
Will be back in 2 days
But thank you so much for the help!
i seriously wouldnt recommend throwing caddy in front of it
I figured it could have something to do with that but no idea why tho
The worst kind of bug. The one you have fixed but you have no idea why it works ๐
its likely just masking a problem, so it isnt a solution i can recommend
What I mean was that I was going to try connecting to an external postgres
maybe keycloak is sending a malformed response, caddy can deal with it no problems and envoy (railways proxy) can't and ends up returning a 503
I'm pretty stuck on this as the reason, since if keycloak wasn't responding at all then it's not like caddy would return 200 while keycloak doesn't respond
@ThallesComH what ya think?
we can probably test it
https://hub.docker.com/r/envoyproxy/envoy
if you've time to test it now, i can only do it at night
hmmmm how easy is it to run envoy?
i've no idea tbh
it's probably as difficult as traefik
seems hard
yeah traefik is hard
well here's another reason for railway to expose envoys logs to the users (they already want to do that, but it's low priority so they haven't gotten around to it yet)
Welp keycloak has not failed for a day or so now, without a separate railway service for caddy
how did you manage brody?
to make it not fail for a day?
with caddy
oh
to be clear, I'm fine with using caddy, I just didn't like the idea of using it without knowing why
sure
is it too difficult to import data from one keycloak instance to another or is it just a matter of importing the tables into a new postgres instance?
I also wasn't fine with using caddy as a separate service because that's not fully backwards compatible with current deployments of keycloak
yeah, it would make things more difficult
can you walk me through what you guys did? I'll update my template
why are wanting to move keycloak data from one database to another? are you wanting to use the database services instead of the plugins?
no, I have my instance set up at fly.io still
I have like $500 in credits
haha did you apply for a job there?
yes
nice
free credit hack
I didn't get it but still would want to make use of those credits xD
hahahah yeah
well you could use pg_dump and pg_restore
I'd want to move the data to railway once I'm out of credits though
yeah, that's what I thought. I believe it'd work, they're basically the same images, configuration and everything
yes I'll provide you a doc with all the changes you need to make, and I'll do a pr on your template repo that adds my modified dockerfile and caddyfile
you could probably even use a desktop tool like dbgate that has a GUI and can clone over a database for you
right, sounds easy enough
and thank you for the pr btw
and I'll get all the information you need in order to update the template and the pr later today
That was my first time using keycloak so I was never sure if I was doing something wrong
I mostly used external auth like firebase or auth0 exclusively
I'm pretty confident in this being an issue with how keycloaks http server is interacting with railways proxy
yeah, that makes sense
as to why caddy would fix it
ideally keycloaks http server would not be sending malformed responses, and Ideally railways envoy proxy should be a little more forgiving, so both sides are at fault
caddy is built in go, and go apps just simply work without any fuss
true
this I can neither confirm nor deny
but I'll trust you
and what do you think about not exposing the postgres service publicly by default?
because in my working implementation I use the private network to communicate to the database without having the database exposed publicly
(this isn't helping keycloak work better, it's just for an extra piece of mind)
sure, that would work
it's not used in anything else
the only thing I can think of for having it exposed publicly would be importing data into it
that requires it to be public
yeah, well, that's an edge case, I think
yeah that's why I like to add a note to the overview that states this information and how to turn the tcp proxy on and back off
I'll do just that
I'll write that for you too
thank you sensei
https://github.com/leonardochappuis/keycloak-docker/pull/1
there are example env files, all the variables would need to be updated on the templates service, if i dont have the variable in my example env files, then you dont need it on the service, for example you have a PORT variable, and i dont, so you can remove it from the template.
please change the service name from "Keycloak-Docker" to just "Keycloak"
your template uses the old postgres plugin, remove that from your template and add another service named "Postgres"
keep the network unexposed, image source is
ghcr.io/railwayapp-templates/postgres-ssl:latest and volume mount point is /var/lib/postgresql/data then just use the environment variables from the example env file
in the keycloak service set the healthcheck path as /health/ready
add this to your template overview
i probably forgot something, so let me know if something doesnt make senseHey brody! thank you for this
I'll make the changes now
Ok, everything should be live
oh no, I'm missing the healtcheck path
but other than that it should be ok
aight, all set 
leave
KEYCLOAK_ADMIN blank
you forgot to fill out POSTGRES_HOST and POSTGRES_PORT with the reference variables I gave
I do see what you where getting at from your description of those variables, but that's not quite how it works, you can fill out those reference variables and set the service to unexposed in the template editor mode, so that when/if the user wants to enable tcp proxying they can do so via the postgres services settings menu
you should join beta, type /beta
@beuz ๐done and done and done
everything looks good from just the template deploy page, have you done another test deploy of this template?
just did
everything look good?
yep, so far so good
awesome
yeah, I'm glad
thank thalles, he came up with the idea I just built upon it
for sure
thank you @ThallesComH, now everyone can use keycloak hassle free