Deployed app responds randomly with 503s
Hello everyone,
when I open the admin dashboard of a deployed KeyCloak instance, I get 503's instead of the resource for random web resources (images, JS files, html files,...).
In the browser's network console you can then see the Railway "Application failed to respond" page in the bodies of the 503.
I have to reload the page several times until all the resources needed for the dashboard could be loaded.
The problem has existed for quite a while. In the logs of the deployed KeyCloak instance there is no indication that anything is wrong (so no noticeable error logs or anything like that)
Solution:Jump to solution
okay so for context, i have seen this before with keycloak, sometimes its server sends malformed responses that envoy (railway's proxy) does not like, the solution was to run a caddy proxy in the same container in parallel and expose the caddy proxy only, caddy would internally proxy to keycloak, thus smoothing over the responses from keycloak's server (for lack of a better term)
You can see how i have implmented that here: https://github.com/leonardochappuis/keycloak-docker...
22 Replies
Project ID:
f751037a-ba7a-42ab-8a24-90e99fcd12d3f751037a-ba7a-42ab-8a24-90e99fcd12d3
you had it working before right?
I've had this problem for what feels like forever.
I think the 503s always appeared randomly for this app
i take it you have not deployed keycloak from the template then?
Yeah, i am using a "selfmade"-DockerImage
Solution
okay so for context, i have seen this before with keycloak, sometimes its server sends malformed responses that envoy (railway's proxy) does not like, the solution was to run a caddy proxy in the same container in parallel and expose the caddy proxy only, caddy would internally proxy to keycloak, thus smoothing over the responses from keycloak's server (for lack of a better term)
You can see how i have implmented that here: https://github.com/leonardochappuis/keycloak-docker
Thank you very much!
I will try it and post the result here
let me know if you have any issues!
I can not believe it but it seems like it works now π
I reloaded the Dashboard 10 times and could not produce one 503er Error
Thank you very much Brody!
I have still some question, about the topic:
- What kind of malformed responses are keycloak sending?
Is there an open ticket at KeyCloak for this?
If the responses do not meet the Http specification, then that needs to be corrected.
Or is the railway proxy too sensitive here?
What kind of malformed responses are keycloak sending?i honestly have no clue
Is there an open ticket at KeyCloak for this?i also have no clue
Or is the railway proxy too sensitive here?this is more likely the case, envoy is getting removed in favour of railway's own home grown proxy anyway at some point
Does it then make sense to address the issue with Railway so that they can configure their proxy accordingly?
keycloak is the only thing i have seen have this issue, so no it doesnt really make sense for them to spend the time to fix this thing that likely only effects keycloak
fair enough (:
new proxy too, so also not worth the time when envoy is getting thrown out a window
just curious too, where you using the latest version of keycloak?
One more little question:
In the keycloak-Template-Repo there is this java.config:
https://github.com/leonardochappuis/keycloak-docker/blob/master/java.config
Do i need that too or why are this file there?
Yeah i was using the latest Version of KeyCloak
i think that was to remove a newer auth version for database communications that the database railway deployed at the time didnt support, but now railway deploys up to date databases so you likely dont need to do that
you are now using multirun and caddy in the container too?
right
awsome!
you can mark your answer as the solution
Bought your more Coffee for the help!
You saved me many hours!
thank you very much, I appreciate that!!