How can I install a Docker image on RunPod?
I had a chat with the maintainer of aphrodite-engine and he said I shouldn't use the existing RunPod image as it's very old.
He said there is a docker that I should utilise: https://github.com/PygmalionAI/aphrodite-engine?tab=readme-ov-file#docker And here is the docker compose file: https://github.com/PygmalionAI/aphrodite-engine/blob/main/docker/docker-compose.yml Sorry if my question is very basic. How do I build a RunPod image with this docker so that I could run it later on RunPod? I'm still learning Docker, I would appreciate clear instructions. Many Thanks
He said there is a docker that I should utilise: https://github.com/PygmalionAI/aphrodite-engine?tab=readme-ov-file#docker And here is the docker compose file: https://github.com/PygmalionAI/aphrodite-engine/blob/main/docker/docker-compose.yml Sorry if my question is very basic. How do I build a RunPod image with this docker so that I could run it later on RunPod? I'm still learning Docker, I would appreciate clear instructions. Many Thanks
16 Replies
Well use the dockerfile
Watch youtube about docker quickstart if you want an explanation
use this docker image tag:
alpindale/aphrodite-engine:latest
in your templateAnd you cant use the docker compose file
one template I made cause official one had some bugs
oh thats nice
Hi guys,
Thanks I understand now. It's much easier than I thought. I thought I had to push my own custom docker image to DockerHub and pull it with RunPod etc. But everything can be achieved simply through RunPod's template. That wasn't clear to me at first.
Papa, there are further issues with your template, that's why I spoke to the maintainer. I raised a ticket with him on Github, let see if this can be fixed in the long term. Two issues that you can fix in the meanwhile are the following:
Passing in PORT has no effect because the docker image is hardcoded to port 7860. You can as well remove it from your template.
Passing in API_KEY also has no effect, which leaves the API unprotected. APHRODITE_API_KEY is however accepted and works correctly.
@houmie it all depends cause RunPod has many community templates. Could you specify issues you get?
The api key do not work with runpod as ther entrypoint sh script is not passing it at startup though it's easy to fix
Hi @Madiator2011 (Work)
Thank you. I created my own private template with the ID
m2oy2t3v78 Feel free to check it out if you have access to it. (I assume you work for RunPod, however I could change it to public if you can't, no problem.)
The two major issues I'm having now is:
1) API_KEY. As you pointed out it seems it is not passing at startup, hence BEARER ${API_KEY} has no effect. Anyone could use my API without authorisation. But the workaround I found is to set APHRODITE_API_KEY instead of API_KEY then it works as expected. Papa Madiator created this public template (https://runpod.io/console/deploy?template=ysb4b332xu&ref=vfker49t) which also has this issue. I would like to learn how you fix this, so I could apply it to my own private template as well, please.
2) How to disable KoboldAI Lite to avoid abuse? After deploying it on aphrodite-engine anyone that opens the URL on the port 7860 gets to see the KoboldAI Lite interface. But we don't need it, because we use this port only for the API. How can we disable this UI to avoid abuse?
The other small issue although not major is the PORT. It is currently hardcoded to 7860 and can't be changed. You can see the env file here: https://github.com/PygmalionAI/aphrodite-engine/blob/main/docker/.env and here is the docker file: https://github.com/PygmalionAI/aphrodite-engine/blob/main/docker/Dockerfile.just need to fix this script https://github.com/PygmalionAI/aphrodite-engine/blob/main/docker/entrypoint.sh
GitHub
aphrodite-engine/docker/entrypoint.sh at main · PygmalionAI/aphrodi...
PygmalionAI's large-scale inference engine. Contribute to PygmalionAI/aphrodite-engine development by creating an account on GitHub.
as you see there not all arguments are passed there
Ok thanks. Now comes the difficult question. I'm not the maintainer of the project. I can't update that file. What are my options to fix it for my own private template?
pro tip instead of having to rebuild container just use image as base on copy new entrypoint
Which image to take as base though? They all seem to have these issues. 🙂
I mean image itself does not have issues the issue is entryfile
Interesting and how would I do that, please?