sdev
RRunPod
•Created by sdev on 10/23/2024 in #⚡｜serverless
Network volume used size
How to check used size of network volume in Storage panel?
2 replies
RRunPod
•Created by sdev on 10/23/2024 in #⚡｜serverless
Flashboot principles
Hello. Who can explain Flashboot principles?
When worker is idle model stays in gpu memory or pc memory?
How long the model stays in memory? Is some LRU eviction policy used?
4 replies
RRunPod
•Created by sdev on 9/17/2024 in #⚡｜serverless
Attaching python debugger to docker image
How is it possible to attach debuger to docker image:
docker run -it --rm --name model_container \
--runtime=nvidia --gpus all \
-p 10002:5678 -p 10082:8000 \
-v /home/devel/model/src:/src \
models_image:0.1 \
sh -c "pip install debugpy && python3.11 -u /src/rp_handler.py -m debugpy --listen 0.0.0.0:5678 --rp_serve_api --rp_api_port 8000 --rp_api_host 0.0.0.0 --rp_log_level DEBUG --rp_debugger"
On host machine i can open http://localhost:10082/ with RunPod | Development Worker API
Debugging port on host machine 10002 is opened but when i try to attach with vscode nothing happens
#my launch.json
{
"name": "Python: Remote Attach runpod model",
"type": "python",
"request": "attach",
"connect": {
"host": "localhost",
"port": 10002
},
"pathMappings": [
{
"localRoot": "/home/devel/model/src",
"remoteRoot": "/src"
}
],
"justMyCode": false
}
#/src/rp_handler.py code is empty
import runpod
print("Starting JOB processor")
def handler(job):
print(""" Handler function that will be used to process jobs. """)
job_input = job['input']
name = job_input.get('name', 'World')
return f"Hello, {name}!"
runpod.serverless.start({"handler": handler})
print("Started JOB processor")
And is it possible to configure hot reloading on code change?
2 replies