JCtheMC
Issue with Huggingface dataset not being cached to storage volume
I want to use https://huggingface.co/datasets/HuggingFaceFW/fineweb-edu for a project. I'm trying to download this dataset through the python datasets package. I want this download to be stored on my storage volume. As per the documentation here: https://huggingface.co/docs/datasets/v3.2.0/en/cache#cache-directory , the package offers the option to either set an environment variable or use a function argument to specify the download directory. I've tried both approaches, but whatever i do, the cached files keep ending up on the Container instead of my storage Volume. Edit: it may very well be that i'm not defining the path correctly - i have limited linux experience. Please help.
45 replies
How to keep training running after disconnecting through VSCode?
I setup vscode to connect with a pod with this guide: https://blog.runpod.io/how-to-connect-vscode-to-runpod/. I then closed one of my own repo's, and started a training script in the bash terminal. How can i close the remote connection in vscode, but keep training running? At a later point i would like to reconnect through vscode again and check on the progress.
I apologise for the extreme noobishness of my question, but i've never worked with ssh or anything remote through vscode.
Thanks in advance.
11 replies