R
RunPod6mo ago
Deeps__

Unable to upgrade linux kernel version from 5.4.0 to 5.15.0 - RunPod A40 GPU

I'm trying to upgrade my linux kernel from version 5.4.0 to 5.15.0. This is required for me to train deep learning models. Here's what I tried 1. I tried to manually upgrade it with apt command. however I'm still getting the same kernel version 2. I tried stopping and restarting the pod but the issue persists 3. Tried with a new pod by deploying the same A40 configuration and still the issue persists. Below is the log from the terminal root@9301aaa036f0:/# uname -r 5.4.0-107-generic root@9301aaa036f0:/# apt install linux-generic Reading package lists... Done Building dependency tree... Done Reading state information... Done The following additional packages will be installed: ... .... Creating config file /etc/default/grub with new version Setting up linux-image-5.15.0-113-generic (5.15.0-113.123) ... I: /boot/vmlinuz.old is now a symlink to vmlinuz-5.15.0-113-generic I: /boot/initrd.img.old is now a symlink to initrd.img-5.15.0-113-generic I: /boot/vmlinuz is now a symlink to vmlinuz-5.15.0-113-generic I: /boot/initrd.img is now a symlink to initrd.img-5.15.0-113-generic Setting up linux-modules-5.15.0-113-generic (5.15.0-113.123) ... Setting up linux-modules-extra-5.15.0-113-generic (5.15.0-113.123) ... Setting up linux-image-generic (5.15.0.113.113) ... Setting up linux-generic (5.15.0.113.113) ... Processing triggers for libc-bin (2.35-0ubuntu3.8) ... Processing triggers for dbus (1.12.20-2ubuntu4.1) ... Processing triggers for initramfs-tools (0.140ubuntu13.4) ... Processing triggers for linux-image-5.15.0-113-generic (5.15.0-113.123) ... /etc/kernel/postinst.d/initramfs-tools: update-initramfs: Generating /boot/initrd.img-5.15.0-113-generic root@9301aaa036f0:/# uname -r 5.4.0-107-generic
13 Replies
nerdylive
nerdylive6mo ago
Not sure if this is really possible Is it @Papa Madiator
Madiator2011
Madiator20116mo ago
You cant as containers use host kernel
Deeps__
Deeps__OP6mo ago
Can we please upgrade the host kernel to any stable version greater than or equal to 5.5.0 ? If not, is it possible to temporarily upgrade the host kernel until we keep the pod active ?
Madiator2011 (Work)
Not possible as kernel update requires full host machine reboot
Deeps__
Deeps__OP6mo ago
oh ok I got more clarity. And, is there a solution to the problem?
Madiator2011 (Work)
Not sure if it's secure cloud or community cloud
Deeps__
Deeps__OP6mo ago
it's a secure cloud
yhlong00000
yhlong000006mo ago
Just curious what deep learning model you’re training that requires newer versions of kernel?😀
Deeps__
Deeps__OP6mo ago
i'm training a deep learning model that sees (via camera), identifies the objects/persons in the world, see what people are doing etc.. using PyTorch with reasoning and explanation
nerdylive
nerdylive6mo ago
wow interesting For now, i'd suggest to use other libraries that doesn't require this linux kernel update
Deeps__
Deeps__OP6mo ago
ok and is there a timeline where the kernel can be updated? I just need an approximate timeline so that I can wait for the kernel upgrade
yhlong00000
yhlong000006mo ago
That's really cool~Not sure how fast RunPod can upgrade kernel, but since they have so many data centers, it seems pretty hard.😆
Deeps__
Deeps__OP6mo ago
ok thanks for the info!
Want results from more Discord servers?
Add your server