GPU Passthrough Questions/Bug?
Having issues doing a few things for a few games (mods). So, as this is the first time I had two GPUs to even bother figured I'd give GPU passthrough a shot... Now I could just be an idiot, probably. However, after running ujust setup-virtualization, enabling virtualization, setting up a VM, and then enabling virtio driver with the kargs to grab my 4070ti Super and rebooting it still shows the monitor connected to the 4070ti Super when booting into the host and logging in.
IOMMU is enabled in BIOS. Figured I'd post this here while I beat my head on this some more.
197 Replies
can you run this
then exit the terminal and open it again and run
ls-iommu -grk and show the outputWow a response already! Yes one moment
was about to go sleep, but im the person who does most of the gpu passthrough stuff here 😅
😫
IOMMU Group 13: 0000:01:00.0 VGA compatible controller [0300] NVIDIA Corporation AD102 [GeForce RTX 4070 Ti SUPER] [10de:2689] (rev a1)
Subsystem: Micro-Star International Co., Ltd. [MSI] AD102 [GeForce RTX 4070 Ti SUPER] [1462:5100]
Kernel driver in use: nvidia
IOMMU Group 13: 0000:01:00.1 Audio device [0403] NVIDIA Corporation AD102 High Definition Audio Controller [10de:22ba] (rev a1)
Subsystem: Micro-Star International Co., Ltd. [MSI] AD102 High Definition Audio Controller [1462:5100]
Kernel driver in use: snd_hda_intel
IOMMU Group 29: 0000:71:00.0 VGA compatible controller [0300] Advanced Micro Devices, Inc. [AMD/ATI] Granite Ridge [Radeon Graphics] [1002:13c0] (rev cb)
Subsystem: Micro-Star International Co., Ltd. [MSI] Granite Ridge [Radeon Graphics] [1462:7e51]
Kernel driver in use: amdgpu
Kernel driver in use: nvidiashow me your
rpm-ostree kargssure.
This is what I ran. rpm-ostree kargs --append-if-missing="vfio-pci.ids=10de:2689"
misunderstood
resume=UUID=f81ba298-29c6-452f-bc01-cef4fd989757 rhgb quiet root=UUID=cdb31415-20c1-4cf2-8b6f-3238a5ec7bb3 rootflags=subvol=root rw ostree=/ostree/boot.0/default/818a0c4d3027e4800631b2c2606dcdc337c932f5d62a9d5a6a6308184f17df59/0 bluetooth.disable_ertm=1 preempt=full kvm.ignore_msrs=1 kvm.report_ignored_msrs=0 amd_iommu=on iommu=pt rd.driver.pre=vfio_pci vfio_pci.disable_vga=1 vfio-pci.ids=10de:2689,10de:22ba vfio-pci.ids=10de:2689
rpm-ostree kargs --delete=vfio-pci.ids=10de:2689
other than that as long as the VGA and Audio device are the only things in group 13 you should be good after reboot
you can check with ls-iommu -i 13error: Unknown option --remove=vfio-pci.ids=10de:2689
sorry its --delete
its 4.30 am 🤣
It's running now.
ok check iommu group 13 in meantime, make sure its only VGA and Audio device that is in it
Those are the only ones.
IOMMU Group 13: 0000:01:00.0 VGA compatible controller [0300] NVIDIA Corporation AD102 [GeForce RTX 4070 Ti SUPER] [10de:2689] (rev a1)
IOMMU Group 13: 0000:01:00.1 Audio device [0403] NVIDIA Corporation AD102 High Definition Audio Controller [10de:22ba] (rev a1)
oh i also see the issue
vfio_pci.ids
supposed to be underscore
i will fix that in testing as we are rewriting the virtualization just for preparation for bootc
Ok. So I'll need to change the other argument too?
yeah just run this
rpm-ostree kargs --delete=vfio-pci.ids --append-if-missing=vfio_pci.ids=10de:2689,10de:22baAlright, running. I can keep poking this bear if you need to get some shut eye.
and in a future update you will be able to
rpm-ostree initramfs --disable and regain a lot of update speed since currently the way we enable vfio slows down updates a lotAh. Cool.
if you want to get ahead of the change for kvmfr let me dig up the command
I wouldn't mind. Says it's done with the commit so I'm about to switch to phone and reboot.
rpm-ostree kargs --append-if-missing=kvmfr.static_size_mb=128
if you run this, you will be able to just disable the initramfs regeneration next time we merge testing into stableSo, I'm stuck here now (this is the screen connected to the virtio gpu) which shouldn't be displaying anything.
I can probably get back up through the backup boot option if you need sleep.
yeah and the screen that is on your other gpu?
It's just blank
what happens if you boot without the nvidia gpu attached to the monitor? as the amdgpu should display stuff
It's actually the one that grub showed up on then the bazzite thing popped up on the one it shouldn't
One min
@HikariKnight so with nothing plugged into that virtio gpu it boots fine.
plug in the gpu afterwards then, not sure why it is angry when its connected
unless thats the one grub shows up on, then that might be an issue
i tend to keep my passthrough gpu in the 2nd gpu slot
No grub shows up on the igpu display
hmm
Funny business
well as long as the monitor is connected before you boot the VM it should be fine
As soon as I connected a cable to the virtio gpu it froze up
hmm not sure about that one, first time i have run into it just freezing up
can you do
ls-iommu -grk and check if everything is bound properly?Once I reboot.
I can't do anything
can you also do
ujust logs-last-boot too
might give some idea wth happenedAlmost in
How do I get the logs to you?
ujust logs-last-boot | fpastePosting
https://paste.centos.org/view/82b8d31b
Here's the iommu output
IOMMU Group 13: 0000:01:00.0 VGA compatible controller [0300] NVIDIA Corporation AD102 [GeForce RTX 4070 Ti SUPER] [10de:2689] (rev a1)
Subsystem: Micro-Star International Co., Ltd. [MSI] AD102 [GeForce RTX 4070 Ti SUPER] [1462:5100]
Kernel driver in use: nvidia
IOMMU Group 13: 0000:01:00.1 Audio device [0403] NVIDIA Corporation AD102 High Definition Audio Controller [10de:22ba] (rev a1)
Subsystem: Micro-Star International Co., Ltd. [MSI] AD102 High Definition Audio Controller [1462:5100]
Kernel driver in use: vfio-pci
IOMMU Group 29: 0000:71:00.0 VGA compatible controller [0300] Advanced Micro Devices, Inc. [AMD/ATI] Granite Ridge [Radeon Graphics] [1002:13c0] (rev cb)
Subsystem: Micro-Star International Co., Ltd. [MSI] Granite Ridge [Radeon Graphics] [1462:7e51]
Kernel driver in use: amdgpu
huh
that might be why
actually that is why its not using the vfio driver instead of nvidia
i would say pin the current image and try rebase to the non nvidia image since youre technically done with the nvidia image
never tested the passthrough on the nvidia image since well usually you sacrifice the nvidia card for the vms
Oh, right. Forgot that might be an issue. I tested the nvidia image with just my igpu in and it worked so...
and yeah nvidia driver got angry when you plugged in the screan because it could not find its audio device anymore
I'll try a rebase.
pin the nvidia image first though
so you have a backup to boot back into to rollback to if for some reason it wont work
sudo ostree admin pin 0
to unpin you do sudo ostree admin pin --unpin 2 or rather the highest index from rpm-ostree status -v if you have more than 0 and 1Having trouble finding the command to rebase to AMD. I'll get there, you go get some sleep. I appreciate the help.
brh rebase bazzite or brh rebase bazzite-gnome depending on which DE you have
also can you try something firstsure
might need
rpm-ostree kargs --delete=rd.driver.pre=vfio_pci --append-if-missing=rd.driver.pre=vfio-pci
remember how i had - in a kernel arg when it was supposed to be _
might been mixing up these 2 kernel args, where the force preload of the vfio driver is supposed to have - while the other args should have _
and you might just been the first where this actually mattered in terms of having to force load the driver early 🤣
either case i am happy its caught if thats why the nvidia driver still snagged the card before vfioI break things a lot, there some enterprise gear that we just break because they don't expect us.
Trying now.
well i do this on the regular but it has just worked for me so i guess i have been lucky so far with that one, same with kyle 🤣
Hehe. It's writing the commit now.
once its applied and you reboot and run
ls-iommu -grk it should list vfio-pci for both audio and vga device for nvidiarebooting will let you know
like that
Looks like I'll need to rebase. Rebooted with the cable plugged in and same behavior
ok
time to pin then rebase 🙂
if that doesnt do it i am a bit unsure
I already pinned. But the iommu was identical to before
Is there a different command? It's asking to rebase me to the same nvidia-open:bazzite tree
oh whops
brh rebase bazzite:stable
forgot it uses the whole name:tag when switching images
so for gnome its bazzite-gnome:stableOh, if i just needed to go to stable then cool, I found that earlier just didn't trust it being that easy
as long as youre not switching image you can just use the tag (stable, testing, etc)
if youre switching image you need image name and tag
Oh well I need to get it off of the nvidia image don't I?
yep
meaning youre going to either bazzite:stable or bazzite-gnome:stable
(if you rather want the deck image, try that after you got the passthrough working since gamemode is gpu picky and will give a black screen or crash to desktop if it is angry)
Oh, ok if that'll take me off the nvidia I'll run it.
I have an actual deck if I want deck image. This is a daily driver that can game.
It's chewing through the rebase now
IOMMU Group 13: 0000:01:00.0 VGA compatible controller [0300] NVIDIA Corporation AD102 [GeForce RTX 4070 Ti SUPER] [10de:2689] (rev a1)
Subsystem: Micro-Star International Co., Ltd. [MSI] AD102 [GeForce RTX 4070 Ti SUPER] [1462:5100]
Kernel driver in use: vfio-pci
IOMMU Group 13: 0000:01:00.1 Audio device [0403] NVIDIA Corporation AD102 High Definition Audio Controller [10de:22ba] (rev a1)
Subsystem: Micro-Star International Co., Ltd. [MSI] AD102 High Definition Audio Controller [1462:5100]
Kernel driver in use: vfio-pci
IOMMU Group 29: 0000:71:00.0 VGA compatible controller [0300] Advanced Micro Devices, Inc. [AMD/ATI] Granite Ridge [Radeon Graphics] [1002:13c0] (rev cb)
Subsystem: Micro-Star International Co., Ltd. [MSI] Granite Ridge [Radeon Graphics] [1462:7e51]
Kernel driver in use: amdgpu
huzzah!
About to try virt manager
does it freeze if you plug the gpu into the monitor now?
No it's likely working. I have had the monitor plugged the whole boott cycle
yeah then its good
Thanks so much. I'll have to figure out how to get it working how I want if possible from here.
The VM at least boots with virtio and is outputting a display
if you get a dummy plug and setup looking-glass you will love it 😉
unless you need multiple monitors for the VM
idk how well looking-glass handles that
Nah one should be fine. I was originally hoping it'd be able to just output straight to the monitor for adaptive sync but I never saw anything where that was possible.
looking-glass + dummy plug for the gpu
even lets you do FSR on everything in the VM too
Do you have that VM running on another separate dedicated monitor then?
dummy plug
its a small hdmi or displayport plug that looks like an usb and it just tells the card "hey im a monitor i support these resolutions up to 4K@14hz" or something
then i can use something like CRU or nvidia control panel to make a custom resolution with whatever refresh rate i want
so if i wanted to i could make it go 4k@180
Yeah I've seen them. I was just thinking you were saying you used one to do what I was saying.
but by default its hardcoded to i think 1080@60, 1440@24 and 4k@14
no the VM screen is displayed by looking-glass
which just shows you the framebuffer in realtime using kvmfr (which is included in bazzites kernel)
so the "monitor" is just an application you open
no need for a physical monitor dedicated for the VM
Using Looking-Glass - Bazzite Documentation
Bazzite is a custom image built upon Fedora Atomic Desktops that brings the best of Linux gaming to all of your devices.
I get that. But I was trying to avoid needing another adapter for the second monitor I'm going to use to view the VM while having my browser open on the other display.
the adapter is for the gpu
Thanks again.
not the monitor
it frees up the monitor to be used by your host gpu
if you rather have a dedicated VM monitor however then you might want a dedicated mouse and keyboard for the VM too, or a software kvm switch
I understand. I just need an adapter to hook to my host GPU now so that will work. Motherboard only has an HDMI and G-Sync only works on Displayport, so I was hoping to just figure out a way to directly view the output from the VM on the monitor attached to the GPU.
you can only view the VM output on the monitor that is attached to the gpu in the VM and there is no seamless way to move keyboard and mouse quickly between the 2
https://github.com/HikariKnight/vfio-setup-docs/wiki/3.-Quality-of-life-tweaks#software-based-kvm-switch
^ would be the best bet
the other solution is getting a dummy plug you put into the gpu that is in the VM and then just setup looking-glass (assuming the VM is windows) and view the framebuffer directly, this gives you seamless access to the monitor and able to seamlessly use the keyboard and mouse in the VM whenever you focus looking-glass.
for gsync you need a physical monitor attached to the card (so looking-glass is out of the question)
for freesync however you can utilize freesync on the host and it will be applied for looking-glass too if your host has a freesync capable monitor that is good.
GitHub
3. Quality of life tweaks
Repo containing the PDF for my detailed vfio configuration setup and performance tweaking. - HikariKnight/vfio-setup-docs
anyways im going to grab some shut eye now
Thanks
@phazer11 after todays update (released earlier today) you should be able to just disable initramfs regeneration with
rpm-ostree initramfs --disable
if you ran rpm-ostree kargs --append-if-missing=kvmfr.static_size_mb=128
if not run that first then disable initramfs regenerationOh will do. Compiled Looking Glass but haven't gotten it working yet so I took a break from this project to bang my head on Cyberpunk which won't boot with Redmod installed via Heroic after the update, fun times.
Update done and yay I can get rid of my USB to Ethernet dongle because my LAN port works now.
@HikariKnight got my dummy plug last night. Thought I had Looking-glass made but apparently it's not working right? looking-glass-client
00:00:00.000 [I] main.c:1860 | main | Looking Glass (B7-rc1-34-ge25492a3a3)
00:00:00.000 [I] main.c:1861 | main | Locking Method: Atomic
00:00:00.000 [I] cpuinfo.c:38 | cpuInfo_log | CPU Model: AMD Ryzen 7 9800X3D 8-Core Processor
00:00:00.000 [I] cpuinfo.c:39 | cpuInfo_log | CPU: 1 sockets, 8 cores, 16 threads
00:00:00.020 [I] main.c:1185 | lg_run | Using font: /usr/share/fonts/google-noto/NotoSansMono-Regular.ttf
00:00:00.021 [I] ivshmem.c:137 | ivshmemOpenDev | KVMFR Device : /dev/kvmfr0
00:00:00.028 [I] audio.c:159 | audio_init | Using AudioDev: PipeWire
00:00:00.028 [I] ps.c:245 | purespice_connect | Connecting to socket 127.0.0.1:5900
00:00:00.028 [E] channel.c:99 | channel_connect | Socket connect failed
00:00:00.028 [E] ps.c:264 | purespice_connect | channel connect failed
00:00:00.028 [E] main.c:1044 | spiceThread
Did you add the kvmfr device to the virtual machine?
Not sure which you mean. I followed. https://looking-glass.io/docs/B6/install/
https://looking-glass.io/docs/B6/module/#libvirt
And the config file part of this (everything above the config we already did for you)
https://looking-glass.io/docs/B6/module/#usage
pretty sure I've done everything specified in those now and still the same result after rebooting host and vm.
gpus pci is added to the VM
the VM has the driver and works using a remote desktop solution like vnc?
do you have multiple VMs?
as you might have to set a custom port if you run multiple vms (as auto in libvirt will use the first free port after and including 5900 and looking-glass does not know the port magically)
i use 6900 for this reason as my passthrough vms spice port
No only one vm.
Not sure how I'd VNC into a locally hosted VM
I'll see is 127.0.0.1:6900 works
need to set that in looking-glass ini and in libvirt for the VM
What do I put in the .ini? Only thing there right now is: [app]
shmFile=/dev/kvmfr0
looking-glass-client --help
example to convert this to an ini setting
so: [spice] port=6900
the VM also needs to still have the spice display, that way looking-glass uses that as a fallback display
the way i did it with newline too
yeah, didn't know how to do newline in discord
shift+enter
also
```ini
text
```
will put the text in a codeblock with ini highlighting
I used to know the ini thing but it's been so long since I needed to do that. Man I'm so glad we can do stuff like this now, I hated trying to do stuff in IRC back in the day.
can you also give me a
sudo dmesg | grep kvmfrI do notice that static size is not what it was earlier when I ran dmesg, should be 256
try 128 as HDR doesnt work with looking-glass yet IIRC since everything HDR has to run through gamescope
which is why we have that as default
I'll try re-running the qemu args then
also just FYI
[ 9.998594] systemd[1]: Unable to fix SELinux security context of /dev/kvmfr0: Permission denied is a red herring, selinux fixes it once the system is booted
rpm-ostree kargs 🙂Yeah, figured as much with the selinux
yeah it uses a custom type enforcement which is applied later in the boot 🙃
resume=UUID=f81ba298-29c6-452f-bc01-cef4fd989757 rhgb quiet root=UUID=cdb31415-20c1-4cf2-8b6f-3238a5ec7bb3 rootflags=subvol=root rw ostree=/ostree/boot.0/default/5565b804220e6ae0ee4720650b38586927b61f886b9531f9eaf14a27101a3ba3/0 bluetooth.disable_ertm=1 preempt=full kvm.ignore_msrs=1 kvm.report_ignored_msrs=0 amd_iommu=on iommu=pt vfio_pci.disable_vga=1 vfio_pci.ids=10de:2689,10de:22ba rd.driver.pre=vfio-pci kvmfr.static_size_mb=128
ok so its already 128mb
the module is working then
so issue would be with the VM config if i were to guess
Hmm...
plus this in the qml at the bottom above and yes those encoded quotes are correct
as weird as it is
</domain>
Yeah I noticed it kept converting the actual quotes to that.
just make sure the size matches the MB you set for kvmfr but in bytes
size_in_MB x 1024 x 1024 = size_in_bytesWoah. I just copied your qemu commandline in (not that I could tell it was any different) and it just removed the lines when I hit applied.
you maybe had the size wrong?
or you copied the one for older qemu version when looking at the documentation
No size was right
idk what else to say
so it works now?
No
hmm sec then its just messing when changing it
try that instead?
other than that not sure
that actually introduced an error
what error
and it goes away when you remove that? because that seems like an error from the disk image
That's the thing all the commandline args disappear from the XML
from the looking-glass documentation i linked to
* Remember to add
xmlns:qemu='http://libvirt.org/schemas/domain/qemu/1.0' to the <domain> tag.
<domain xmlns:qemu="http://libvirt.org/schemas/domain/qemu/1.0" type="kvm">
at the very top
otherwise it will not allow qemu commandsI did. but it's gone now and like the commandline args disappears when I hit apply. I'm starting to think this vm is possessed.
I'll mess with this in a new VM next year (which means tomorrow for me). Happy New Year @HikariKnight!
if you apply after adding the domain part before adding any qemu commands then it removes the domain part because you dont need it
SO then I guess it's also removing the qemu args part for the same reason. IDK I still think I need to start the VM from scratch. Which page of docs do you recommend for that?
make the VM, make sure windows works on it (add vnc server to it so you can connect to it and verify things)
then do the passthrough
verify the VM sees the card through VNC and get it working
then setup looking-glass and add the domain xmlns and the qemu commands at the same time to the domain tag then apply
if you rather add the xml stuff through cli you can use
virsh edit vmdomainname and it might give better output if something in the xml is wrong@HikariKnight Happy New Year! Working on the new VM and it keeps telling me that the VNC needs a GL Context. I'm still trying to google it but maybe you have an idea.
I originally thought it had something to do with needing another display but...
no install the vnc server in the vm
Oh
the server in the VM wont have the same issues as the one through libvirt
since as soon as the passthrough gpu gets it driver, you wont have output on the vnc display through libvirt
or you will have display but the cursor location is scaled to match the display in the VM and the gpu
so it will be very wonky
What are you using for NIC mode, bridge, macvtap or something else?
nat
should work fine with that
just check the ip or set it as static in the VM within the nat lan
host will have access to it
macvtap is for easy bridge mode (but you lose host <--> vm network communication)
Yeah I use MacVTap at work for the multiport NIC I have at the office, I do a bunch with VMs on a Fedora Kinoixe host which is why I'm so surprised I'm having an issue with this. Never seen some of this behavior. Will try NAT mode again, was having issues connecting to the VNC server it was acting like it was on a nother subnet
if you dont want to go through setting up a bridge you can make a host only network
https://github.com/HikariKnight/vfio-setup-docs/wiki/1.-Setting-everything-up#local-host-only-connection-between-host-and-guest
GitHub
1. Setting everything up
Repo containing the PDF for my detailed vfio configuration setup and performance tweaking. - HikariKnight/vfio-setup-docs
I'm re-re-building this stupid VM. Bunch of just weird stuff, gave me a memory allocation error so I might need to know how to increase my user's allowance here in Bazzite. Will let you know how it goes.
OK VNC working now. Now to try the rest of the VM config. I'm gonna scream and probably disable vfio for the time being if this doesn't work
just remove the vfio binding kargs to disable it if you have to 🙂
@HikariKnight '''ini
looking-glass-client -f /dev/kvmfr0
00:00:00.000 [I] main.c:1860 | main | Looking Glass (B7-rc1-34-ge25492a3a3)
00:00:00.000 [I] main.c:1861 | main | Locking Method: Atomic
00:00:00.000 [I] cpuinfo.c:38 | cpuInfo_log | CPU Model: AMD Ryzen 7 9800X3D 8-Core Processor
00:00:00.000 [I] cpuinfo.c:39 | cpuInfo_log | CPU: 1 sockets, 8 cores, 16 threads
00:00:00.012 [I] main.c:1185 | lg_run | Using font: /usr/share/fonts/google-noto/NotoSansMono-Regular.ttf
00:00:00.013 [I] ivshmem.c:137 | ivshmemOpenDev | KVMFR Device : /dev/kvmfr0
00:00:00.015 [I] audio.c:159 | audio_init | Using AudioDev: PipeWire
00:00:00.015 [I] ps.c:245 | purespice_connect | Connecting to socket 127.0.0.1:5900
00:00:00.016 [E] channel.c:413 | channel_readNL | Failed to read from the socket: -1
00:00:00.016 [E] channel.c:118 | channel_connect | Failed to read the reply to the connect packet
00:00:00.016 [E] ps.c:264 | purespice_connect | channel connect failed
00:00:00.016 [E] main.c:1044 | spiceThread | Failed to connect to spice server
'''
did you set the spice port for the VM?
yes
set the spice port in the ini for looking-glass
Connecting to socket 127.0.0.1:5900
Oh. That's what VNC is on right now...
Let's see if that helps.
looking-glass does not use vnc, it uses the spice port
No I'm saying VNC is running on that port right this moment
No lookingglass ini has 6900 wth
make sure it matches the libvirt port
the ini should be
~/.config/looking-glass/client.ini
so if its right in the ini file but its in the wrong location it does nothingyeah, I was following the directions which said to put it right next to the client but their directions was the wrong location. Facepalm. Consequences of no sleep and frustration. I hate my neighbors fireworks. Been going off all week for hours every night.
Looks like it's running now.
Is there a way to auto run the host app on boot of vm?
I'll work on this more tomorrow. Finally no fireworks gonna try to sleep. Night.
the installer literally sets up a service by default
@HikariKnight about to have to leave but it seems its saying the issue is it can't find the NvFBC64.dll so I'm guessing I'll need to switch to DXGI for now and figure out how to get that dll.
dxdgi is the recommended for looking-glass b7 and newer
b6 does not need it but is a bit higher latency
plus you said it worked
I grabbed b6 since that's the stable. I was premature, client in linux was reporting up
Getting that message above
install b6 host in VM
use b6 client on linux
Ok, was a client mismatch. In my sleep deprived haze I forgot I compled/installed rc-bc7 linux client for the D12 support not the stable and used the stable host. Still some odd behavior like not being able to open any windows in the vm the programs are open but will not come to foreground. Need to change the resolution from 1920x1080 somehow too but progress, now I'm not in shopping hell.
Go into display settings on the VM and turn off the qxl monitor
It gets re enabled again if you remove the passthrough card
@HikariKnight There's no second monitor just the Display Spice and Video VGA.
in windows
video = virtual video card
you need that for spice to work and for the spice fallback for looking-glass to work
but you want the display for that card in windows to be off
so only the passthrough cards display is active
also video should be QXL
Yeah I set it so the Dummy plug display was not showing anything.
its the QXL display you want to be off
the dummy plug you want on
Ok just set that up last night so I could keep setting things up in virt-managers window since looking glass was not lettinng things come up. It's set to the dummy plug now via vnc. I'll see how that goes. So it needs to be QXL and not VGA? Was pretty sure the guide said VGA.
Might be good now. Just gotta figure out how to connect a portable hard drive to transfer some games. Redirection is not working
redirection should work or pass through a whole usb controller.
alternative is to make 10000000000% sure the disk is not mounted in linux and just add the device node
/dev/disk/by-id/blah (do not use /dev/sdX as that might change by next time you connect it)
if the disk is mounted when you add the device node as a raw disk, you will corrupt the whole drive since both systems will access it and not know about the other systemIs there a way to do regular redirection from the looking glass viewer? @HikariKnight
no because looking-glass is for viewing the framebuffer and controlling the vm
Thought as much .
redirection has to go through virt-manager
Yeah. That's what I've been trying
So, it might be just not recognizing my NVME to USB enclosure on the VM? See one device that says invalid device descriptor. Never had any issues on a Windows 10 VM so that'd be weird but we'll see trying a thumb drive now.
some usb devices do funky stuff so redirection does not work properly
in those cases you need to passthrough the whole controller (meaning you pass through the ports directly and its always a group of 2 or 4)
and that follows the same rules as with the gpu
cant share IOMMU group with anything else
last resort would be adding the device node (the disk block) as a raw disk do the VM but you need to make damn sure that the host does not mount the partition at any point while the VM is running unless you want to nuke the disk
I'm gonna try a usb drive if it ever finishes copying and then if that works try my other nvme enclosure. Never had a problem redirecting that enclosure that's not showing up using virt-manager before but eh.
as i said it depends on how the usb device is made
some do funky stuff before being fully connected
here is an example of a controller doing funky stuff
keep an eye on device number 2 in the list when it connects
hmm
that controller is the least problematic
there are others that connect, disconnect, then connects again
making redirection impossible
Thinking that's what's going on with this port.
Not sure what is up but performance is not what I'm expecting. Does not come close to what I was getting in the same game running through proton.
10 fps with same settings vs like 70 before.
definitely something wrong, raytracing on, raytracing off 10 FPS.
Also noticing a lot of flickering when focused on the VM through looking glass. If I just have the VM on my other monitor but say I'm typing here no flicker.
Edit: Flickering only seems to happen when fullscreened.
the flickering can be if you have VRR enabled and your panel has a bad implementation of it, pretty common
Yeah not sure. More concerned by the performance at the moment. Cannot figure out why it's so bad,m other than a CPU or storage bottleneck from going through VM I guess but this should be overkill.
@HikariKnight Yes the flicker is likely my monitors VRR implementation, can't figure out how to even turn VRR on in windows (I'm thinking because it's a Nvidia GPu and I only have HDMI atm) so I might as well disable it for VM use. No ideas on the performance yet.
you need to turn VRR off on your host
windows has no idea of it existing because looking-glass is what renders windows in the end
the fps however make sure your gpu is in the device manager and has no issues and the driver is installed
Nvidia Driver is installed, it sees it fine, detects it as what it is, that's why I'm confused.
then it could be as simple as you needing to actually optimize the VM config
like making sure the X amount of cores you have given it are correctly allocated as cores and threads instead of sockets
among other things
this is something you will have to figure out yourself though since it all depends on your system
all i can say though is if your cpu is amd you will need this in the <cpu></cpu> section
<feature policy="require" name="topoext"/>Yep
yep anything else you will have to figure out since it varies what you will need
like setting up hugepages (reserving memory for the vm, this memory will always be in use and inaccessible for non VMs), or pinning cpu cores
this should be in the
<domain></domain> section and you should run lstopo (from the hwloc package) to see your topology to plan out how you are going to pin these cores
then you can also setup cpu core isolation through systemd to stop linux from using those cores
thats the bare minimum universal things you can do to potentially increase performanceI'll look into that some more. To revert I just need to yank the ostree kargs right?
to unbind the card from the vfio driver yes
if you want to boot the VM without the gpu you need to just remove the pci devices that belongs to the gpu (it will then enable the QXL display again so you can view the VM screen through virt manager)
and once you add the pci devices for the gpu back it will disable the QXL screen again