Hi! I'm working on top of the zmq_samples example, trying to set up 2 Ethernet devices on 2 of the 3 VM's. The passthrough for both VM's has been done, and both detect the NICs. For more information, this is the setup for the irqs in the camkes file: vm0.vm_irqs = [ {"name":"inner_eth", "ioapic":0, "source":0xb, "level_trig":1, "active_low":1, "dest":10} ]; vm2.vm_irqs = [ {"name":"outer_eth", "ioapic":0, "source":0xb, "level_trig":1, "active_low":1, "dest":12} ]; My problem is related to having the same IRQ11 in both e1000e devices. I can't control this; that is what QEMU provides. When I set up the first NIC in any VM, say VM0, doing ifconfig eth1 up, it works correctly, but when I try to bring up the other one, VM2, the whole system freezes for a moment. Then VM0 complains about this: [ 48.557120] irq 10: nobody cared (try booting with the "irqpoll" option) [ 48.557120] CPU: 0 PID: 0 Comm: swapper/0 Tainted: G W 4.9.337+ #19 [ 48.557120] df811f94 d08f1778 df807d00 00000000 df811fa8 d08f08e0 df807d00 00000000 [ 48.557120] d0459f68 df811fc4 d04596ec 0000000b 00000000 df807d00 00000000 d0459f68 [ 48.557120] df811fd8 d0457a55 00000000 df807d00 df807d64 df811fe8 d0457a7e df807d00 [ 48.557120] Call Trace: [ 48.557120] [ 48.557120] [] dump_stack+0x4f/0x63 [ 48.557120] [] __report_bad_irq+0x33/0x98 [ 48.557120] [] ? unmask_irq+0x22/0x22 [ 48.557120] [] note_interrupt+0x185/0x1c3 [ 48.557120] [] ? unmask_irq+0x22/0x22 [ 48.557120] [] handle_irq_event_percpu+0x31/0x38 [ 48.557120] [] handle_irq_event+0x22/0x3b [ 48.557120] [] handle_level_irq+0x52/0x80 [ 48.557120] [] handle_irq+0x91/0xb6 [ 48.557120] [ 48.557120] [ 48.557120] [] do_IRQ+0x33/0x99 [ 48.557120] [] common_interrupt+0x2c/0x34 [ 48.557120] [] ? __do_softirq+0x55/0x1a1 [ 48.557120] [] ? _local_bh_enable+0x6c/0x6c [ 48.557120] [] call_on_stack+0x40/0x46 [ 48.557120] [ 48.557120] [] ? irq_exit+0x56/0x91 [ 48.557120] [] ? do_IRQ+0x7f/0x99 [ 48.557120] [] ? common_interrupt+0x2c/0x34 [ 48.557120] [] ? default_idle+0x16/0x18 [ 48.557120] [] ? arch_cpu_idle+0xd/0xf [ 48.557120] [] ? default_idle_call+0x14/0x16 [ 48.557120] [] ? cpu_startup_entry+0xbb/0x158 [ 48.557120] [] ? rest_init+0x5d/0x5f [ 48.557120] [] ? start_kernel+0x339/0x343 [ 48.557120] [] ? i386_start_kernel+0x3e/0x40 [ 48.557120] handlers: [ 48.557120] [] e1000_intr [ 48.557120] Disabling IRQ #10 Any interruption goes to VM2; they all go to VM0 but are not intended to go to that one, so they are ignored. Any thoughts on this would be appreciated!
Hello Alberto, First of all, does it work with only one vm config enabled? That is, first test with only vm0 and check if it works, then check with only vm2. Don't try combining them before testing each individually. That limits the number of problems you have to solve at once. On 2025-05-13 14:04, Alberto Lange via Devel wrote:
The passthrough for both VM's has been done, and both detect the NICs. For more information, this is the setup for the irqs in the camkes file: vm0.vm_irqs = [ {"name":"inner_eth", "ioapic":0, "source":0xb, "level_trig":1, "active_low":1, "dest":10} ]; vm2.vm_irqs = [ {"name":"outer_eth", "ioapic":0, "source":0xb, "level_trig":1, "active_low":1, "dest":12} ];
My problem is related to having the same IRQ11 in both e1000e devices. I can't control this; that is what QEMU provides.
Your problem is that both NICs share one IRQ. This a QEMU configuration problem, you want different interrupts. Perhaps you can achieve this by adding the card to different PCI roots, or with some other options. If not, you can work around this problem in the VM monitor, but you will need to modify the code: When receiving interrupt 11, you need to check which NIC caused it. And then inject the IRQs into the VMs that have the corresponding NIC (which could be both).
Any interruption goes to VM2; they all go to VM0 but are not intended to go to that one, so they are ignored.
This is because you are passing the same IOAPIC source twice, I don't think the code expect you to do that. It's probably not possible to get two IOAPIC handlers for the same IRQ in seL4 and your config triggers some bugs in the camkes VMM code (it should give at least an error message). It is strange that vm0 is complaining about IRQ 10 though, it's what you configured. Does networking work for vm0 if you pass it as IRQ 11? Greetings, Indan
Hello Indan, Thanks for the quick response! I've tried with both VMs individually, and they work fine. Some of the curious things that I found doing more tests are that: - Changing the level_trig option to 0 resolves both the freeze and error that appear, but one of the VMs still doesn't work with the NIC. - I found that the working VM, the one that works well with the NIC, is the one that has the major "dest" number. Maybe it is a sort of priority thing in seL4 sharing IRQ11 for both VMs. I was able to have a different IRQ to each NIC in QEMU by using different addresses for the devices (0x3 and 0x4), and both VMs worked perfectly, even when using the same IOAPIC. Here is the QEMU command I used: sudo qemu-system-x86_64 \ -machine q35,accel=kvm \ -cpu host \ -m 2G \ -kernel camkes-vm-examples-manifest/build_zmq/images/kernel-x86_64-pc99 \ -initrd camkes-vm-examples-manifest/build_zmq/images/capdl-loader-image-x86_64-pc99 \ -nographic \ -serial mon:stdio \ -device intel-iommu,intremap=off \ -device e1000e,mac=00:00:00:00:00:01,id=network0.0,netdev=network0,addr=0x3 \ -netdev tap,ifname=tap_inA,id=network0,script=no,downscript=no \ -device e1000e,mac=00:00:00:00:00:02,id=network1.0,netdev=network1,addr=0x4 \ -netdev tap,ifname=tap_outA,id=network1,script=no,downscript=no Greetings, Albert
participants (2)
-
Alberto Lange -
Indan Zupancic