forward mode open is adding libvirt iptables rules

Bug #1971619 reported by Jeff
8
This bug affects 1 person
Affects Status Importance Assigned to Milestone
libvirt (Ubuntu)
Invalid
Undecided
Unassigned

Bug Description

After "auto security updates" updated my libvirt I have noticed that forward mode open is adding LIBVIRT rules to my iptables for the default network. This was supposed to happen with forward mode nat, but not with forward mode open.

apt-cache policy libvirt-daemon:
libvirt-daemon:
  Installed: 6.0.0-0ubuntu8.16
  Candidate: 6.0.0-0ubuntu8.16
  Version table:
 *** 6.0.0-0ubuntu8.16 500
        500 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 Packages
        500 http://archive.ubuntu.com/ubuntu focal-security/main amd64 Packages
        100 /var/lib/dpkg/status
     6.0.0-0ubuntu8 500
        500 http://archive.ubuntu.com/ubuntu focal/main amd64 Packages

lsb_release -rd:
Description: Ubuntu 20.04.3 LTS
Release: 20.04

VM network settings:
virsh net-dumpxml --inactive default
<network>
  <name>default</name>
  <uuid>17c58686-736c-49e4-8ae9-99a8d25f032c</uuid>
  <forward mode='open'/>
  <bridge name='virbr0' stp='on' delay='0'/>
  <mac address='52:54:00:f6:68:91'/>
  <ip address='192.168.122.1' netmask='255.255.255.0'>
    <dhcp>
      <range start='192.168.122.2' end='192.168.122.254'/>
    </dhcp>
  </ip>
</network>

What I expect to happen:
Because I have forward mode='open' I expect that when libvirtd gets restarted that it will NOT load its own rules into my firewall. This has been the case for me since mode open was added.

What is happening:
(staging) root@server:~$ iptables-save|grep -i virt
(staging) root@server:~$ service libvirtd restart
(staging) root@server:~$ iptables-save|grep -i virt
:LIBVIRT_PRT - [0:0]
-A POSTROUTING -j LIBVIRT_PRT
-A LIBVIRT_PRT -o virbr0 -p udp -m udp --dport 68 -j CHECKSUM --checksum-fill
:LIBVIRT_PRT - [0:0]
-A POSTROUTING -j LIBVIRT_PRT
-A LIBVIRT_PRT -s 192.168.122.0/24 -d 224.0.0.0/24 -j RETURN
-A LIBVIRT_PRT -s 192.168.122.0/24 -d 255.255.255.255/32 -j RETURN
-A LIBVIRT_PRT -s 192.168.122.0/24 ! -d 192.168.122.0/24 -p tcp -j MASQUERADE --to-ports 1024-65535
-A LIBVIRT_PRT -s 192.168.122.0/24 ! -d 192.168.122.0/24 -p udp -j MASQUERADE --to-ports 1024-65535
-A LIBVIRT_PRT -s 192.168.122.0/24 ! -d 192.168.122.0/24 -j MASQUERADE
:LIBVIRT_FWI - [0:0]
:LIBVIRT_FWO - [0:0]
:LIBVIRT_FWX - [0:0]
:LIBVIRT_INP - [0:0]
:LIBVIRT_OUT - [0:0]
-A INPUT -j LIBVIRT_INP
-A FORWARD -j LIBVIRT_FWX
-A FORWARD -j LIBVIRT_FWI
-A FORWARD -j LIBVIRT_FWO
-A OUTPUT -j LIBVIRT_OUT
-A LIBVIRT_FWI -d 192.168.122.0/24 -o virbr0 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A LIBVIRT_FWI -o virbr0 -j REJECT --reject-with icmp-port-unreachable
-A LIBVIRT_FWO -s 192.168.122.0/24 -i virbr0 -j ACCEPT
-A LIBVIRT_FWO -i virbr0 -j REJECT --reject-with icmp-port-unreachable
-A LIBVIRT_FWX -i virbr0 -o virbr0 -j ACCEPT
-A LIBVIRT_INP -i virbr0 -p udp -m udp --dport 53 -j ACCEPT
-A LIBVIRT_INP -i virbr0 -p tcp -m tcp --dport 53 -j ACCEPT
-A LIBVIRT_INP -i virbr0 -p udp -m udp --dport 67 -j ACCEPT
-A LIBVIRT_INP -i virbr0 -p tcp -m tcp --dport 67 -j ACCEPT
-A LIBVIRT_OUT -o virbr0 -p udp -m udp --dport 53 -j ACCEPT
-A LIBVIRT_OUT -o virbr0 -p tcp -m tcp --dport 53 -j ACCEPT
-A LIBVIRT_OUT -o virbr0 -p udp -m udp --dport 68 -j ACCEPT
-A LIBVIRT_OUT -o virbr0 -p tcp -m tcp --dport 68 -j ACCEPT

Revision history for this message
Jeff (jeffck) wrote :

I also filed this bug directly with the libvirt team at: https://gitlab.com/libvirt/libvirt/-/issues/307

Revision history for this message
Jeff (jeffck) wrote :

I was able to confirm this bug exists in 6.0.0-0ubuntu8.15 as well.

Revision history for this message
Paride Legovini (paride) wrote :

Thanks Jeff for filing this bug report. To work on this it would greatly help to be able to pinpoint which Focal update introduced the problem. One way to do this is to download one version after the other, install the packages and test if the bug is present. Something along these lines:

  apt install --no-install-recommends ubuntu-dev-tools
  dpkg -l | grep libvirt # check which libvirt pkgs are installed

  pull-lp-debs libvirt 6.0.0-0ubuntu8.9 # for example
  dpkg -i <list of ubuntu8.9 .deb files for packages you have installed>

Then: verify if bug is present. If not present >= proceed with version ubuntu8.10. If present, rollback the the *previous* version and test again to be really sure the buggy version has been found. (Note: this shouldn't be harmful but it obviously not the recommended way to manage packages, do this on a test system.)

Do you think you can perform this testing and let us know your findings?

While waiting for more info I'm marking this as Incomplete for now.

Thanks!

Changed in libvirt (Ubuntu):
status: New → Incomplete
Revision history for this message
Jeff (jeffck) wrote :

Thank you for the instructions on how to test old versions. I was wrong above when i said it was not present in 8.8, indeed it was but I had not noticed it until more recently. I have tested back to 8.1 and I confirm it was there as well. I am unable to get/test 8.0 and i do not know what the version before that is.

$ pull-lp-debs libvirt 6.0.0-0ubuntu8.0
Source package lookup failed, trying lookup of binary package libvirt
The binary package 'libvirt' version 6.0.0-0ubuntu8.0 does not exist in the Ubuntu primary archive for architecture amd64
The source package 'libvirt' version 6.0.0-0ubuntu8.0 does not exist in the Ubuntu primary archive

Can I help more?

Revision history for this message
Jeff (jeffck) wrote :

I found it was 6.0.0-0ubuntu8 and I am triggering the bug even while that version is installed, which feels wrong... How would I be the first to notice this after 2 years? it seems unlikely, but here we are

Revision history for this message
Jeff (jeffck) wrote :
Download full text (3.2 KiB)

It seems to be present all the way back to 6.0.0-0ubuntu2 which isas far as i could find in an old changelog... is it possible im not rolling the proper packages back?
I have been rolling back these debs:

libvirt0_6.0.0-0ubuntu2_amd64.deb
libvirt-daemon-driver-qemu_6.0.0-0ubuntu2_amd64.deb
libvirt-daemon-system-systemd_6.0.0-0ubuntu2_amd64.deb
libvirt-clients_6.0.0-0ubuntu2_amd64.deb
libvirt-daemon-driver-storage-rbd_6.0.0-0ubuntu2_amd64.deb
libvirt-dev_6.0.0-0ubuntu2_amd64.deb
libvirt-daemon_6.0.0-0ubuntu2_amd64.deb
libvirt-daemon-system_6.0.0-0ubuntu2_amd64.deb

(staging) root@server:~$ iptables-save|grep -i virt
(staging) root@server:~$ service libvirtd restart
(staging) root@server:~$ iptables-save|grep -i virt
:LIBVIRT_FWI - [0:0]
:LIBVIRT_FWO - [0:0]
:LIBVIRT_FWX - [0:0]
:LIBVIRT_INP - [0:0]
:LIBVIRT_OUT - [0:0]
-A INPUT -j LIBVIRT_INP
-A FORWARD -j LIBVIRT_FWX
-A FORWARD -j LIBVIRT_FWI
-A FORWARD -j LIBVIRT_FWO
-A OUTPUT -j LIBVIRT_OUT
-A LIBVIRT_FWI -o virbr0 -j REJECT --reject-with icmp-port-unreachable
-A LIBVIRT_FWO -i virbr0 -j REJECT --reject-with icmp-port-unreachable
-A LIBVIRT_FWX -i virbr0 -o virbr0 -j ACCEPT
-A LIBVIRT_INP -i virbr0 -p udp -m udp --dport 53 -j ACCEPT
-A LIBVIRT_INP -i virbr0 -p tcp -m tcp --dport 53 -j ACCEPT
-A LIBVIRT_INP -i virbr0 -p udp -m udp --dport 67 -j ACCEPT
-A LIBVIRT_INP -i virbr0 -p tcp -m tcp --dport 67 -j ACCEPT
-A LIBVIRT_OUT -o virbr0 -p udp -m udp --dport 53 -j ACCEPT
-A LIBVIRT_OUT -o virbr0 -p tcp -m tcp --dport 53 -j ACCEPT
-A LIBVIRT_OUT -o virbr0 -p udp -m udp --dport 68 -j ACCEPT
-A LIBVIRT_OUT -o virbr0 -p tcp -m tcp --dport 68 -j ACCEPT
:LIBVIRT_PRT - [0:0]
-A POSTROUTING -j LIBVIRT_PRT
:LIBVIRT_PRT - [0:0]
-A POSTROUTING -j LIBVIRT_PRT
-A LIBVIRT_PRT -o virbr0 -p udp -m udp --dport 68 -j CHECKSUM --checksum-fill
(staging) root@server:~$ iptables-restore < /etc/iptables/rules.v4
(staging) root@server:~$ dpkg -l | grep libvirt
ii libvirt-clients 6.0.0-0ubuntu2 amd64 Programs for the libvirt library
ii libvirt-daemon 6.0.0-0ubuntu2 amd64 Virtualization daemon
ii libvirt-daemon-driver-qemu 6.0.0-0ubuntu2 amd64 Virtualization daemon QEMU connection driver
ii libvirt-daemon-driver-storage-rbd 6.0.0-0ubuntu2 amd64 Virtualization daemon RBD storage driver
ii libvirt-daemon-system 6.0.0-0ubuntu2 amd64 Libvirt daemon configuration files
ii libvirt-daemon-system-systemd 6.0.0-0ubuntu2 amd64 Libvirt daemon configuration files (systemd)
ii libvirt-dev:amd64 6.0.0-0ubuntu2 amd64 development files for the libvirt library
ii libvirt-glib-1.0-0:amd64 3.0.0-1 amd64 libvirt GLib and GObject mapping library
ii libvirt0:amd64 6.0.0-0ubuntu2 amd64 library for interfacing with different virtualization systems
ii python3-libvirt 6.1.0-1 ...

Read more...

Revision history for this message
Lena Voytek (lvoytek) wrote :

Hello Jeff,

From your comment it looks like you rolled back the packages correctly. When testing did you remove the existing rules libvirt provided before restarting libvirtd? If not they may have been left over from the previous rules load. Otherwise it may be possible that a new version of a different package is causing the issue.

Thanks

Revision history for this message
Jeff (jeffck) wrote :

Thank you for the reply. I am certain that I removed the rules between each test. I agree that it could be a different package, I better go through my dpkg log and figure out whats going on here. Is there any way somebody could confirm that they do not see this?

Revision history for this message
Sergio Durigan Junior (sergiodj) wrote :

Thanks for the followup, Jeff.

I tried to reproduce the bug locally but failed. Here are the steps I did:

1) Inside a Focal test environment, installed all the libvirt packages mentioned by you.

2) Verified that the iptables rules added by libvirt are also present, as in your case.

3) Edited (virsh net-edit --network default) the XML file and changed the forward mode to "open", and restarted the libvirt service.

4) Noticed that the iptables rules are *still* present and exactly as they were before the libvirt restart. This is because, as Lena explained, these rules need to be cleaned up manually.

5) Restarted the machine in order to guarantee a clean environment.

6) Verified that the libvirt service is still active, but now the iptables rules are:

# iptables-save | grep -i virt
:LIBVIRT_PRT - [0:0]
-A POSTROUTING -j LIBVIRT_PRT
:LIBVIRT_PRT - [0:0]
-A POSTROUTING -j LIBVIRT_PRT
:LIBVIRT_FWI - [0:0]
:LIBVIRT_FWO - [0:0]
:LIBVIRT_FWX - [0:0]
:LIBVIRT_INP - [0:0]
:LIBVIRT_OUT - [0:0]
-A INPUT -j LIBVIRT_INP
-A FORWARD -j LIBVIRT_FWX
-A FORWARD -j LIBVIRT_FWI
-A FORWARD -j LIBVIRT_FWO
-A OUTPUT -j LIBVIRT_OUT

which is different than before, and reflect what I'd expect from the "open" forward mode.

The test was made using the following packages:

# dpkg -l | grep libvirt
ii libvirt-clients 6.0.0-0ubuntu8.16 amd64 Programs for the libvirt library
ii libvirt-daemon 6.0.0-0ubuntu8.16 amd64 Virtualization daemon
ii libvirt-daemon-driver-qemu 6.0.0-0ubuntu8.16 amd64 Virtualization daemon QEMU connection driver
ii libvirt-daemon-driver-storage-rbd 6.0.0-0ubuntu8.16 amd64 Virtualization daemon RBD storage driver
ii libvirt-daemon-system 6.0.0-0ubuntu8.16 amd64 Libvirt daemon configuration files
ii libvirt-daemon-system-systemd 6.0.0-0ubuntu8.16 amd64 Libvirt daemon configuration files (systemd)
ii libvirt0:amd64 6.0.0-0ubuntu8.16 amd64 library for interfacing with different virtualization systems

Could you please double check and make sure that your rules are indeed being cleaned before you restart the libvirt service? It seems to me that this may be the root cause of what you're experiencing.

Thanks.

Revision history for this message
Jeff (jeffck) wrote :

Thank you for checking!
It seems I will need to retest versions, i was simply checking if it added ANY libvirt rules (iptables-save|grep -i virt), as we expected "open" to NOT add any firewall rules. I was not checking for specific libvirt rules.

Revision history for this message
Jeff (jeffck) wrote :

For the record I was sure to clear (and check they were gone) all libvirt rules between tests using iptables-restore

Revision history for this message
Christian Ehrhardt  (paelzer) wrote :

Hi Jeff, we didn't hear back from you neither were we able to recreate or understand your case.
Was there more found either insight to what was broken for us to take a look at for all of Ubuntu or just any kind of local fix/workaroudn that helped you?

To see through the remaining open bugs I'll mark this one invalid as it lacks further feedback.
But if you have more please add it and reopen!

Changed in libvirt (Ubuntu):
status: Incomplete → Invalid
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.