package libvirt-daemon-system 4.0.0-1ubuntu8.19 failed to install/upgrade: installed libvirt-daemon-system package post-removal script subprocess returned error exit status 128

Hi Patrick,
I had to smile about you feeling "accused to modify files", on one does so.
It is an automated detection of files not matching or being part of the installed archive, but those listed here are generated on installation - so I can confirm you did not do anything wrong in regard to those files. BTW newer versions (I think 21.04+) will handle them slightly differently and no more 'accuse' people.

But all of that isn't the actual problem, the issue that you faced was this:

```
Removing libvirt-daemon dnsmasq configuration
Purging configuration files for libvirt-daemon-system (4.0.0-1ubuntu8.19) ...
/usr/sbin/delgroup: `libvirt-dnsmasq' still has `libvirtd' as their primary group!
/usr/sbin/delgroup: `libvirt-dnsmasq' still has `libvirt' as their primary group!
userdel: group libvirt-dnsmasq not removed because it is not the primary group of user libvirt-dnsmasq.
dpkg: error processing package libvirt-daemon-system (--purge):
```

I can't see why - as part of the upgrade - it tried to remove libvirt for you, the log isn't long enough to show that. But while removing it the user failed to be removed and that is the problem.

Back in 16.04 this was created as:
    if ! getent passwd libvirt-dnsmasq >/dev/null; then
        adduser --quiet \
            --system \
            --ingroup libvirtd \
            --quiet \
            --disabled-login \
            --disabled-password \
            --home /var/lib/libvirt/dnsmasq \
            --no-create-home \
            -gecos "Libvirt Dnsmasq" \
            libvirt-dnsmasq
    fi

In 18.04 that changed and it indeed got its own group also named libvirt-dnsmasq (which is better as it has less permissions).

    if ! getent group libvirt-dnsmasq >/dev/null; then
        addgroup --quiet --system libvirt-dnsmasq
    fi
    if ! getent passwd libvirt-dnsmasq >/dev/null; then
        adduser --quiet \
                --system \
                --ingroup libvirt-dnsmasq \
                --disabled-login \
                --disabled-password \
                --home /var/lib/libvirt/dnsmasq \
                --no-create-home \
                --gecos "Libvirt Dnsmasq" \
                libvirt-dnsmasq
    fi

It seems you still have the user as it was created back in Xenial.
Then the removal/purging code of Bionic runs and fails.

All the removals I find are properly guarded by || true to not make it fail if anything is odd (like here by having the old user/group relation). So I wonder why this breaks for you:

        if getent passwd libvirt-dnsmasq >/dev/null; then
                deluser libvirt-dnsmasq >/dev/null || true
        fi

    # a running libvirt-dnsmasq will break these removals
    # yet the lifecycle of the network is non-related to the pkg purge
    # Therefore ignore errors on these removals, better leave a user than break
    if getent group libvirt-dnsmasq >/dev/null; then
        delgroup libvirt-dnsmasq --system 2>/dev/null >/dev/null || true
    fi
    if getent passwd libvirt-dnsmasq >/dev/null; then
        deluser libvirt-dnsmasq --system 2>/dev/null >/dev/null || true
    fi

I'm trying to recreate this ...

repro:
1. get a xenial system

2. install libvirt-bin
 $ apt install libvirt-bin
 $ id libvirt-dnsmasq
  uid=112(libvirt-dnsmasq) gid=116(libvirtd) groups=116(libvirtd)

3. upgrade to Bionic
 $ do-release-upgrade
 $ dpkg -l | grep libvirt
ii libvirt-bin 4.0.0-1ubuntu8.19 amd64 programs for the libvirt library
ii libvirt-clients 4.0.0-1ubuntu8.19 amd64 Programs for the libvirt library
ii libvirt-daemon 4.0.0-1ubuntu8.19 amd64 Virtualization daemon
ii libvirt-daemon-driver-storage-rbd 4.0.0-1ubuntu8.19 amd64 Virtualization daemon RBD storage driver
ii libvirt-daemon-system 4.0.0-1ubuntu8.19 amd64 Libvirt daemon configuration files
ii libvirt0:amd64 4.0.0-1ubuntu8.19 amd64 library for interfacing with different virtualization systems
 $ id libvirt-dnsmasq
uid=112(libvirt-dnsmasq) gid=116(libvirtd) groups=116(libvirtd)

4. since the upgrade usually isn't purging, call purge manually
 $ apt remove --purge libvirt-daemon-system
...
Removing libvirt-daemon dnsmasq configuration
(Reading database ... 31960 files and directories currently installed.)
Purging configuration files for libvirt-daemon-system (4.0.0-1ubuntu8.19) ...
/usr/sbin/delgroup: `libvirt-dnsmasq' still has `libvirtd' as their primary group!
/usr/sbin/delgroup: `libvirt-dnsmasq' still has `libvirt' as their primary group!
userdel: group libvirt-dnsmasq not removed because it is not the primary group of user libvirt-dnsmasq.
dpkg: error processing package libvirt-daemon-system (--purge):
 installed libvirt-daemon-system package post-removal script subprocess returned error exit status 128

So this is reproducible, odd to come up so late - seems no one ever wanted to purge a pre 18.04 installed libvirt since 2016 :-)

Still the user handling might be a red-herring and the issue a different one.

To retrigger it from here (a bit faster than the upgrade)
$ apt install libvirt-daemon-system
$ killall dnsmasq
$ userdel libvirt-dnsmasq
$ addgroup --system libvirtd
$ adduser --quiet --system --ingroup libvirtd --disabled-login --disabled-password --home /var/lib/libvirt/dnsmasq --no-create-home libvirt-dnsmasq
$ apt remove --purge libvirt-daemon-system

Running that with some options for verbosity confirmed that it isn't the deluser/delgroup calls that break.
Instead later on in the postrm (/var/lib/dpkg/info/libvirt-daemon-system.postrm) the generated snippets enter

# Automatically added by dh_installdebconf/11.1.6ubuntu2
if [ "$1" = purge ] && [ -e /usr/share/debconf/confmodule ]; then
        . /usr/share/debconf/confmodule
        db_purge
fi
# End automatically added section

That section triggers this log:

+ [ -e /usr/share/debconf/confmodule ]
+ . /usr/share/debconf/confmodule
+ [ ! 1 ]
+ [ -z ]
+ exec
+ [ ]
+ exec
+ DEBCONF_REDIR=1
+ export DEBCONF_REDIR
+ db_purge
+ _db_cmd PURGE
+ _db_internal_IFS=

+ IFS=
+ printf %s\n PURGE
+ IFS=

+ IFS=
 read -r _db_internal_line
+ RET=20 Unsupported command "removing" (full line was "Removing group `libvirtd' ...") received from confmodule.
+ return 20

So the issue seems about removing group libvirtd (and not libvirt-dnsmaq as it first seemed) and somehow that error message creeps into the debconf handling and then breaks it there.

We know that libvirtd can't be removed as it is early in the postrm and at this time we know it will trigger:

$ delgroup libvirtd
/usr/sbin/delgroup: `libvirt-dnsmasq' still has `libvirtd' as their primary group!

But removing it afterwards when it can be removed only ends up breaking slightly different:

RET=20 Unsupported command "the" (full line was "The group `libvirtd' does not exist.") received from confmodule.

Maybe the sysuser config of libvirtd makes debconf want to remove it, but in the combination we have in place this fails and that makes it fail badly. I need to track this down in more detail, but this week will unlikely find time for it.

As a workaround for now, I found that removing/purging libvirt-daemon-system once more passed fine so you can just re-run whatever you did to trigger the purges and the second time it should pass.

 Hey Christian, don't spend anymore time on my problem! It isn't a problem since it doesn't "break" anything.
Your diligence and expertise is recognized by me and appreciated! (it explained I don't need to worry!)

Have a pleasant Holiday!
Patrick

     On Tuesday, November 23, 2021, 12:15:46 AM PST, Christian Ehrhardt  <email address hidden> wrote:

 We know that libvirtd can't be removed as it is early in the postrm and
at this time we know it will trigger:

$ delgroup libvirtd
/usr/sbin/delgroup: `libvirt-dnsmasq' still has `libvirtd' as their primary group!

But removing it afterwards when it can be removed only ends up breaking slightly different:

RET=20 Unsupported command "the" (full line was "The group `libvirtd'
does not exist.") received from confmodule.

Maybe the sysuser config of libvirtd makes debconf want to remove it,
but in the combination we have in place this fails and that makes it
fail badly. I need to track this down in more detail, but this week will
unlikely find time for it.

As a workaround for now, I found that removing/purging libvirt-daemon-
system once more passed fine so you can just re-run whatever you did to
trigger the purges and the second time it should pass.

** Changed in: libvirt (Ubuntu)
      Status: Confirmed => Triaged

** Changed in: libvirt (Ubuntu)
  Importance: Undecided => Medium

** Tags added: server-todo

--
You received this bug notification because you are subscribed to the bug
report.
https://bugs.launchpad.net/bugs/1951903

Title:
  package libvirt-daemon-system 4.0.0-1ubuntu8.19 failed to
  install/upgrade: installed libvirt-daemon-system package post-removal
  script subprocess returned error exit status 128

Status in libvirt package in Ubuntu:
  Triaged

Bug description:
  upgraded from 16.04? to 18.04 at end of upgrade I got libvirt-daemon-system error 128.
  I didn't change anything in the /etc/livirt/nwfilter... files like the system
  now accuses me of doing! I logged in this time through Unity to get my old desktop back and that is when I got this system report.
  I was getting a system error when I logged in after the upgrade but no error # or anything;

  ProblemType: Package
  DistroRelease: Ubuntu 18.04
  Package: libvirt-daemon-system 4.0.0-1ubuntu8.19
  ProcVersionSignature: Ubuntu 4.15.0-162.170-generic 4.15.18
  Uname: Linux 4.15.0-162-generic i686
  ApportVersion: 2.20.9-0ubuntu7.27
  Architecture: i386
  Date: Sun Nov 21 13:46:10 2021
  ErrorMessage: installed libvirt-daemon-system package post-removal script subprocess returned error exit status 128
  InstallationDate: Installed on 2021-11-05 (18 days ago)
  InstallationMedia: Ubuntu 16.04.5 LTS "Xenial Xerus" - Release i386 (20180731)
  ProcCmdline: BOOT_IMAGE=/boot/vmlinuz-4.15.0-162-generic root=UUID=8c89ff08-e57d-4fb0-b472-adfde83e73b5 ro quiet splash vt.handoff=1
  Python3Details: /usr/bin/python3.6, Python 3.6.9, python3-minimal, 3.6.7-1~18.04
  PythonDetails: /usr/bin/python2.7, Python 2.7.17, python-minimal, 2.7.15~rc1-1
  SourcePackage: libvirt
  Title: package libvirt-daemon-system 4.0.0-1ubuntu8.19 failed to install/upgrade: installed libvirt-daemon-system package post-removal script subproce...

I'm happy that you are fine with it Patrick, but I want it resolve now.

While sleeping about that I think I found what is missing/broken and needs evaluation.
As I said the libvirt group was considered insecure and therefore the new group add is different.
But on upgrade to Bionic that should have been adapted and that didn't happen.
There I need to check if this needs a fixup.

pkg/ubuntu/bionic-devel:debian/libvirt-daemon-system.postinst
    # For upgrades that still have the insecure libvirt group (too much privileges)
    if [ -n "$2" ] && dpkg --compare-versions -- "$2" le-nl "4.0.0-1ubuntu5~"; then
        if [ "$(id -r -g -n libvirt-dnsmasq)" == "libvirt" ]; then
            echo "assigning libvirt-dnsmasq a less privileged group (libvirt->libvirt-dnsmasq)"
            usermod libvirt-dnsmasq -g libvirt-dnsmasq
        fi
    fi

We need to debug why this failed (probably because the PKG split libvirt-bin into libvirt-* makes this not having a prior version) and fix it up to avoid the issue.

I didn't find anything more on this case, neither a great idea on the bit we have nor anyone else being affected. In the try to focus on what is important I'd close the bug to have a better view on what is left to tackle.

 Thank you Christian, I remember this from many months ago, (I think), unless this is a new bug. I only had one problem
once lately but I am not worried about it. Thanks for doing anything but don't waste your time on Me! (unless it affects other people)
Take care and you are appreciated!
Sincerely
Patrick Timblin
Ellensburg (Washington state)USA

     On Wednesday, September 7, 2022, 11:21:52 PM PDT, Christian Ehrhardt  <email address hidden> wrote:

 I didn't find anything more on this case, neither a great idea on the
bit we have nor anyone else being affected. In the try to focus on what
is important I'd close the bug to have a better view on what is left to
tackle.

** Changed in: libvirt (Ubuntu)
      Status: Triaged => Invalid

--
You received this bug notification because you are subscribed to the bug
report.
https://bugs.launchpad.net/bugs/1951903

Title:
  package libvirt-daemon-system 4.0.0-1ubuntu8.19 failed to
  install/upgrade: installed libvirt-daemon-system package post-removal
  script subprocess returned error exit status 128

Status in libvirt package in Ubuntu:
  Invalid

Bug description:
  upgraded from 16.04? to 18.04 at end of upgrade I got libvirt-daemon-system error 128.
  I didn't change anything in the /etc/livirt/nwfilter... files like the system
  now accuses me of doing! I logged in this time through Unity to get my old desktop back and that is when I got this system report.
  I was getting a system error when I logged in after the upgrade but no error # or anything;

  ProblemType: Package
  DistroRelease: Ubuntu 18.04
  Package: libvirt-daemon-system 4.0.0-1ubuntu8.19
  ProcVersionSignature: Ubuntu 4.15.0-162.170-generic 4.15.18
  Uname: Linux 4.15.0-162-generic i686
  ApportVersion: 2.20.9-0ubuntu7.27
  Architecture: i386
  Date: Sun Nov 21 13:46:10 2021
  ErrorMessage: installed libvirt-daemon-system package post-removal script subprocess returned error exit status 128
  InstallationDate: Installed on 2021-11-05 (18 days ago)
  InstallationMedia: Ubuntu 16.04.5 LTS "Xenial Xerus" - Release i386 (20180731)
  ProcCmdline: BOOT_IMAGE=/boot/vmlinuz-4.15.0-162-generic root=UUID=8c89ff08-e57d-4fb0-b472-adfde83e73b5 ro quiet splash vt.handoff=1
  Python3Details: /usr/bin/python3.6, Python 3.6.9, python3-minimal, 3.6.7-1~18.04
  PythonDetails: /usr/bin/python2.7, Python 2.7.17, python-minimal, 2.7.15~rc1-1
  SourcePackage: libvirt
  Title: package libvirt-daemon-system 4.0.0-1ubuntu8.19 failed to install/upgrade: installed libvirt-daemon-system package post-removal script subprocess returned error exit status 128
  UpgradeStatus: Upgraded to bionic on 2021-11-21 (1 days ago)
  mtime.conffile..etc.libvirt.nwfilter.allow-arp.xml: 2021-11-22T12:30:12.396265
  mtime.conffile..etc.libvirt.nwfilter.allow-dhcp-server.xml: 2021-11-22T12:30:12.428264
  mtime.conffile..etc.libvirt.nwfilter.allow-dhcp.xml: 2021-11-22T12:30:12.408265
  mtime.conffile..etc.libvirt.nwfilter.allow-incoming-ipv4.xml: 2021-11-22T12:30:12.416264
  mtime.conffile..etc.libvirt.nwfilter.allow-ipv4.xml: 2021-11-22T12:30:12.444263
  mtime.conffile..etc.libvirt.nwfilter.clean-traffic.xml: 2021-11-22T12:30:12.440263
  mtime.conffile..etc.libvirt....