wait-online does not correctly identify managed links

The attachment "0001-wait-online-Check-if-links-are-in-the-configuring-st.patch" seems to be a patch. If it isn't, please remove the "patch" flag from the attachment, remove the "patch" tag, and if you are a member of the ~ubuntu-reviewers, unsubscribe the team.

[This is an automated message performed by a Launchpad user owned by ~brian-murray, for any issues please contact him.]

Hi Mike,

Do you have a test case that can demonstrate/reproduce this issue?

[Expired for systemd (Ubuntu) because there has been no activity for 60 days.]

Apologies for the late response:

Yes, we have a Ubuntu 22.04 LTS image that we use across a set of machines, some of which have multiple Ethernet ports, but for which we only connect a single port. For this image we use the following Netplan configuration (/etc/netplan/ethernet.yaml):

network:
  ethernets:
    en:
      match:
        name: en*
      dhcp4: true
  version: 2

Which places _all_ Ethernet interfaces under systemd-networkd management, allowing us to bring up at least one interface via DHCP without needing to know the exact interface name (which depends on the exact hardware) and enables us to use any of the available Ethernet ports on the machine.

As part of this image we also have a boot-time systemd service that requires functional network connectivity in order to successfully start. The unit file for this service depends on the network-online target ("Wants=network-online.target"). For example purposes you could replace this service with a remote NFS mount specified in /etc/fstab, since boot-time NFS mounts implicitly depend on the network-online target as well.

The default behavior of systemd-networkd-wait-online is to wait until all networkd-managed interfaces are routable before the network-online target is started, but since we only ever connect a single Ethernet interface to the network, we override its behavior here in our image (/etc/systemd/system/systemd-networkd-wait-online.service.d/any.conf):

[Service]
ExecStart=
ExecStart=/usr/local/lib/systemd-networkd-wait-online --any --timeout=0

Now, the problem introduced by the UBUNTU-wait-online-exit-if-no-links-are-managed patch is that it places us in a race against our DHCP server. During a successful boot, where networkd is able to obtain an address via DHCP _before_ the wait-online service is reached, networkctl reports this status:

$ sudo networkctl
IDX LINK TYPE OPERATIONAL SETUP
  1 lo loopback carrier unmanaged
  2 enp0s1 ether routable configured
  3 enp0s2 ether no-carrier configuring

Since the second interface (enp0s1) is in the configured state and is routable, systemd-networkd-wait-online behaves correctly by exiting immediately. However, during an unsuccessful boot, where networkd has yet to obtain an address via DHCP at the time the wait-online service is reached, networkctl reports this status:

$ sudo networkctl
IDX LINK TYPE OPERATIONAL SETUP
  1 lo loopback carrier unmanaged
  2 enp0s1 ether degraded configuring
  3 enp0s2 ether no-carrier configuring

Here, both of the managed Ethernet interfaces are still in the configuring state. Since UBUNTU-wait-online-exit-if-no-links-are-managed doesn't check if managed interfaces are in this state, only configured, it assumes those links are unmanaged and systemd-networkd-wait-online also exits immediately. This results in failure of our network-dependent service to start (or equivalently, failure of the remote NFS volume to mount).

With the provided patch, systemd-networkd-wait-online identifies two managed interfaces in the configuring state and waits until at least one of them becomes routable, and thus waits until the network is fun...

Thanks for that additional information.

So, looking into this a bit, I am a bit tempted to just drop this patch from Jammy. For one, I will be dropping this patch in the current cycle because upstream has a patch[1] now that accomplishes the same goal in a different way.

More importantly though, looking at bug 1728181 (for which this patch exists), it's not clear to me that this ever worked exactly as it was intended. I will follow up to try and find out what happened, but based on the comments it sounds like the fix was incomplete and wasn't further developed.

Testing on Lunar, this patch only works as intended when the --any flag is specified, which I think is because when this patch was refreshed over the years, it was not correctly adapted for the context.

TL;DR - I do think this is something we can address in the next Jammy SRU, and AFAICT just dropping the patch should solve your issue.

[1]

This bug was fixed in the package systemd - 253.5-1ubuntu1

---------------
systemd (253.5-1ubuntu1) mantic; urgency=medium

  * Merge 253.5-1 from Debian unstable. Remaining changes:
    - debian/tests/upstream{,-1,-2}: split upstream tests into two parts
    - debian/tests/tests-in-lxd: run some autopkgtests in LXD too
    - debian/tests/storage: skip tests if scsi_debug module is not available
    - debian/tests/boot-and-services:
      + skip test_syslog on s390x
      + skip coredump tests on s390x
      + skip apparmor tests on armhf
      + consume stderr in systemctl status call in test_service
    - debian/systemd.postinst: skip daemon-reexec and try-restarts during shutdown
    - debian/systemd.postinst,debian/rules:
      Disable dh_installinit generation of tmpfiles for systemd package, and
      manually call systemd-tmpfiles --create in postinst
    - debian/systemd-resolved.postinst: copy existing /etc/resolv.conf to
      /run/systemd/resolve/stub-resolv.conf
    - debian/rules:
      + Set default user path
      + Disable LLMNR by default
      + Remove unneeded efi artifacts on i386 to avoid debugedit errors
    - debian/rules,debian/control,debian/tests/control:
      + Do not build with tpm libraries on i386
      + Do not build with libqrencode on i386
    - debian/gbp.conf,debian/extra/wrap_cl.py:
      Use a customization script to add LP commit links to changelog
    - debian/extra/systemd-oomd-defaults/-.slice.d/10-oomd-root-slice-defaults.conf:
      Set ManagedOOMSwap=auto, disabling swap kill by default
    - debian/extra/start-udev: Set scsi_mod scan=sync even if it's builtin
      to the kernel (we previously only set it in modprobe.d)
    - debian/control:
      + Add Recommends: networkd-dispatcher systemd-resolved to systemd package
      + Provide upgrade path (Replaces:) for Jammy's systemd-repart
      + Give systemd-resolved Priority: important
      + Add Recommends: systemd-hwe-hwdb to udev package
      + Add Breaks: systemd (<< ${binary:Version}) to udev package so that
        systemd is upgraded as well when upgrading udev
      + Make systemd-sysv Depends: on matching version of systemd
      + Drop Recommends: libnss-myhostname libnss-resolve from systemd-resolved
    - d/p/debian/Ubuntu-UseDomains-by-default.patch:
      Enable UseDomains by default on Ubuntu.
    - d/p/debian/UBUNTU-Add-AssumedApparmorLabel-unconfined-to-timedate1-dbus.patch:
      Set AssumedAppArmorLabel=unconfined in timedate1 dbus service file
    - d/p/debian/UBUNTU-test-test-functions-launch-qemu-with-vga-none.patch:
      Launch QEMU with -vga none in test-functions
    - d/p/debian/UBUNTU-journald.service-set-Nice-1-to-dodge-watchdog-on-soft-loc.patch:
      Set Nice=-1 in journald.service to dodge watchdog on soft lockups
    - d/p/debian/UBUNTU-Revert-namespace-be-more-careful-when-handling-namespacin.patch:
      Revert "namespace: be more careful when handling namespacing failures gracefully"
      upstream commit
    - d/p/debian/UBUNTU-resolved-default-no-negative-caching.patch:
      Default to Cache=no-negative in systemd-resolved
    - d/p/Revert-network-if-sys-is-rw-then-udev-should-be-around.patch:
      Rever...

> This test uses a VM managed by libvirt. The specifics of how to create the
> VM and how to define networks for use in libvirt are outside the scope of
> this test plan. But, in this scenario we have the "default" network which
> provides DHCP to the VM, and the "no-dhcp" network which does not provide DHCP.

That's not appropriate. The test case should tell anyone the steps required to reproduce this bug and verify the fix. Setting up the VM is thus not out of scope.

> That's not appropriate. The test case should tell anyone the steps required
> to reproduce this bug and verify the fix. Setting up the VM is thus not out
> of scope.

Added instructions for creating the VM.

Hello Mike, or anyone else affected,

Accepted systemd into lunar-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/systemd/252.5-2ubuntu3.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-lunar to verification-done-lunar. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-lunar. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Hello Mike, or anyone else affected,

Accepted systemd into jammy-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/systemd/249.11-0ubuntu3.10 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-jammy to verification-done-jammy. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-jammy. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

All autopkgtests for the newly accepted systemd (249.11-0ubuntu3.10) for jammy have finished running.
The following regressions have been reported in tests triggered by the package:

apt/2.4.10 (armhf)
casync/2+20201210-1build1 (ppc64el)
comitup/1.15-1 (armhf)
dbus/1.12.20-2ubuntu4.1 (armhf)
initramfs-tools/0.140ubuntu13.4 (s390x)
linux-azure-5.19/5.19.0-1027.30~22.04.2 (arm64)
linux-gcp-6.2/6.2.0-1011.11~22.04.3 (arm64)
linux-lowlatency/5.15.0-83.92 (arm64)
linux-lowlatency-hwe-5.19/5.19.0-1030.30 (arm64)
linux-nvidia-tegra/5.15.0-1016.16 (arm64)
linux-oracle-5.19/5.19.0-1027.30 (arm64)
mkosi/unknown (s390x)
munin/2.0.57-1ubuntu2 (armhf)
netplan.io/0.105-0ubuntu2~22.04.3 (arm64)
prometheus-postfix-exporter/unknown (s390x)
samba/2:4.15.13+dfsg-0ubuntu1.4 (arm64)

Please visit the excuses page listed below and investigate the failures, proceeding afterwards as per the StableReleaseUpdates policy regarding autopkgtest regressions [1].

https://people.canonical.com/~ubuntu-archive/proposed-migration/jammy/update_excuses.html#systemd

[1] https://wiki.ubuntu.com/StableReleaseUpdates#Autopkgtest_Regressions

Thank you!

All autopkgtests for the newly accepted systemd (252.5-2ubuntu3.1) for lunar have finished running.
The following regressions have been reported in tests triggered by the package:

apt/2.6.0ubuntu0.1 (armhf)
clutter-1.0/unknown (amd64)
corosync/3.1.7-1ubuntu1 (i386)
cryptsetup/unknown (s390x)
libreswan/unknown (s390x)
libsdl2/unknown (s390x)
linux-lowlatency/6.2.0-1012.12 (arm64)
mutter/44.3-0ubuntu1 (amd64)
netplan.io/0.106-0ubuntu3 (arm64)
samba/2:4.17.7+dfsg-1ubuntu2.2 (arm64)
stunnel4/3:5.68-2 (i386)
stunnel4/unknown (s390x)
systemd-bootchart/234-2 (amd64)
ubuntu-drivers-common/1:0.9.7.1.1 (ppc64el)

Please visit the excuses page listed below and investigate the failures, proceeding afterwards as per the StableReleaseUpdates policy regarding autopkgtest regressions [1].

https://people.canonical.com/~ubuntu-archive/proposed-migration/lunar/update_excuses.html#systemd

[1] https://wiki.ubuntu.com/StableReleaseUpdates#Autopkgtest_Regressions

Thank you!

I have verified the fix using systemd 249.11-0ubuntu3.10 from jammy-proposed:

The VM and test networks are already configured on my system:

nr@six:/t/tmp.jTS3wdPdT3$ virsh list
 Id Name State
-----------------------------------
 8 clean-jammy-amd64 running

nr@six:/t/tmp.jTS3wdPdT3$ virsh net-list
 Name State Autostart Persistent
--------------------------------------------
 default active yes yes
 no-dhcp active no no

nr@six:/t/tmp.jTS3wdPdT3$ virsh net-dumpxml --network default
<network connections='2'>
  <name>default</name>
  <uuid>04260896-2701-422d-84e0-8e0df1122db3</uuid>
  <forward mode='nat'>
    <nat>
      <port start='1024' end='65535'/>
    </nat>
  </forward>
  <bridge name='virbr0' stp='on' delay='0'/>
  <mac address='52:54:00:bd:9f:3a'/>
  <ip address='192.168.122.1' netmask='255.255.255.0'>
    <dhcp>
      <range start='192.168.122.2' end='192.168.122.254'/>
    </dhcp>
  </ip>
</network>

nr@six:/t/tmp.jTS3wdPdT3$ virsh net-dumpxml --network no-dhcp
<network>
  <name>no-dhcp</name>
  <uuid>2c047740-caab-4c90-8421-70da6732a759</uuid>
  <forward mode='nat'>
    <nat>
      <port start='1024' end='65535'/>
    </nat>
  </forward>
  <bridge name='virbr1' stp='on' delay='0'/>
  <mac address='52:54:00:ac:12:45'/>
  <ip address='172.16.1.1' netmask='255.255.0.0'>
  </ip>
</network>

Running the test case:

(Host)

nr@six:/t/tmp.jTS3wdPdT3$ virsh detach-interface clean-jammy-amd64 network
Interface detached successfully

(VM)

root@clean-jammy-amd64:/home/nr# apt-cache policy systemd
systemd:
  Installed: 249.11-0ubuntu3.10
  Candidate: 249.11-0ubuntu3.10
  Version table:
 *** 249.11-0ubuntu3.10 500
        500 http://archive.ubuntu.com/ubuntu jammy-proposed/main amd64 Packages
        500 http://security.ubuntu.com/ubuntu jammy-proposed/main amd64 Packages
        100 /var/lib/dpkg/status
     249.11-0ubuntu3.9 500
        500 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages
     249.11-0ubuntu3.7 500
        500 http://security.ubuntu.com/ubuntu jammy-security/main amd64 Packages
     249.11-0ubuntu3 500
        500 http://archive.ubuntu.com/ubuntu jammy/main amd64 Packages
root@clean-jammy-amd64:/home/nr# cat > /etc/systemd/network/10-dhcp.network << EOF
[Match]
Name=en*

[Network]
DHCP=yes
EOF
root@clean-jammy-amd64:/home/nr# systemctl restart systemd-networkd

(Host)

nr@six:/t/tmp.jTS3wdPdT3$ virsh attach-interface clean-jammy-amd64 network no-dhcp
Interface attached successfully

(VM)

root@clean-jammy-amd64:/home/nr# networkctl
IDX LINK TYPE OPERATIONAL SETUP
  1 lo loopback carrier unmanaged
  3 ens3 ether degraded configuring

2 links listed.
root@clean-jammy-amd64:/home/nr# /lib/systemd/systemd-networkd-wait-online --any --timeout 10
Timeout occurred while waiting for network connectivity.
root@clean-jammy-amd64:/home/nr# networkctl
IDX LINK TYPE OPERATIONAL SETUP
  1 lo loopback carrier unmanaged
  3 ens3 ether carrier configuring

2 links listed.

(Host)

nr@six:/t/tmp.jTS3wdPdT3$ virsh attach-interface clean-jammy-amd64 network default
Interface attached successfully

(VM)

root@clean-jammy-...

I have verified the fix using systemd 252.5-2ubuntu3.1 from lunar-proposed:

The VM and test networks are already configured on my system:

nr@six:/t/tmp.jTS3wdPdT3$ virsh list
 Id Name State
-----------------------------------
 9 clean-lunar-amd64 running

nr@six:/t/tmp.jTS3wdPdT3$ virsh net-list
 Name State Autostart Persistent
--------------------------------------------
 default active yes yes
 no-dhcp active no no

nr@six:/t/tmp.jTS3wdPdT3$ virsh net-dumpxml --network default
<network connections='2'>
  <name>default</name>
  <uuid>04260896-2701-422d-84e0-8e0df1122db3</uuid>
  <forward mode='nat'>
    <nat>
      <port start='1024' end='65535'/>
    </nat>
  </forward>
  <bridge name='virbr0' stp='on' delay='0'/>
  <mac address='52:54:00:bd:9f:3a'/>
  <ip address='192.168.122.1' netmask='255.255.255.0'>
    <dhcp>
      <range start='192.168.122.2' end='192.168.122.254'/>
    </dhcp>
  </ip>
</network>

nr@six:/t/tmp.jTS3wdPdT3$ virsh net-dumpxml --network no-dhcp
<network>
  <name>no-dhcp</name>
  <uuid>2c047740-caab-4c90-8421-70da6732a759</uuid>
  <forward mode='nat'>
    <nat>
      <port start='1024' end='65535'/>
    </nat>
  </forward>
  <bridge name='virbr1' stp='on' delay='0'/>
  <mac address='52:54:00:ac:12:45'/>
  <ip address='172.16.1.1' netmask='255.255.0.0'>
  </ip>
</network>

Running the test case:

(Host)

nr@six:/t/tmp.jTS3wdPdT3$ virsh detach-interface clean-lunar-amd64 network
Interface detached successfully

(VM)

root@clean-lunar-amd64:/home/nr# apt-cache policy systemd
systemd:
  Installed: 252.5-2ubuntu3.1
  Candidate: 252.5-2ubuntu3.1
  Version table:
 *** 252.5-2ubuntu3.1 500
        500 http://security.ubuntu.com/ubuntu lunar-proposed/main amd64 Packages
        500 http://archive.ubuntu.com/ubuntu lunar-proposed/main amd64 Packages
        100 /var/lib/dpkg/status
     252.5-2ubuntu3 500
        500 http://archive.ubuntu.com/ubuntu lunar/main amd64 Packages
root@clean-lunar-amd64:/home/nr# cat > /etc/systemd/network/10-dhcp.network << EOF
[Match]
Name=en*

[Network]
DHCP=yes
EOF
root@clean-lunar-amd64:/home/nr# systemctl restart systemd-networkd

(Host)

nr@six:/t/tmp.jTS3wdPdT3$ virsh attach-interface clean-lunar-amd64 network no-dhcp
Interface attached successfully

(VM)

root@clean-lunar-amd64:/home/nr# networkctl
IDX LINK TYPE OPERATIONAL SETUP
  1 lo loopback carrier unmanaged
  5 ens3 ether degraded configuring

2 links listed.
root@clean-lunar-amd64:/home/nr# /lib/systemd/systemd-networkd-wait-online --any --timeout=10
Timeout occurred while waiting for network connectivity.
root@clean-lunar-amd64:/home/nr# networkctl
IDX LINK TYPE OPERATIONAL SETUP
  1 lo loopback carrier unmanaged
  5 ens3 ether degraded configuring

2 links listed.

(Host)

nr@six:/t/tmp.jTS3wdPdT3$ virsh attach-interface clean-lunar-amd64 network default
Interface attached successfully

(VM)

root@clean-lunar-amd64:/home/nr# /lib/systemd/systemd-networkd-wait-online --any --timeout=0
root@clean-lunar-amd64:/home/nr# echo $?
0
root@clean-lunar-amd64:/home/nr# networkctl
IDX LINK TYPE OPERATIONAL SETUP
  1 lo loopback...

This bug was fixed in the package systemd - 252.5-2ubuntu3.1

---------------
systemd (252.5-2ubuntu3.1) lunar; urgency=medium

  * debian/tests/tests-in-lxd: use --reuse flag in lxc publish (LP: #2023229)
    File: debian/tests/tests-in-lxd
    https://git.launchpad.net/~ubuntu-core-dev/ubuntu/+source/systemd/commit/?id=2a443ddd5b67596647e463eee64d58c3c71bbf08
  * udev-rules: fix nvme symlink creation on namespace changes (LP: #2028180)
    File: debian/patches/lp2028180-udev-rules-fix-nvme-symlink-creation-on-namespace-changes.patch
    https://git.launchpad.net/~ubuntu-core-dev/ubuntu/+source/systemd/commit/?id=064c7c00aa133b697ab3c0082b825e2f197f91cc
  * core: reorder systemd arguments on reexec (LP: #2013543)
    File: debian/patches/lp2013543-core-reorder-systemd-arguments-on-reexec.patch
    https://git.launchpad.net/~ubuntu-core-dev/ubuntu/+source/systemd/commit/?id=45541ccbc16ed30b6a120cda7401f69cce164d54
  * network/dhcp4: do not ignore the gateway even if the destination is in same network (LP: #2009743)
    Files:
    - debian/patches/lp2009743/network-dhcp4-do-not-ignore-the-gateway-even-if-the-desti.patch
    - debian/patches/lp2009743/test-network-add-one-more-testcase-for-DHCPv4-classless-r.patch
    https://git.launchpad.net/~ubuntu-core-dev/ubuntu/+source/systemd/commit/?id=ddea3bcbca3421299b13cfe897157cfbf9f0f06d
  * debian/systemd.postint: do not daemon re-exec if we could hit LP: #2013543
    File: debian/systemd.postinst
    https://git.launchpad.net/~ubuntu-core-dev/ubuntu/+source/systemd/commit/?id=e1a08bb427e19ced6c16f054ebdee43ccbc9c528
  * Drop debian/UBUNTU-wait-online-exit-if-no-links-are-managed.patch (LP: #1982218)
    File: debian/patches/debian/UBUNTU-wait-online-exit-if-no-links-are-managed.patch
    https://git.launchpad.net/~ubuntu-core-dev/ubuntu/+source/systemd/commit/?id=8235427fd8859ee393a08d425e1a52e1ebcbe2f1
  * sd-bus: bus_message_type_from_string is not pure (LP: #2031683)
    File: debian/patches/lp2031683-sd-bus-bus_message_type_from_string-is-not-pure.patch
    https://git.launchpad.net/~ubuntu-core-dev/ubuntu/+source/systemd/commit/?id=2d65b6c74864976a2bb42c884baf75d1674215fc

 -- Nick Rosbrook <email address hidden> Fri, 18 Aug 2023 14:15:03 -0400

The verification of the Stable Release Update for systemd has completed successfully and the package is now being released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

This bug was fixed in the package systemd - 249.11-0ubuntu3.10

---------------
systemd (249.11-0ubuntu3.10) jammy; urgency=medium

  [ Nick Rosbrook ]
  * debian/tests/tests-in-lxd: use --reuse flag in lxc publish (LP: #2023229)
    File: debian/tests/tests-in-lxd
    https://git.launchpad.net/~ubuntu-core-dev/ubuntu/+source/systemd/commit/?id=85b2ceddff1a6cc1ddbca8a1b7e5381d146e6313
  * pstore: only try to load efi_pstore module (LP: #2023462)
    File: debian/patches/lp1978079-efi-pstore-not-cleared-on-boot.patch
    https://git.launchpad.net/~ubuntu-core-dev/ubuntu/+source/systemd/commit/?id=7600bbfb1e8a399e5aeb1010a20deda3e5a06c89
  * shutdown: get only active md arrays. (LP: #2025563)
    File: debian/patches/lp2025563-shutdown-get-only-active-md-arrays.patch
    https://git.launchpad.net/~ubuntu-core-dev/ubuntu/+source/systemd/commit/?id=416a9245c8f0efbedcc4395cada23cb09c685ec3
  * udev-rules: fix nvme symlink creation on namespace changes (LP: #2028180)
    File: debian/patches/lp2028180-udev-rules-fix-nvme-symlink-creation-on-namespace-changes.patch
    https://git.launchpad.net/~ubuntu-core-dev/ubuntu/+source/systemd/commit/?id=26e85b944da9098e66fc0c39f64ee40254c0c278
  * core: reorder systemd arguments on reexec (LP: #2013543)
    File: debian/patches/lp2013543-core-reorder-systemd-arguments-on-reexec.patch
    https://git.launchpad.net/~ubuntu-core-dev/ubuntu/+source/systemd/commit/?id=19ba0f20d311642596dc65fa5d6eb96a2d4be280
  * network/dhcp4: do not ignore the gateway even if the destination is in same network (LP: #2009743)
    Files:
    - debian/patches/lp2009743/network-dhcp4-do-not-ignore-the-gateway-even-if-the-desti.patch
    - debian/patches/lp2009743/test-network-add-one-more-testcase-for-DHCPv4-classless-r.patch
    https://git.launchpad.net/~ubuntu-core-dev/ubuntu/+source/systemd/commit/?id=73e774a2fb99c82df6d0edd770bb84ab735ec2f0
  * Drop debian/UBUNTU-wait-online-exit-if-no-links-are-managed.patch (LP: #1982218)
    File: debian/patches/debian/UBUNTU-wait-online-exit-if-no-links-are-managed.patch
    https://git.launchpad.net/~ubuntu-core-dev/ubuntu/+source/systemd/commit/?id=cf82f08feea456e1c65895b34bffa8c33d421588
  * debian/systemd.postint: do not daemon re-exec if we could hit LP: #2013543
    File: debian/systemd.postinst
    https://git.launchpad.net/~ubuntu-core-dev/ubuntu/+source/systemd/commit/?id=be484dab06d590b1792a8f016f4292373d0174b7
  * binfmt: fix systemd-binfmt in LXD containers (LP: #1999275)
    Files:
    - debian/patches/lp1999275/binfmt-check-if-binfmt-is-mounted-before-applying-rules.patch
    - debian/patches/lp1999275/binfmt-util-also-check-if-binfmt-is-mounted-in-read-write.patch
    - debian/patches/lp1999275/binfmt-util-split-out-binfmt_mounted.patch
    - debian/patches/lp1999275/unit-check-more-specific-path-to-be-written-by-systemd-bi.patch
    https://git.launchpad.net/~ubuntu-core-dev/ubuntu/+source/systemd/commit/?id=0b9eadf05ee6db0e009dacbc71521480095880b3

  [ Dan Streetman ]
  * Fix machinectl pull-tar and import-tar (LP: #1977630)
    Author: Dan Streetman
    File: debian/patches/lp1977630-fix_machinectl_pull_tar.patch
    https://git.launchpad.net/~ubuntu-core-dev/ubuntu/+sourc...

The release of this SRU has been rolled back in jammy because of bug #2035406 which appears to be a regression introduced in the new version of systemd.

This bug was fixed in the package systemd - 249.11-0ubuntu3.10

---------------
systemd (249.11-0ubuntu3.10) jammy; urgency=medium

  [ Nick Rosbrook ]
  * debian/tests/tests-in-lxd: use --reuse flag in lxc publish (LP: #2023229)
    File: debian/tests/tests-in-lxd
    https://git.launchpad.net/~ubuntu-core-dev/ubuntu/+source/systemd/commit/?id=85b2ceddff1a6cc1ddbca8a1b7e5381d146e6313
  * pstore: only try to load efi_pstore module (LP: #2023462)
    File: debian/patches/lp1978079-efi-pstore-not-cleared-on-boot.patch
    https://git.launchpad.net/~ubuntu-core-dev/ubuntu/+source/systemd/commit/?id=7600bbfb1e8a399e5aeb1010a20deda3e5a06c89
  * shutdown: get only active md arrays. (LP: #2025563)
    File: debian/patches/lp2025563-shutdown-get-only-active-md-arrays.patch
    https://git.launchpad.net/~ubuntu-core-dev/ubuntu/+source/systemd/commit/?id=416a9245c8f0efbedcc4395cada23cb09c685ec3
  * udev-rules: fix nvme symlink creation on namespace changes (LP: #2028180)
    File: debian/patches/lp2028180-udev-rules-fix-nvme-symlink-creation-on-namespace-changes.patch
    https://git.launchpad.net/~ubuntu-core-dev/ubuntu/+source/systemd/commit/?id=26e85b944da9098e66fc0c39f64ee40254c0c278
  * core: reorder systemd arguments on reexec (LP: #2013543)
    File: debian/patches/lp2013543-core-reorder-systemd-arguments-on-reexec.patch
    https://git.launchpad.net/~ubuntu-core-dev/ubuntu/+source/systemd/commit/?id=19ba0f20d311642596dc65fa5d6eb96a2d4be280
  * network/dhcp4: do not ignore the gateway even if the destination is in same network (LP: #2009743)
    Files:
    - debian/patches/lp2009743/network-dhcp4-do-not-ignore-the-gateway-even-if-the-desti.patch
    - debian/patches/lp2009743/test-network-add-one-more-testcase-for-DHCPv4-classless-r.patch
    https://git.launchpad.net/~ubuntu-core-dev/ubuntu/+source/systemd/commit/?id=73e774a2fb99c82df6d0edd770bb84ab735ec2f0
  * Drop debian/UBUNTU-wait-online-exit-if-no-links-are-managed.patch (LP: #1982218)
    File: debian/patches/debian/UBUNTU-wait-online-exit-if-no-links-are-managed.patch
    https://git.launchpad.net/~ubuntu-core-dev/ubuntu/+source/systemd/commit/?id=cf82f08feea456e1c65895b34bffa8c33d421588
  * debian/systemd.postint: do not daemon re-exec if we could hit LP: #2013543
    File: debian/systemd.postinst
    https://git.launchpad.net/~ubuntu-core-dev/ubuntu/+source/systemd/commit/?id=be484dab06d590b1792a8f016f4292373d0174b7
  * binfmt: fix systemd-binfmt in LXD containers (LP: #1999275)
    Files:
    - debian/patches/lp1999275/binfmt-check-if-binfmt-is-mounted-before-applying-rules.patch
    - debian/patches/lp1999275/binfmt-util-also-check-if-binfmt-is-mounted-in-read-write.patch
    - debian/patches/lp1999275/binfmt-util-split-out-binfmt_mounted.patch
    - debian/patches/lp1999275/unit-check-more-specific-path-to-be-written-by-systemd-bi.patch
    https://git.launchpad.net/~ubuntu-core-dev/ubuntu/+source/systemd/commit/?id=0b9eadf05ee6db0e009dacbc71521480095880b3

  [ Dan Streetman ]
  * Fix machinectl pull-tar and import-tar (LP: #1977630)
    Author: Dan Streetman
    File: debian/patches/lp1977630-fix_machinectl_pull_tar.patch
    https://git.launchpad.net/~ubuntu-core-dev/ubuntu/+sourc...

Somehow, wait-online now times out, while it didn't before this update.

OS: Ubuntu 23.04 aarch64
Host: Raspberry Pi 4 Model B Rev 1.2
Kernel: 6.2.0-1012-raspi

Both ethernet and wi-fi are configured through netplan, and work as expected.

> Somehow, wait-online now times out, while it didn't before this update.

I just created https://bugs.launchpad.net/ubuntu/+source/systemd/+bug/2036358 to track this.

> Both ethernet and wi-fi are configured through netplan, and work as expected.

Can you clarify which backend you're using in netplan? Are these configured to use systemd-networkd, or NetworkManager?

I too got affected by this regression.
On servers with network-manager, without netplan (01-network-manager-all.yaml), systemd-networkd-wait-online.service started failing after the latest updates and leaves systemd in a degraded state.

Example:

root@gldap:/var/log/apt# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: ens18: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
    link/ether 52:54:00:10:20:64 brd ff:ff:ff:ff:ff:ff
    altname enp0s18
    inet 10.16.32.100/21 brd 10.16.39.255 scope global noprefixroute ens18
       valid_lft forever preferred_lft forever
    inet6 fe80::843e:3702:1c3b:2854/64 scope link noprefixroute
       valid_lft forever preferred_lft forever

root@gldap:~# networkctl
IDX LINK TYPE OPERATIONAL SETUP
  1 lo loopback carrier unmanaged
  2 ens18 ether routable unmanaged

2 links listed.
root@gldap:~# /lib/systemd/systemd-networkd-wait-online --any --timeout=5
Timeout occurred while waiting for network connectivity.