autopkgtest: upstream tests that run in qemu hang on ppc64el

This bug was fixed in the package systemd - 251.4-1ubuntu7

---------------
systemd (251.4-1ubuntu7) kinetic; urgency=medium

  [ Nick Rosbrook ]
  * autopkgtest: do not allow qemu to be used on ppc64el (LP: #1992178)
    Almost every run on ppc64el takes 12 to 24 hours, so do this as a last
    resort to relieve pressure on autopkgtest infrastructure.
    File: debian/tests/upstream
    https://git.launchpad.net/~ubuntu-core-dev/ubuntu/+source/systemd/commit/?id=5ab2b8ad13a3d7441935d6f43bb81fbd540e2c33

  [ Dan Streetman ]
  * make machinectl read-only work (LP: #1991829)
    Author: Dan Streetman
    File: debian/patches/lp1991829-add-CAP_LINUX_IMMUTABLE-to-systemd-machined-so-it-ca.patch
    https://git.launchpad.net/~ubuntu-core-dev/ubuntu/+source/systemd/commit/?id=a8a41b11d712c8cfbf595d34f39d95096f23e202

 -- Nick Rosbrook <email address hidden> Tue, 11 Oct 2022 11:51:25 -0400

Upstream has encountered this issue on their CI as well: https://github.com/systemd/systemd/issues/25091.

There have been several updates on the upstream bug, so I have opened a task for jammy.

We are suspecting issues in the ppc64el toolchain. The problem is only reproducible on Ubuntu ppc, and not on other distributions. The backtrace show some borken pointers that can't be explained. The upstream issue has all the details. None of us speak ppc assembly so we can't really check the binaries.

------- Comment From <email address hidden> 2023-02-14 17:28 EDT-------
Howdy. Adding developers from KVM on Power and Toolchain to our bug but it would be good to understand exactly what IBM needs to look at. For example, you mention a backtrace you see but I don't see that in the bug. Any details you can provide are appreciated.

Backtraces are on the upstream bug, eg:

https://github.com/systemd/systemd/issues/25091#issuecomment-1401824195
https://github.com/systemd/systemd/issues/25091#issuecomment-1402425787

It would be better if the IBM developers could engage directly on the upstream ticket for the initial investigation, given the information is recorded there. Thanks!

------- Comment From <email address hidden> 2023-06-06 01:35 EDT-------
(In reply to comment #6)
> Backtraces are on the upstream bug, eg:
>
> https://github.com/systemd/systemd/issues/25091#issuecomment-1401824195
> https://github.com/systemd/systemd/issues/25091#issuecomment-1402425787
>
> It would be better if the IBM developers could engage directly on the
> upstream ticket for the initial investigation, given the information is
> recorded there. Thanks!

Peter, Could you check this upstream bug and confirm if this needs to be looked by tools chain team as claimed by Ubuntu distro team?

Hi, did any PPC expert have a chance to look at this issue? We keep seeing it when running systemd ppc64 tests in qemu on Ubuntu jammy (both host and guest, same version)

------- Comment From <email address hidden> 2023-08-23 15:16 EDT-------
(In reply to comment #8)
> Hi, did any PPC expert have a chance to look at this issue? We keep seeing
> it when running systemd ppc64 tests in qemu on Ubuntu jammy (both host and
> guest, same version)

I commented in the upstream github issue. What I said is that it doesn't seem like a compiler issue to me, since the problem only seems to be exposed when running inside autopkgtest and not when running outside of autopkgtest, even when using the same compiler options. It seems more the memory corruption to me given the random failure locations and memory corruption can be caused by just about anything.

Thanks, could it be a problem in the jammy ppc64 qemu then? We never saw this memory corruption outside of qemu

------- Comment From <email address hidden> 2023-11-20 03:40 EDT-------
Harsh, This issue/bug is reported on Ubuntu 22.04 Distro and hence we cannot track behavior reported in other Distro like fedora though internally Development can check those behaviour.

I will ask Jamie if he can pass the machine recreate information and steps on Ubuntu distro.
.