new libseccomp 2.4 (in proposed) makes rebuilds need but not generate a dependency to 2.4

Task for now is:
- Critical (for the chance of breaking a potentially arbitrary amount of SRUs that were built against it in proposed)
- Assigned to ubuntu-security as they are driving the libseccomp update

We need to understand it more to then re-triage severity and potential actions.

Until a simpler testcase is found, the qemu in PPA [1] built for Disco will trigger it.
=> https://launchpad.net/~ci-train-ppa-service/+archive/ubuntu/3729/+packages

To be clear - this will likely resolve over time when people upgrade to seccomp 2.4.
But right now SRUs could release things built in -proposed against it and then end up broken until 2.4 is released as well.
Furthermore without a strict dependency it might stay broken as nothing enforces 2.4 to be installed.

I have set up two new PPAs
https://launchpad.net/~paelzer/+archive/ubuntu/bug-1830859-with-proposed-seccomp
https://launchpad.net/~paelzer/+archive/ubuntu/bug-1830859-without-proposed-seccomp

They both build amd64 qemu with/without proposed for Disco.
Including debug symbols for further debugging.

This decouples this from my other PPA to allow me going on there with a temporary workaround.

Two Disco containers (one per PPA)

$ sudo add-apt-repository ppa:paelzer/bug-1830859-with-proposed-seccomp
or
$ sudo add-apt-repository ppa:paelzer/bug-1830859-without-proposed-seccomp
# add main/debug and enable the deb-src in the PPA sources

$ sudo apt install gdb qemu-system-x86 qemu-system-x86-dbgsym dpkg-dev
$ apt source qemu
$ cd qemu-3.1+dfsg/

break on qemu_seccomp

Hrm ??
equal qemu source
Good:
Breakpoint 1 at 0x4d63f4: file ./qemu-seccomp.c, line 293.
Bad:
Breakpoint 1 at 0x4d63f4: qemu_seccomp. (2 location
(gdb) info b
Num Type Disp Enb Address What
1 breakpoint keep y <MULTIPLE>
1.1 y 0x00000000004d63f4 in qemu_seccomp at ./qemu-seccomp.c:293
1.2 y 0x00000000004d656e in qemu_seccomp at ./qemu-seccomp.c:244

Now the second hit isn't an actual qemu_seccomp call, but it is very close to the error that we get reported.

(gdb) l qemu-seccomp.c:293
288
289 #if defined(SECCOMP_FILTER_FLAG_TSYNC)
290 int check;
291
292 /* check host TSYNC capability, it returns errno == ENOSYS if unavailable */
293 check = qemu_seccomp(SECCOMP_SET_MODE_FILTER,
294 SECCOMP_FILTER_FLAG_TSYNC, NULL);
295 if (check < 0 && errno == EFAULT) {
296 add = true;
297 }
(gdb) l qemu-seccomp.c:244
239 error_setg(errp, "invalid argument for resourcecontrol");
240 return -1;
241 }
242 }
243
244 if (seccomp_start(seccomp_opts) < 0) {
245 error_setg(errp, "failed to install seccomp syscall filter "
246 "in the kernel");
247 return -1;
248 }

Well maybe with seccomp 2.4 development headers something changed and this now gets inlined?
That is not an error, but suspicious.

qemu_seccomp should ALWAYS be inlined

static inline __attribute__((unused)) int
qemu_seccomp(unsigned int operation, unsigned int flags, void *args)

The one at line 293 is at seccomp_register and depends on #if defined(SECCOMP_FILTER_FLAG_TSYNC).
This one is active in both cases.

But the one at line 244 is part of qemu_seccomp_get_kill_action
That has some checks as well being:
#if defined(SECCOMP_GET_ACTION_AVAIL) && \
    defined(SCMP_ACT_KILL_PROCESS) && \
    defined(SECCOMP_RET_KILL_PROCESS)

There is no other way to "avoid" the path of
  parse_sandbox -> seccomp_start -> qemu_seccomp_get_kill_action -> qemu_seccomp
in terms of precompiler.
Both have CONFIG_SECCOMP set, so it might really come down to the three checks above.

Reading the commit [1] that added this to qemu seems to match the current theories.

Lets see what happens when it tries to use things buildt with 2.4 but without 2.4 being installed at runtime.

The returned value for action is matching expected /usr/include/seccomp.h numbers:
- good case => 196608 ==> SCMP_ACT_TRAP = 0x00030000U
- bad case => 2147483648 ==> SCMP_ACT_KILL_PROCESS = 0x80000000U

As a summary:
- the libseccomp-dev 2.4 headers being installed at build time
- code can detect now the availa...

[13:09] <mdeslaur> cpaelzer: oh, we knew about that one :(
[13:09] <cpaelzer> well
[13:09] <cpaelzer> then had you a plan how we should resolve that
[13:10] <cpaelzer> either in general (magic great code I haven't thought of) or in particular (how I should resolve it for qemu) ?
[13:10] <cpaelzer> and good morning mdeslaur sorry to start with such issues :-/
[13:10] <mdeslaur> cpaelzer: good morning! :)
[13:11] <-- lborda-eod (lborda@173-246-24-26.qc.cable.ebox.net) has left this server (Quit: Ex-Chat).
[13:11] <mdeslaur> cpaelzer: the plan is to publish the libseccomp update ;)
[13:11] --> lborda (lborda@173-246-24-26.qc.cable.ebox.net) has joined this channel.
[13:11] <mdeslaur> cpaelzer: we searched the archive, and qemu was the only problematic package that used those new apis
[13:12] <mdeslaur> jdstrand: are we ready to publish the seccomp updates?
[13:12] <cpaelzer> mdeslaur: ok then should I add an explicit dependency for Disco
[13:12] <cpaelzer> only there the "transition" might occur
[13:13] <cpaelzer> that we rebuild with the new version around but the old might be installed
[13:13] <jjohansen> hey mdeslaur
[13:13] <cpaelzer> I'm prepping an SRU anyway and that would avoid the "rebuilt to 2.4 but not installed"
[13:13] <cpaelzer> if qemu really is the only one I'm ok to carry this
[13:13] <mdeslaur> cpaelzer: yeah, you can add an explicit dependency, or you can wait until we publish seccomp...we may publish it today or monday
[13:13] <cpaelzer> and in Eoan things are fine as that will release with >=2.4
[13:14] <cpaelzer> mdeslaur: well, my point is even if you publish it there are no guarantees
[13:14] <cpaelzer> people might update qemu but not libseccomp
[13:14] <cpaelzer> and they would be screwed
[13:14] <cpaelzer> so the first rebuild of qemu in disco (=now) will need to add the dependency
[13:14] <cpaelzer> mdeslaur: ^^
[13:15] <mdeslaur> yes, selective installing of security updates is an untested and probably broken configuration
[13:15] <mdeslaur> if you add the explicit dependency, you'll need to wait until we do publish seccomp too
[13:15] <cpaelzer> yes, but I'll need a few days anyway
[13:15] <cpaelzer> and you seem close
[13:16] <mdeslaur> ok
[13:16] <cpaelzer> and for testing interim wise I can throw a no change rebuild of libseccomp 2.4 in my ppa
[13:16] <mdeslaur> I'll try and remember to add the explicit dependency to the next round of qemu updates
[13:16] <cpaelzer> Hmm, you have some in the pipe as well?
[13:16] <mdeslaur> I don't currently, no
[13:16] <cpaelzer> I have
[13:16] <cpaelzer> I'll add them right away
[13:17] <cpaelzer> if testing goes well those should hit -unapproved early next week
[13:17] <mdeslaur> sorry our timezones don't overlap and you had to spend time debugging this
[13:17] <mdeslaur> we spotted this issue when we prepared the emergency qemu update last week

TL;DR
we will make is a qemu fix in Disco, by adding an explicit dependency.
All others don't need the change.

FYI, libseccomp is now published: https://usn.ubuntu.com/4001-1/

Thanks Jamie,
as discussed an explicit dependency as fix will be part of the next SRU upload to qemu to avoid anybody running into trouble due to this.

Tested on the PPA, working as expected (after moved to qemu-system-*packacges)
The reason to no more use qemu-system-common is that this only was a recommends which would not enforce the update.
Even qemu-system-data which is a depends would be wrong as people could install that without the actual binaries and they would not need the new libseccomp.

Lets take the bitter pill and add them exactly where needed.
I analyzed binaries - qemu-user and others don't need it.

Checked on the PPA:

root@d-wily:~# apt install libseccomp2=2.3.3-3ubuntu2
Reading package lists... Done
Building dependency tree
Reading state information... Done
[...]
The following packages will be REMOVED:
  qemu-kvm qemu-system-x86
The following packages will be DOWNGRADED:
  libseccomp2
0 upgraded, 0 newly installed, 1 downgraded, 2 to remove and 0 not upgraded.

... and similar.
So the dependency works and makes sure a recent libseccomp2 will have to go along and by that prevent the error.

Uploaded to Disco-unapproved.

Hello Christian, or anyone else affected,

Accepted qemu into disco-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/qemu/1:3.1+dfsg-2ubuntu3.2 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-disco to verification-done-disco. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-disco. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

On this rebuild not only did it pick up the new libseccomp for build
   libseccomp2 amd64 2.4.1-0ubuntu0.19.04.3
but also has the dependency to libseccomp2 (>= 2.4.1) to ensure it has to be upgraded.

Seen in e.g.: https://launchpadlibrarian.net/427212564/buildlog_ubuntu-disco-amd64.qemu_1%3A3.1+dfsg-2ubuntu3.2_BUILDING.txt.gz

In a container I tried to fail-install (keep old libseccomp) but the dependency makes sure this won't happen

Works fine, setting verified

This bug was fixed in the package qemu - 1:3.1+dfsg-2ubuntu3.2

---------------
qemu (1:3.1+dfsg-2ubuntu3.2) disco; urgency=medium

  * d/p/ubuntu/define-ubuntu-machine-types.patch: fix wily machine type being
    broken since 2.11 due to 2.3/2.4 version mismatch in its definition to
    fix migrations from old machines (LP: #1829868).
  * d/p/ubuntu/lp-1830704-s390x-cpumodel-ignore-csske-for-expansion.patch
    toleration for future machines (LP: #1830704
  * d/control-in, d/control: add versioned dependencies to libseccomp 2.4 as
    any rebuild against 2.4 as it is in proposed right now will otherwise
    crash (LP: #1830859).

 -- Christian Ehrhardt <email address hidden> Tue, 28 May 2019 10:52:47 +0200

The verification of the Stable Release Update for qemu has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

As Eoan releases with >=2.4 this isn't a problem for Eoan, closing the task.