classic snaps do not work on ARM64 kernel configured to use 64k pages

Bug #2002427 reported by Joao Andre Simioni
12
This bug affects 2 people
Affects Status Importance Assigned to Milestone
Snapcraft
Fix Committed
Undecided
Unassigned
subiquity
New
Undecided
Unassigned

Bug Description

patchelf (in the version bundled with snapcraft) creates segments that overlap when aligned to a page size greater than 4k, e.g. from readelf -h on the python3.8 interpreter in the current subiquity snap:

  LOAD 0x000000 0x00000000003ff000 0x00000000003ff000 0x001000 0x001000 RW 0x1000
  LOAD 0x001000 0x0000000000400000 0x0000000000400000 0x4bab44 0x4bab44 R E 0x1000

This makes the kernel very unhappy indeed and it refuses to load such a binary (execve returns EINVAL). Upstream patchelf fixed this in https://github.com/NixOS/patchelf/pull/216. I don't know if cherry picking that fix or just updating the version of patchelf bundled by snapcraft to the latest release is more appropriate.

[Problem Description]

Subiquity fails to execute when running on ARM64 with -64k Kernel. It exits with the "Segmentation fault" message

[Additional Info]

The problem seems to be with python3.8 binary in the snap. The same problem occurs with wget binary in the same snap, but ubuntu-distro-info works fine. Both python3.8 and wget are statically compiled, while ubuntu-distro-info is not.

root@jammy-arm:~# file /snap/subiquity/4236/usr/bin/python3.8
/snap/subiquity/4236/usr/bin/python3.8: ELF 64-bit LSB executable, ARM aarch64, version 1 (SYSV), dynamically linked, interpreter /snap/core20/current/lib/ld-linux-aarch64.so.1, for GNU/Linux 3.7.0, BuildID[sha1]=bad3f5d001ec1e2ec539f16d8f6729a06cdd68df, stripped
root@jammy-arm:~# /snap/subiquity/4236/usr/bin/python3.8
Segmentation fault
root@jammy-arm:~# ldd /snap/subiquity/4236/usr/bin/python3.8
 not a dynamic executable

root@jammy-arm:~# file /snap/subiquity/4236/usr/bin/wget
/snap/subiquity/4236/usr/bin/wget: ELF 64-bit LSB pie executable, ARM aarch64, version 1 (SYSV), dynamically linked, interpreter /snap/core20/current/lib/ld-linux-aarch64.so.1, for GNU/Linux 3.7.0, BuildID[sha1]=0f2234825d67c22b6b320139445759f6662aa01e, stripped
root@jammy-arm:~# /snap/subiquity/4236/usr/bin/wget
Segmentation fault
root@jammy-arm:~# ldd /snap/subiquity/4236/usr/bin/wget
 not a dynamic executable

root@jammy-arm:~# file /snap/subiquity/4236/usr/bin/ubuntu-distro-info
/snap/subiquity/4236/usr/bin/ubuntu-distro-info: ELF 64-bit LSB pie executable, ARM aarch64, version 1 (SYSV), dynamically linked, interpreter /snap/core20/current/lib/ld-linux-aarch64.so.1, for GNU/Linux 3.7.0, BuildID[sha1]=5d8c4c52d7ce614024eba1ba9069e48ad4192508, stripped
root@jammy-arm:~# /snap/subiquity/4236/usr/bin/ubuntu-distro-info
ubuntu-distro-info: You have to select exactly one of --all, --devel, --latest, --lts, --stable, --supported, --supported-esm, --series, --unsupported.
root@jammy-arm:~# ldd /snap/subiquity/4236/usr/bin/ubuntu-distro-info
 linux-vdso.so.1 (0x0000fffe72270000)
 libc.so.6 => /snap/core20/current/lib/aarch64-linux-gnu/libc.so.6 (0x0000fffe720b0000)
 /snap/core20/current/lib/ld-linux-aarch64.so.1 => /lib/ld-linux-aarch64.so.1 (0x0000fffe72280000)

The same VM, using the non -64k kernel works:

# uname -a
Linux jammy-arm 5.15.0-27-generic #28-Ubuntu SMP Thu Apr 14 12:56:31 UTC 2022 aarch64 aarch64 aarch64 GNU/Linux
# /snap/subiquity/4236/usr/bin/python3.8 -c "print('Works')"
Works

Tried other kernels (5.17, 5.19) with the same error.

[Reproducer]

1. Run a VM in ARM64 architecture:
virt-install --arch aarch64 --boot uefi --osinfo detect=on,require=off --name jammy-arm --memory 8096 --vcpus 4 --disk=jammy-server-cloudimg-arm64.img,bus=virtio --disk=jammy-arm-seed.qcow2,bus=virtio --network network=default,model=virtio --boot hd --noautoconsole

2. Connect to the VM and install a -64k kernel
https://launchpad.net/~canonical-kernel-team/+archive/ubuntu/bootstrap/+build/23546569

3. Reboot in the kernel (I disabled secure boot)
# uname -a
Linux jammy-arm 5.15.0-27-generic-64k #28-Ubuntu SMP Thu Apr 14 19:01:31 UTC 2022 aarch64 aarch64 aarch64 GNU/Linux

4. Install subiquity
sudo snap install subiquity

5. Try to run it
# /snap/bin/subiquity
Segmentation fault
---
ProblemType: Bug
AlsaDevices:
 total 0
 crw-rw---- 1 root audio 116, 1 Jan 10 18:24 seq
 crw-rw---- 1 root audio 116, 33 Jan 10 18:24 timer
AplayDevices: Error: [Errno 2] No such file or directory: 'aplay'
ApportVersion: 2.20.11-0ubuntu82.3
Architecture: arm64
ArecordDevices: Error: [Errno 2] No such file or directory: 'arecord'
AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq', '/dev/snd/timer'] failed with exit code 1:
CRDA: N/A
CasperMD5CheckResult: unknown
DistroRelease: Ubuntu 22.04
IwConfig: Error: [Errno 2] No such file or directory: 'iwconfig'
Lsusb:
 Bus 002 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
 Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Lsusb-t:
 /: Bus 02.Port 1: Dev 1, Class=root_hub, Driver=xhci_hcd/15p, 5000M
 /: Bus 01.Port 1: Dev 1, Class=root_hub, Driver=xhci_hcd/15p, 480M
MachineType: QEMU QEMU Virtual Machine
Package: linux (not installed)
PciMultimedia:

ProcEnviron:
 TERM=xterm-256color
 PATH=(custom, no user)
 LANG=C.UTF-8
 SHELL=/bin/bash
ProcFB:

ProcKernelCmdLine: BOOT_IMAGE=/boot/vmlinuz-5.15.0-27-generic-64k root=UUID=a6c92eea-e1ae-4ec4-9fbc-69e3ab11d0f7 ro console=tty1 console=ttyS0
ProcVersionSignature: Ubuntu 5.15.0-27.28-generic-64k 5.15.30
RelatedPackageVersions:
 linux-restricted-modules-5.15.0-27-generic-64k N/A
 linux-backports-modules-5.15.0-27-generic-64k N/A
 linux-firmware N/A
RfKill: Error: [Errno 2] No such file or directory: 'rfkill'
Tags: jammy uec-images
Uname: Linux 5.15.0-27-generic-64k aarch64
UpgradeStatus: No upgrade log present (probably fresh install)
UserGroups: N/A
_MarkForUpload: True
dmi.bios.date: 02/06/2015
dmi.bios.release: 0.0
dmi.bios.vendor: EFI Development Kit II / OVMF
dmi.bios.version: 0.0.0
dmi.chassis.type: 1
dmi.chassis.vendor: QEMU
dmi.chassis.version: virt-6.2
dmi.modalias: dmi:bvnEFIDevelopmentKitII/OVMF:bvr0.0.0:bd02/06/2015:br0.0:svnQEMU:pnQEMUVirtualMachine:pvrvirt-6.2:cvnQEMU:ct1:cvrvirt-6.2:sku:
dmi.product.name: QEMU Virtual Machine
dmi.product.version: virt-6.2
dmi.sys.vendor: QEMU

Revision history for this message
Ubuntu Kernel Bot (ubuntu-kernel-bot) wrote : Missing required logs.

This bug is missing log files that will aid in diagnosing the problem. While running an Ubuntu kernel (not a mainline or third-party kernel) please enter the following command in a terminal window:

apport-collect 2002427

and then change the status of the bug to 'Confirmed'.

If, due to the nature of the issue you have encountered, you are unable to run this command, please add a comment stating that fact and change the bug status to 'Confirmed'.

This change has been made by an automated script, maintained by the Ubuntu Kernel Team.

Changed in linux (Ubuntu):
status: New → Incomplete
Revision history for this message
Joao Andre Simioni (jasimioni) wrote : CurrentDmesg.txt

apport information

tags: added: apport-collected jammy uec-images
description: updated
Revision history for this message
Joao Andre Simioni (jasimioni) wrote : Lspci.txt

apport information

Revision history for this message
Joao Andre Simioni (jasimioni) wrote : Lspci-vt.txt

apport information

Revision history for this message
Joao Andre Simioni (jasimioni) wrote : Lsusb-v.txt

apport information

Revision history for this message
Joao Andre Simioni (jasimioni) wrote : ProcCpuinfo.txt

apport information

Revision history for this message
Joao Andre Simioni (jasimioni) wrote : ProcCpuinfoMinimal.txt

apport information

Revision history for this message
Joao Andre Simioni (jasimioni) wrote : ProcInterrupts.txt

apport information

Revision history for this message
Joao Andre Simioni (jasimioni) wrote : ProcModules.txt

apport information

Revision history for this message
Joao Andre Simioni (jasimioni) wrote : UdevDb.txt

apport information

Revision history for this message
Joao Andre Simioni (jasimioni) wrote : WifiSyslog.txt

apport information

Revision history for this message
Joao Andre Simioni (jasimioni) wrote : acpidump.txt

apport information

no longer affects: linux (Ubuntu)
summary: - Subiquity segfault in ARM64 with -64k Kernel
+ classic snaps do not work on ARM64 kernel configured to use 64k pages
description: updated
tags: added: craft-1546
Revision history for this message
Sergio Schvezov (sergiusens) wrote :

Can you try snapcraft itself, which is classic and also patchelf'ed with the patch you mention applied. It is on 7.x/beta/patchelf

Moving to a latter version of patchelf might prove to be more difficult as there seems to be segmentation faults that would need to be researched.

Revision history for this message
Tobias Heider (tobhe) wrote :

The 7.x/beta/patchelf beta version works on my 16k page size arm64 apple silicon kernel. The previous version crashed in the configure hook because of a 4k aligned python executable. Thank you for the quick fix!

Changed in snapcraft:
status: New → Fix Committed
Revision history for this message
dann frazier (dannf) wrote :

For the subiquity task here, do we just need to make sure a fixed version of snapcraft is being used to build it? If so, how do we do that?

Revision history for this message
Dan Bungert (dbungert) wrote :

The Subiquity beta/lp-2002427-patchelf-4k subiquity channel contains Subiquity 23.02.1 rebuilt with snapcraft from channel 7.x/beta/patchelf.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.