Bionic/Xenial minimal cloud image: failed to apply load kernel module

Bug #1888000 reported by Joshua Powers
10
This bug affects 2 people
Affects Status Importance Assigned to Milestone
linux (Ubuntu)
Confirmed
Undecided
Unassigned
Xenial
Won't Fix
Undecided
Unassigned
Bionic
New
Undecided
Unassigned
open-iscsi (Ubuntu)
Fix Released
Undecided
Unassigned
Xenial
Won't Fix
Undecided
Unassigned
Bionic
Won't Fix
Undecided
Unassigned

Bug Description

Overview
---
It appears the Bionic and Xenial minimal images have always shipped with a failing systemd unit: "systemd-modules-load.service" with the error "Failed to find module 'ib_iser'"

Expected results
---
$ sudo systemctl list-units --failed --no-legend
# Return no failed units

Actual results
---
$ sudo systemctl list-units --failed --no-legend
systemd-sysctl.service loaded failed failed Apply Kernel Variables
# cloud-config.service may show up failing to setup the timezone...

From the journal:
Jul 17 17:48:27 ubuntu systemd-modules-load[98]: Module 'iscsi_tcp' is builtin
Jul 17 17:48:27 ubuntu systemd-modules-load[98]: Failed to find module 'ib_iser'
Jul 17 17:48:27 ubuntu systemd-sysctl[106]: Couldn't write '176' to 'kernel/sysrq', ignoring: No such file or directory
Jul 17 17:48:27 ubuntu systemd-sysctl[106]: Couldn't write 'fq_codel' to 'net/core/default_qdisc', ignoring: No such file or directory

Steps to reproduce
---
$ wget https://cloud-images.ubuntu.com/daily/server/minimal/releases/bionic/release/ubuntu-18.04-minimal-cloudimg-amd64.img
$ multipass launch file:///$(pwd)/ubuntu-18.04-minimal-cloudimg-amd64.img --name=bionic-minimal
$ multipass exec bionic-minimal -- sudo systemctl list-units --failed --no-legend

---
ProblemType: Bug
AlsaDevices: Error: command ['ls', '-l', '/dev/snd/'] failed with exit code 2: ls: cannot access '/dev/snd/': No such file or directory
AplayDevices: Error: [Errno 2] No such file or directory: 'aplay': 'aplay'
ApportVersion: 2.20.9-0ubuntu7.15
Architecture: amd64
ArecordDevices: Error: [Errno 2] No such file or directory: 'arecord': 'arecord'
AudioDevicesInUse: Error: [Errno 2] No such file or directory: 'fuser': 'fuser'
CRDA: N/A
DistroRelease: Ubuntu 18.04
IwConfig: Error: [Errno 2] No such file or directory: 'iwconfig': 'iwconfig'
Lspci: Error: [Errno 2] No such file or directory: 'lspci': 'lspci'
Lsusb: Error: [Errno 2] No such file or directory: 'lsusb': 'lsusb'
MachineType: QEMU Standard PC (i440FX + PIIX, 1996)
Package: linux (not installed)
PciMultimedia:

ProcEnviron:
 TERM=xterm
 PATH=(custom, no user)
 XDG_RUNTIME_DIR=<set>
 LANG=C.UTF-8
 SHELL=/bin/bash
ProcFB: Error: [Errno 2] No such file or directory: '/proc/fb'
ProcKernelCmdLine: BOOT_IMAGE=/boot/vmlinuz-4.15.0-1069-kvm root=PARTUUID=ffb7214d-c783-45ed-b736-278d4ee0cac0 ro console=tty1 console=ttyS0
ProcVersionSignature: User Name 4.15.0-1069.70-kvm 4.15.18
RelatedPackageVersions:
 linux-restricted-modules-4.15.0-1069-kvm N/A
 linux-backports-modules-4.15.0-1069-kvm N/A
 linux-firmware N/A
RfKill: Error: [Errno 2] No such file or directory: 'rfkill': 'rfkill'
Tags: bionic uec-images
Uname: Linux 4.15.0-1069-kvm x86_64
UpgradeStatus: No upgrade log present (probably fresh install)
UserGroups: adm audio cdrom dialout dip floppy lxd netdev plugdev sudo video
_MarkForUpload: True
dmi.bios.date: 04/01/2014
dmi.bios.vendor: SeaBIOS
dmi.bios.version: 1.10.2-1ubuntu1
dmi.chassis.type: 1
dmi.chassis.vendor: QEMU
dmi.chassis.version: pc-i440fx-bionic
dmi.modalias: dmi:bvnSeaBIOS:bvr1.10.2-1ubuntu1:bd04/01/2014:svnQEMU:pnStandardPC(i440FX+PIIX,1996):pvrpc-i440fx-bionic:cvnQEMU:ct1:cvrpc-i440fx-bionic:
dmi.product.name: Standard PC (i440FX + PIIX, 1996)
dmi.product.version: pc-i440fx-bionic
dmi.sys.vendor: QEMU

Revision history for this message
Joshua Powers (powersj) wrote : CurrentDmesg.txt

apport information

tags: added: apport-collected bionic uec-images
description: updated
Revision history for this message
Joshua Powers (powersj) wrote : ProcCpuinfo.txt

apport information

Revision history for this message
Joshua Powers (powersj) wrote : ProcCpuinfoMinimal.txt

apport information

Revision history for this message
Joshua Powers (powersj) wrote : ProcInterrupts.txt

apport information

Revision history for this message
Joshua Powers (powersj) wrote : ProcModules.txt

apport information

Revision history for this message
Joshua Powers (powersj) wrote : UdevDb.txt

apport information

Revision history for this message
Joshua Powers (powersj) wrote : WifiSyslog.txt

apport information

Revision history for this message
Ubuntu Kernel Bot (ubuntu-kernel-bot) wrote : Missing required logs.

This bug is missing log files that will aid in diagnosing the problem. While running an Ubuntu kernel (not a mainline or third-party kernel) please enter the following command in a terminal window:

apport-collect 1888000

and then change the status of the bug to 'Confirmed'.

If, due to the nature of the issue you have encountered, you are unable to run this command, please add a comment stating that fact and change the bug status to 'Confirmed'.

This change has been made by an automated script, maintained by the Ubuntu Kernel Team.

Changed in linux (Ubuntu):
status: New → Incomplete
Revision history for this message
Joshua Powers (powersj) wrote :

Apport info was collected and attached... moving to confirmed.

Changed in linux (Ubuntu):
status: Incomplete → Confirmed
Revision history for this message
Andy Whitcroft (apw) wrote :

This might be the same problem as reported and fix in this earlier but: https://bugs.launchpad.net/ubuntu/+source/open-iscsi/+bug/1566468

Revision history for this message
Andy Whitcroft (apw) wrote :

Could someone check the disposition of ib_iser.ko in either linux-modules or linux-modules-extra.

Revision history for this message
Khaled El Mously (kmously) wrote :

From linux-modules-4.15.0-112-generic_4.15.0-112.113_amd64.deb :

drwxr-xr-x root/root 0 2020-07-09 19:13 ./lib/modules/4.15.0-112-generic/kernel/drivers/infiniband/ulp/
drwxr-xr-x root/root 0 2020-07-09 19:13 ./lib/modules/4.15.0-112-generic/kernel/drivers/infiniband/ulp/iser/
-rw-r--r-- root/root 75089 2020-07-09 19:13 ./lib/modules/4.15.0-112-generic/kernel/drivers/infiniband/ulp/iser/ib_iser.ko
drwxr-xr-x root/root 0 2020-07-09 19:13 ./lib/modules/4.15.0-112-generic/kernel/drivers/infiniband/ulp/isert/
-rw-r--r-- root/root 72401 2020-07-09 19:13 ./lib/modules/4.15.0-112-generic/kernel/drivers/infiniband/ulp/isert/ib_isert.ko

Revision history for this message
Khaled El Mously (kmously) wrote :

(Ignore comment #12, that was for the master kernel not -kvm)

To clarify: linux-modules-4.15.0-1069-kvm_4.15.0-1069.70_amd64.deb does *not* contain the ib_iser.ko module.

I don't think it was actually removed from that kernel recently though. Nothing has changed in that area as far as I know

Revision history for this message
David Krauser (davidkrauser) wrote :

This may not be a new problem. Running the test case against a bionic image from June shows similar behavior:

$ multipass launch 'https://cloud-images.ubuntu.com/minimal/releases/bionic/release-20200610/ubuntu-18.04-minimal-cloudimg-amd64.img'
Launched: main-gnu
$ multipass exec main-gnu -- sudo systemctl list-units --failed --no-legend
cloud-config.service loaded failed failed Apply the settings specified in cloud-config
systemd-modules-load.service loaded failed failed Load Kernel Modules
$ multipass exec main-gnu -- cat /etc/cloud/build.info
build_name: minimal
serial: 20200610

Revision history for this message
Terry Rudd (terrykrudd) wrote :

We have determined nothing has changed in those kernels from last cycle or this cycle. We think something changed in the CPC environment that might have triggered what we think might be an old bug. But we need to CPC help to identify the "last good kernel" and the "first bad kernel". Once we can isolate, we can have a more useful investigation

We have determined that ib_iser.ko is not in 4.15.0-1069-kvm but it wasn't there in the previous cycle's kernel and is not in the current proposed 4.15.0-1071.72 cycle either. I don't think we should try to make a change, re-spin, then ask the CPC team to test, especially since we never made the breaking-change in the first place (as far as we know at least). Once we know what changed we can make the right choice.

Hopefully this is a reasonable approach

Revision history for this message
Joshua Powers (powersj) wrote :

Thanks for the review, it turns out that this failure has always been present.

Going back to the Bionic 20180705 and Xenial 20180705, which were the first releases of minimal, also showed that the system "Failed to find module 'ib_iser'".

Is this something that could be fixed in order to prevent us from shipping with failed systemd units?

description: updated
Revision history for this message
Sultan Alsawaf (kerneltoast) wrote :

This is a bug in the open-iscsi package in xenial and bionic. That package installs this file: /lib/modules-load.d/open-iscsi.conf

Which contains the following:
iscsi_tcp
ib_iser

This causes systemd to always try and load the iscsi_tcp and ib_iser modules on boot. In eoan, the open-iscsi package doesn't do this; there's no /lib/modules-load.d/open-iscsi.conf file in the eoan cloud image. An update to xenial/bionic open-iscsi packages to match the behavior of eoan's open-iscsi package should resolve this I think.

Revision history for this message
Joshua Powers (powersj) wrote :

Based on the update above, #17, I am adding the open-iscsi package task.

Revision history for this message
Paride Legovini (paride) wrote :

I think this is basically a duplicate of LP: #1833586. SRUing the fix was discussed there but vorlon was not positive about it, see comment #6 there.

I'm marking the open-iscsi task as Triaged, but if stick with Steve's opinion then this is to be considered a Won't Fix.

Changed in open-iscsi (Ubuntu):
status: New → Fix Released
Changed in open-iscsi (Ubuntu Xenial):
status: New → Triaged
Changed in open-iscsi (Ubuntu Bionic):
status: New → Triaged
Revision history for this message
Sergio Durigan Junior (sergiodj) wrote :

Xenial has reached end of standard support.

Changed in open-iscsi (Ubuntu Xenial):
status: Triaged → Won't Fix
Changed in linux (Ubuntu Xenial):
status: New → Won't Fix
Revision history for this message
Christian Ehrhardt  (paelzer) wrote :

Hi,
I'm coming by cleaning old open bugs to ensure nothing gets forgotten for too long and or clutters the view to the remaining issues.

This case here - as Paride outlined almost two years ago - is already covered and discussed in bug 1833586 . We won't/can't SRU it and there was no further input/discussion here.

Setting Bionic to Won't Fix - and from latter releases it is already fine.

Changed in open-iscsi (Ubuntu Bionic):
status: Triaged → Won't Fix
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.