subiquity generated storage config fails validation when ventoy is used

Bug #2012722 reported by Olivier Gayot
184
This bug affects 15 people
Affects Status Importance Assigned to Milestone
curtin
Fix Released
Undecided
Olivier Gayot
subiquity
Fix Released
Undecided
Olivier Gayot
ubuntu-desktop-installer
Fix Released
Undecided
Unassigned

Bug Description

Based on recent reports, there seems to be an issue when loading the desktop installer from a ventoy image. The server installer seems to be unaffected. The storage configuration based on the probert output is invalid because we seem to lack support for /dev/mapper/ventoy partitions.

Workaround: flashing the installer directly onto a USB stick (or other media) would be enough to work-around the issue but this needs to be confirmed.

2023-03-24 06:05:17,814 ERROR block-discover:847 block probing failed restricted=False
Traceback (most recent call last):
  File "/snap/ubuntu-desktop-installer/880/bin/subiquity/subiquity/server/controllers/filesystem.py", line 841, in _probe
    await asyncio.wait_for(self._probe_once_task.task, 90.0)
  File "/snap/ubuntu-desktop-installer/880/usr/lib/python3.10/asyncio/tasks.py", line 445, in wait_for
    return fut.result()
  File "/snap/ubuntu-desktop-installer/880/bin/subiquity/subiquitycore/context.py", line 148, in decorated_async
    return await meth(self, **kw)
  File "/snap/ubuntu-desktop-installer/880/bin/subiquity/subiquity/server/controllers/filesystem.py", line 825, in _probe_once
    self.model.load_probe_data(storage)
  File "/snap/ubuntu-desktop-installer/880/bin/subiquity/subiquity/models/filesystem.py", line 1542, in load_probe_data
    self.reset()
  File "/snap/ubuntu-desktop-installer/880/bin/subiquity/subiquity/models/filesystem.py", line 1125, in reset
    self._orig_config = storage_config.extract_storage_config(
  File "/snap/ubuntu-desktop-installer/880/lib/python3.10/site-packages/curtin/storage_config.py", line 1353, in extract_storage_config
    tree = get_config_tree(cfg.get('id'), final_config)
  File "/snap/ubuntu-desktop-installer/880/lib/python3.10/site-packages/curtin/storage_config.py", line 306, in get_config_tree
    for dep in find_item_dependencies(item, sconfig):
  File "/snap/ubuntu-desktop-installer/880/lib/python3.10/site-packages/curtin/storage_config.py", line 276, in find_item_dependencies
    _validate_dep_type(item_id, dep_key, dep, config)
  File "/snap/ubuntu-desktop-installer/880/lib/python3.10/site-packages/curtin/storage_config.py", line 223, in _validate_dep_type
    raise ValueError(
ValueError: Invalid dep_id (format-disk-dm-0) not in storage config

Here's a redacted excerpt of the probe-data that involves ventoy and causes the failed installed.

        "/dev/dm-0": {
            "CURRENT_TAGS": ":systemd:",
            "DEVLINKS": "/dev/mapper/ventoy",
            "DEVNAME": "/dev/dm-0",
            "DEVPATH": "/devices/virtual/block/dm-0",
            "DEVTYPE": "disk",
            "DM_NAME": "ventoy",
            },
            "partitiontable": {
                "label": "gpt",
                "device": "/dev/dm-0",
                "partitions": [
                    {
                        "node": "/dev/mapper/ventoy-part1",
                    },
                    {
                        "node": "/dev/mapper/ventoy-part2",
                    },
                    {
                        "node": "/dev/mapper/ventoy-part3",
                    }
                ]
            }

Tags: fr-3786

Related branches

Revision history for this message
Olivier Gayot (ogayot) wrote (last edit ):

My attempts to reproduce the issue have some far been unsuccessful. I generated a ventoy image using ventoy-1.0.89 from Linux and tried the two following:

* Ventoy2Disk.sh -i /dev/sda -g <--- (gpt partition table)
* Ventoy2Disk.sh -i /dev/sda <--- (dos partition table)

I suspect that building the ventoy image from Windows might have a different result.

I would appreciate if someone affected could send us or detailed test-case (or maybe a `dd` of the media?) to reproduce the issue.

Olivier Gayot (ogayot)
description: updated
summary: - generated storage config fails validation when ventoy is used
+ subiquity generated storage config fails validation when ventoy is used
Revision history for this message
Dominik S (dominiktoja) wrote :

In reply to bug: 2012810

I used the official ventoy-1.0.89-linux.tar.gz. I see that there is an update (1.0.90). I will check if this error still occurs. If so, I will check using belenaether

Revision history for this message
Olivier Gayot (ogayot) wrote :

Thanks @dominiktoja! I've finally been able to reproduce the crash after flashing ventoy-1.0.89-linux. For some reason, only the desktop installer is affected, although the probing code is theoretically the same for the server installer. Something seems different in the layout of the block devices when using the desktop installer. I'm investigating.

Changed in subiquity:
status: New → In Progress
assignee: nobody → Olivier Gayot (ogayot)
tags: added: fr-3786
Revision history for this message
Michael Hudson-Doyle (mwhudson) wrote :

It might be that the server installer live installer environment has some relevant packages and thus udev rules that mean the state of the block devices are better represented in the udev database? Not sure which package that would be though. lvm2 maybe?

Olivier Gayot (ogayot)
description: updated
description: updated
Revision history for this message
Olivier Gayot (ogayot) wrote :

I did a quick comparison between desktop and server installers.

Desktop: https://pastebin.ubuntu.com/p/ZFrFRjj27Q/
Server: https://pastebin.ubuntu.com/p/XGkxMxd6kv/

Running `udevadm trigger` in the ubuntu server live installer makes the differences go away and essentially makes the server installer crash when reaching the storage configuration.

Revision history for this message
Dominik S (dominiktoja) wrote :

I checked after updating ventoy to 1.0.90 and the problem still occurs. I then used belenaetcher and there this problem does not appear.

I would also add that the bug appeared to me right at the very beginning of the ISO launch i.e. when selecting the language. A little strange because I use ventoy for many different ISOs (8 different versions of linux and 2 versions of windows) and never had any problems.

Olivier Gayot (ogayot)
Changed in curtin:
status: New → In Progress
assignee: nobody → Olivier Gayot (ogayot)
Revision history for this message
Olivier Gayot (ogayot) wrote :

Draft PR in subiquity (requires the fix in curtin first): https://github.com/canonical/subiquity/pull/1621

The same change will need to be propagated in the desktop installer too.

Olivier Gayot (ogayot)
Changed in curtin:
status: In Progress → Fix Committed
Olivier Gayot (ogayot)
Changed in subiquity:
status: In Progress → Fix Committed
Revision history for this message
Dan Bungert (dbungert) wrote :

We believe this issue has been resolved in Subiquity 23.04.2.

If you had tested this with a pre-final version of Ubuntu 23.04, it's
recommended to download the final install media.

For testing with Ubuntu Server 22.04.x or 20.04.x, when running
Subiquity, you should offered a new version of the installer. Please
take that update to version 23.04.2 or later to get the fix.

If this is still a problem for you, please make a comment and set the state
back to New. Thank you for the bug report.

Changed in ubuntu-desktop-installer:
status: New → Fix Released
Changed in subiquity:
status: Fix Committed → Fix Released
Changed in curtin:
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.