UEFI install onto an MBR disk, with a pre-existing os/partition, results in a non-bootable ESP on a logical partition

Bug #1796260 reported by Jean-Baptiste Lallement
8
This bug affects 1 person
Affects Status Importance Assigned to Milestone
efibootmgr (Ubuntu)
Invalid
Undecided
Unassigned
efivar (Ubuntu)
New
Low
Unassigned
grub2 (Ubuntu)
Invalid
Undecided
Unassigned
partman-auto (Ubuntu)
Fix Released
Undecided
Unassigned
partman-partitioning (Ubuntu)
Fix Released
Undecided
Unassigned
ubiquity (Ubuntu)
Fix Released
Low
Unassigned

Bug Description

Ubuntu Cosmic Desktop 20181005

Test Case:
1. Boot in BIOS mode and install on entire disk
=> Verify that the system boots
2. Boot in UEFI and perform a new installation alongside previous one
=> Verify that you can boot to the new system

Actual result
It fails immediately on boot with a message from the uefi firmware (its very brief because the system reboots immediately in a loop):

"""""
System BootOrder not found. Initializing defaults.
Creating boot entry "Boot0011" with label "Ubuntu" for file "\EFI\ubuntu\shimx64.efi"

Reset System
"""""

The BIOS installation boots fine if I switch back to BIOS mode.
An UEFI installation on entire disk boots fine.

Log files of the UEFI installation attached.

The EFI partition is created and its content is:
.
./EFI
./EFI/ubuntu
./EFI/ubuntu/grubx64.efi
./EFI/ubuntu/shimx64.efi
./EFI/ubuntu/mmx64.efi
./EFI/ubuntu/BOOTX64.CSV
./EFI/ubuntu/grub.cfg
./EFI/BOOT
./EFI/BOOT/BOOTX64.EFI
./EFI/BOOT/fbx64.efi

ProblemType: Bug
DistroRelease: Ubuntu 18.10
Package: ubiquity 18.10.9
ProcVersionSignature: Ubuntu 4.18.0-8.9-generic 4.18.7
Uname: Linux 4.18.0-8-generic x86_64
ApportVersion: 2.20.10-0ubuntu11
Architecture: amd64
CasperVersion: 1.396
CurrentDesktop: ubuntu:GNOME
Date: Fri Oct 5 09:40:31 2018
InstallCmdLine: BOOT_IMAGE=/casper/vmlinuz file=/cdrom/preseed/ubuntu.seed boot=casper quiet splash ---
LiveMediaBuild: Ubuntu 18.10 "Cosmic Cuttlefish" - Beta amd64 (20181005)
ProcEnviron:
 TERM=xterm-256color
 PATH=(custom, no user)
 XDG_RUNTIME_DIR=<set>
 LANG=C.UTF-8
 SHELL=/bin/bash
SourcePackage: ubiquity
UpgradeStatus: No upgrade log present (probably fresh install)

Revision history for this message
Jean-Baptiste Lallement (jibel) wrote :
description: updated
description: updated
Steve Langasek (vorlon)
description: updated
Revision history for this message
Mathieu Trudel-Lapierre (cyphermox) wrote :

Is this running on real hardware or in a virtual machine? If it's on metal, what kind of hardware?

Revision history for this message
Jean-Baptiste Lallement (jibel) wrote :

This is on bare metal, DELL Inspiron 13

Revision history for this message
Jean-Baptiste Lallement (jibel) wrote :

On this machine there is a boot setting to select the boot mode UEFI or Legacy

Revision history for this message
Steve Langasek (vorlon) wrote :

I have tried and failed to reproduce this in a VM. After installing the second Ubuntu in UEFI mode, I wind up without any settings persisting in the nvram store at all, so I just get dropped to the ovmf shell. That is a bug in my KVM setup, but it makes it rather difficult to reproduce the problem.

If you can reproduce this, are you able to boot the ISO again under UEFI mode? If you are able to, what is the output of 'efibootmgr -v' after booting the ISO?

Changed in grub2 (Ubuntu):
status: New → Incomplete
description: updated
summary: - UEFI installation alongside BIOS installation: System fails to boot with
- cannot find a valid boot entry
+ UEFI installation alongside BIOS installation: System BootOrder not
+ found
description: updated
description: updated
Revision history for this message
Jean-Baptiste Lallement (jibel) wrote : Re: UEFI installation alongside BIOS installation: System BootOrder not found

output of efibootmgr -v:

BootCurrent: 0017
Timeout: 0 seconds
BootOrder: 0002,0003,0004,0005,0006,0017,0018
Boot0000* ubuntu HD(5,MBR,0x39a7181f,0x0,0x0)/File(\EFI\ubuntu\shimx64.efi)
Boot0001* ubuntu HD(2,MBR,0x39a7181f,0xe6717fe,0xd8b6002)/HD(1,MBR,0x0,0xe671800,0x100800)/File(\EFI\ubuntu\shimx64.efi)
Boot0002* Diskette Drive BBS(Floppy,Diskette Drive,0x0)..BO
Boot0003* Internal HDD BBS(HD,P0: SanDisk SSD PLUS 240 GB ,0x0)..BO
Boot0004* USB Storage Device BBS(USB,Wilk PMAP,0x0)..BO
Boot0005* CD/DVD/CD-RW Drive BBS(CDROM,CD/DVD/CD-RW Drive,0x0)..BO
Boot0006* Onboard NIC BBS(Network,Onboard NIC,0x0)..BO
Boot0007* ubuntu HD(2,MBR,0x39a7181f,0xe6717fe,0xd8b6002)/HD(1,MBR,0x0,0xe671800,0x100800)/File(\EFI\ubuntu\shimx64.efi)
Boot0008* ubuntu HD(2,MBR,0x39a7181f,0xe6717fe,0xd8b6002)/HD(1,MBR,0x0,0xe671800,0x100800)/File(\EFI\ubuntu\shimx64.efi)
Boot0009* ubuntu HD(2,MBR,0x39a7181f,0xe6717fe,0xd8b6002)/HD(1,MBR,0x0,0xe671800,0x100800)/File(\EFI\ubuntu\shimx64.efi)
Boot000A* ubuntu HD(2,MBR,0x39a7181f,0xe6717fe,0xd8b6002)/HD(1,MBR,0x0,0xe671800,0x100800)/File(\EFI\ubuntu\shimx64.efi)
Boot000B* ubuntu HD(2,MBR,0x39a7181f,0xe6717fe,0xd8b6002)/HD(1,MBR,0x0,0xe671800,0x100800)/File(\EFI\ubuntu\shimx64.efi)
Boot000C* ubuntu HD(2,MBR,0x39a7181f,0xe6717fe,0xd8b6002)/HD(1,MBR,0x0,0xe671800,0x100800)/File(\EFI\ubuntu\shimx64.efi)
Boot000D* ubuntu HD(2,MBR,0x39a7181f,0xe6717fe,0xd8b6002)/HD(1,MBR,0x0,0xe671800,0x100800)/File(\EFI\ubuntu\shimx64.efi)
Boot000E* ubuntu HD(2,MBR,0x39a7181f,0xe6717fe,0xd8b6002)/HD(1,MBR,0x0,0xe671800,0x100800)/File(\EFI\ubuntu\shimx64.efi)
Boot000F* ubuntu HD(2,MBR,0x39a7181f,0xe6717fe,0xd8b6002)/HD(1,MBR,0x0,0xe671800,0x100800)/File(\EFI\ubuntu\shimx64.efi)
Boot0010* ubuntu HD(2,MBR,0x39a7181f,0xe6717fe,0xd8b6002)/HD(1,MBR,0x0,0xe671800,0x100800)/File(\EFI\ubuntu\shimx64.efi)
Boot0011* ubuntu HD(2,MBR,0x39a7181f,0xe6717fe,0xd8b6002)/HD(1,MBR,0x0,0xe671800,0x100800)/File(\EFI\ubuntu\shimx64.efi)
Boot0012* ubuntu HD(2,MBR,0x39a7181f,0xe6717fe,0xd8b6002)/HD(1,MBR,0x0,0xe671800,0x100800)/File(\EFI\ubuntu\shimx64.efi)
Boot0013* ubuntu HD(2,MBR,0x39a7181f,0xe6717fe,0xd8b6002)/HD(1,MBR,0x0,0xe671800,0x100800)/File(\EFI\ubuntu\shimx64.efi)
Boot0014* ubuntu HD(2,MBR,0x39a7181f,0xe6717fe,0xd8b6002)/HD(1,MBR,0x0,0xe671800,0x100800)/File(\EFI\ubuntu\shimx64.efi)
Boot0015* ubuntu HD(2,MBR,0x39a7181f,0xe6717fe,0xd8b6002)/HD(1,MBR,0x0,0xe671800,0x100800)/File(\EFI\ubuntu\shimx64.efi)
Boot0016* ubuntu HD(2,MBR,0x39a7181f,0xe6717fe,0xd8b6002)/HD(1,MBR,0x0,0xe671800,0x100800)/File(\EFI\ubuntu\shimx64.efi)
Boot0017* UEFI: Wilk PMAP, Partition 1 PciRoot(0x0)/Pci(0x14,0x0)/USB(12,0)/HD(1,MBR,0x336b2c37,0x3a538c,0x1340)..BO
Boot0018* UEFI: SanDisk SSD PLUS 240 GB, Partition 2 HD(2,MBR,0x39a7181f,0xe6717fe,0xd8b6002)/File(EFI\boot\bootx64.efi)..BO

Revision history for this message
Ubuntu QA Website (ubuntuqa) wrote :

This bug has been reported on the Ubuntu ISO testing tracker.

A list of all reports related to this bug can be found here:
http://iso.qa.ubuntu.com/qatracker/reports/bugs/1796260

tags: added: iso-testing
Revision history for this message
Dimitri John Ledkov (xnox) wrote :

@jibel

you really should clear all the boot entries, you may be overflowing.

@steve

it is quite easy to give an nvram to the VM, but i think we have a spare laptop here that can do legacy & uefi, we'll retest this there.

Revision history for this message
Dimitri John Ledkov (xnox) wrote :

with persistent uefi vars, it rebooted into uefi system fine.

I'd recommend clearing all of your boot entries....

Changed in grub2 (Ubuntu):
status: Incomplete → Invalid
Changed in ubiquity (Ubuntu):
status: New → Invalid
Revision history for this message
Jean-Baptiste Lallement (jibel) wrote :

@Dimitri, what do you mean by "Clearing all the boot entries" ? I didn't do anything but installing this machine with ubuntu.

Revision history for this message
Steve Langasek (vorlon) wrote :

Jean-Baptiste, your efibootmgr -v output shows no fewer than 18 boot entries for Ubuntu. That didn't happen as a result of a single install of Ubuntu (.... I hope). I think Dimitri's suggestion is that you zero these out, either with efibootmgr or from the firmware UI, to see if the system is less confused then and able to boot after install.

Like Dimitri, I do not see any evidence here of an Ubuntu bug, or if there is a bug, what that bug would be.

Revision history for this message
Dimitri John Ledkov (xnox) wrote :

So we have zeroed these out. And we have now done bios-mbr install, and then side-by-side install in uefi mode with mok util.

1) mok util did not register a `correct` esp boot path, it pointed at \efi\boot\mmx64.efi .... but that is a wrong path, as it should be at \efi\ubuntu\mmx64.efi.

2) the boot entry generated in the uefi side-by-side install on top of bios-mbr install has ESP on partition 5, ie. extended 1. and the ubuntu bootentry as seen in efibootmgr is not recognized by dell firmware. If i setup a manual entry by navigating the file-finder, it ends up with a quite different one:

BootCurrent: 0001
Timeout: 0 seconds
BootOrder: 0001
Boot0000* ubuntu HD(5,MBR,0x3ecb93d5,0x0,0x0)/File(\EFI\ubuntu\shimx64.efi)
Boot0001* xnox PciRoot(0x0)/Pci(0x17,0x0)/Sata(0,65535,0)/HD(2,MBR,0x3ecb93d5,0xe62bffe,0xd8fb802)/HD(1,MBR,0x0,0xe62c000,0x100800)/File(\EFI\ubuntu\shimx64.efi)

The xnox entry is bootable and regonizable in UEFI UI, but the ubuntu one is not. Note how the manual entry doesn't define parition 5, but instead uses HD(2,MBR...)/HD(1,MBR,....)/File syntax. Reading the spec, maybe we are specifying the extended MBR partition wrong.

Revision history for this message
Dimitri John Ledkov (xnox) wrote :

and when it doesn't like HD(5,MBR...) entry and has no other entries it tries to "autogenerate" the entry and injects it into bootorder.... and then boot loops.... and then the entries pile up

Revision history for this message
Dimitri John Ledkov (xnox) wrote :

So.... i don't think efibootmgr generates "right" entries for ESP on extended mbr partition.

or maybe it should learn to generate HD(2,MBR)/HD(1,MBR)....

And/or maybe we shouldn't put ESP on the extended partition if we can / have primary partitions available.

Changed in ubiquity (Ubuntu):
status: Invalid → New
summary: - UEFI installation alongside BIOS installation: System BootOrder not
- found
+ UEFI install onto an MBR disk, with a pre-existing os/partition, results
+ in a non-bootable ESP on a logical parition
summary: UEFI install onto an MBR disk, with a pre-existing os/partition, results
- in a non-bootable ESP on a logical parition
+ in a non-bootable ESP on a logical partition
Revision history for this message
Mathieu Trudel-Lapierre (cyphermox) wrote :

This is an efivar bug, that's what will generate file paths for efibootmgr, specifically in efi_va_generate_file_device_path_from_esp() and the code it calls.

Changed in efibootmgr (Ubuntu):
status: New → Invalid
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package partman-auto - 134ubuntu10

---------------
partman-auto (134ubuntu10) cosmic; urgency=medium

  * recipes-amd64-efi/*: enforce creating ESP on a primary partition, as
    efibootmgr fails to create correct (nested) logical MBR parition
    bootentries as per UEFI spec. LP: #1796260

 -- Dimitri John Ledkov <email address hidden> Tue, 16 Oct 2018 17:03:15 +0100

Changed in partman-auto (Ubuntu):
status: New → Fix Released
tags: added: id-5bc6279427553a601ca8de18
tags: added: rls-cc-notfixing
removed: rls-cc-incoming
tags: added: rls-ee-incoming
Changed in efivar (Ubuntu):
importance: Undecided → Low
Changed in ubiquity (Ubuntu):
importance: Undecided → Low
tags: added: rls-ee-notfixing
removed: rls-ee-incoming
Revision history for this message
Michael Hudson-Doyle (mwhudson) wrote :

So subiquity/curtin sort of tiptoe around this bug currently. Curtin's data model uses the same field ("flag") to indicate that a partition is an ESP (set flag to "boot") and that a partition is a logical partition (set flag to "logical").

As it happens in the code in curtin that converts probert data to curtin actions, "logicalness" wins over "ESPness", so the user will never be given the chance to reuse a logical ESP (and subiquity only ever creates GPT partitions).

To enable reuse of a logical ESP we'd need to change curtin somehow, maybe just to allow "boot+logical" as a flag value, or maybe something less crass. And then fix this bug. It seems somehow rather low priority though.

Revision history for this message
Dimitri John Ledkov (xnox) wrote :

well, maybe in bios mode we should install as gpt by default, on fresh disks.

Such that "1. Boot in BIOS mode and install on entire disk" results in GPT.

Changed in partman-partitioning (Ubuntu):
status: New → Fix Committed
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package partman-partitioning - 120ubuntu3

---------------
partman-partitioning (120ubuntu3) groovy; urgency=medium

  * Default to GPT on amd64 for any fresh disk installs, be it bios or EFI.
    LP: #1796260

 -- Dimitri John Ledkov <email address hidden> Thu, 28 May 2020 17:55:46 +0100

Changed in partman-partitioning (Ubuntu):
status: Fix Committed → Fix Released
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package ubiquity - 20.10.2

---------------
ubiquity (20.10.2) groovy; urgency=medium

  [ Jean-Baptiste Lallement ]
  * zsys-setup: Use persistent device name for vdevs (LP: #1880869)
  * Only export pools created during installation and containing dataset
    mounted under /target (LP: #1875045)

  [ Dimitri John Ledkov ]
  * Automatic update of included source packages: partman-partitioning
    120ubuntu3. LP: #1796260

 -- Dimitri John Ledkov <email address hidden> Thu, 28 May 2020 22:41:16 +0100

Changed in ubiquity (Ubuntu):
status: New → Fix Released
tags: added: fr-393
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.